Hyperspectral image segmentation method based on fusion point prompt and markov diffusion

Abstract

The application discloses a hyperspectral image segmentation method and device based on fusion point prompting and Markov diffusion, and relates to the technical field of hyperspectral image processing. The method comprises the following steps: performing spectral-spatial dimension reduction processing according to initial hyperspectral data; performing rough segmentation guided by point prompting using a spectral-spatial feature adaptation module based on a preset uniform distribution point and according to spatial block characteristics; performing text-image feature fusion using a multi-modal fusion module based on a cross double attention mechanism; performing multi-scale feature extraction using a U-net encoder according to the initial hyperspectral data; performing diffusion reconstruction based on a symmetric encoder-decoder convolutional network of a Markov diffusion model, and extracting denoised hyperspectral features and high-order fusion masks; and performing model optimization according to segmentation prediction data based on a cross-entropy loss function. The application is a hyperspectral image segmentation method with high efficiency and strong robustness which fully considers the characteristics of hyperspectral images based on text semantic features.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral image processing technology, and in particular to a hyperspectral image segmentation method and apparatus based on fusion point cues and Markov diffusion. Background Technology

[0002] Hyperspectral imaging is a cutting-edge remote sensing technology that acquires high-resolution spectral information of objects, making each pixel correspond to a unique spectral curve. This provides rich information for identifying and distinguishing different materials and surfaces, allowing observation of information that the human eye and ordinary images cannot capture. In the field of land cover classification, the application potential of hyperspectral remote sensing is reflected in three dimensions: first, its ability for fine classification, distinguishing between different objects with the same spectrum and those with different spectra through spectral matching, such as different varieties of crops or mineral subspecies; second, its quantitative inversion advantages, such as directly retrieving physicochemical parameters like leaf area index and soil organic matter content based on spectral radiative transfer models; and third, its dynamic monitoring value, such as capturing subtle ecosystem evolution through multi-temporal spectral change analysis, like early warning of forest pests and diseases. Therefore, remote sensing images obtained through hyperspectral imaging technology have enormous potential in land cover classification tasks.

[0003] In the semantic segmentation of hyperspectral remote sensing images, most existing hyperspectral image segmentation methods have not fully adapted to its unique properties and still continue the processing logic of low-dimensional images. Although these methods attempt to process hyperspectral data, they have significant limitations: the number of bands in hyperspectral images can be more than 70 times that of traditional three-channel images. The high-dimensionality of these images leads to an exponential increase in the computational complexity of the models, and the strong correlation between bands causes serious information redundancy, making it difficult to automatically select key spectral features. Although existing hyperspectral segmentation methods partially focus on spectral information or spatial features, they have not built an effective cross-modal fusion module, and cannot deeply combine spectral dimensional information with spatial location information, resulting in the failure of spectral decomposition of mixed pixels and ultimately making it difficult to identify subtle ground object boundaries in high-resolution hyperspectral images. Traditional methods rely solely on image visual features for segmentation, lacking prior knowledge guidance on "semantic attributes of ground objects." When faced with ground objects with similar spectral features, the lack of semantic distinction makes it difficult for the model to capture subtle differences. Existing technologies cannot integrate multimodal information such as text and prior knowledge bases, resulting in a single segmentation logic and insufficient generalization ability in complex scenarios.

[0004] In the existing technology, there is a lack of a highly efficient and robust hyperspectral image segmentation method that fully considers the characteristics of hyperspectral images based on text semantic features. Summary of the Invention

[0005] To address the technical problems of low efficiency and feature redundancy in high-dimensional data processing, lack of spatial-spectral feature fusion mechanisms and insufficient detail capture in existing technologies, as well as the lack of semantic priors and insufficient utilization of multimodal information, this invention provides a hyperspectral image segmentation method and apparatus based on fusion point cues and Markov diffusion. The technical solution is as follows:

[0006] On the one hand, a hyperspectral image segmentation method based on fusion point cues and Markov diffusion is provided. This method is implemented using a hyperspectral image segmentation device and includes:

[0007] Image acquisition is performed using a hyperspectral sensor to obtain initial hyperspectral data; based on principal component analysis and spectral angle matching algorithm, spectral-spatial dimensionality reduction is performed on the initial hyperspectral data to obtain spatial block features.

[0008] Based on the preset uniformly distributed points, and according to the spatial block characteristics, the spectral-spatial feature adaptation module is used to perform coarse segmentation guided by point prompts to obtain a coarse-grained segmentation mask.

[0009] Based on the cross-dual attention mechanism, text-image feature fusion is performed using a multimodal fusion module based on the initial hyperspectral data and the pre-defined target ground cover structured text to obtain multimodal features;

[0010] Based on the initial hyperspectral data, multi-scale feature extraction was performed using the U-net encoder to obtain multi-scale features;

[0011] A symmetric encoder-decoder convolutional network based on a Markov diffusion model performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation masks, multimodal features, and multi-scale features, and extracts denoised hyperspectral features and high-order fusion masks.

[0012] Multi-scale fine segmentation is performed based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data. Based on the cross-entropy loss function, the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric codec convolutional network are optimized by inverse parameters according to the initial hyperspectral data and segmentation prediction data. This results in optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric codec convolutional network.

[0013] Acquire target hyperspectral data; based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network, perform image segmentation prediction based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

[0014] On the other hand, a hyperspectral image segmentation apparatus based on fusion point cueing and Markov diffusion is provided. This apparatus is applied to a hyperspectral image segmentation method based on fusion point cueing and Markov diffusion, and includes:

[0015] The initial data dimensionality reduction module is used to acquire initial hyperspectral data based on image acquisition using a hyperspectral sensor; based on principal component analysis and spectral angle matching algorithm, it performs spectral-spatial dimensionality reduction processing on the initial hyperspectral data to obtain spatial block features.

[0016] The spectral-spatial feature adaptation module is used to perform coarse segmentation guided by point prompts based on preset uniformly distributed points and spatial block characteristics, thereby obtaining a coarse-grained segmentation mask.

[0017] The multimodal feature fusion module is used to perform text-image feature fusion based on the cross-dual attention mechanism, according to the initial hyperspectral data and the preset target ground cover structured text, to obtain multimodal features;

[0018] The multi-scale feature extraction module is used to extract multi-scale features from the initial hyperspectral data using the U-net encoder to obtain multi-scale features.

[0019] The feature diffusion reconstruction module is used in a symmetric encoder-decoder convolutional network based on a Markov diffusion model. It performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation mask, multimodal features and multi-scale features, and extracts denoised hyperspectral features and high-order fusion mask.

[0020] The model parameter optimization module is used to perform multi-scale fine segmentation based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data. Based on the cross-entropy loss function, the module performs inverse parameter optimization on the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric encoder-decoder convolutional network according to the initial hyperspectral data and segmentation prediction data, resulting in optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network.

[0021] The target image segmentation module is used to acquire target hyperspectral data. Based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network, image segmentation prediction is performed based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

[0022] On the other hand, a hyperspectral image segmentation device is provided, the hyperspectral image segmentation device comprising: a processor; a memory storing computer-readable instructions, wherein when the computer-readable instructions are executed by the processor, any one of the hyperspectral image segmentation methods based on fusion point cueing and Markov diffusion described above is implemented.

[0023] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being loaded and executed by a processor to implement any of the above-described hyperspectral image segmentation methods based on fusion point cueing and Markov diffusion.

[0024] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following:

[0025] This invention proposes a hyperspectral image segmentation method based on point cueing and Markov diffusion. It achieves coarse-grained feature extraction through point cueing, addressing the problems of low efficiency and feature redundancy in high-dimensional data processing. The spatial semantic constraints of point cueing adapt to the challenges of hyperspectral annotation. Leveraging the Markov forward and reverse diffusion processes of the diffusion model, it deeply fuses spatial and spectral features, enhancing the capture of subtle details and optimizing mixed pixel processing and noise robustness. A text-conditional information-driven multimodal fusion mechanism is introduced to compensate for the lack of semantic priors in traditional methods. Text-image cross-modal interaction provides semantic guidance for segmentation, improving the ability to distinguish similar features and generalize to complex scenes, ultimately achieving high-precision semantic segmentation of hyperspectral remote sensing images. This invention is a highly efficient and robust hyperspectral image segmentation method that fully considers the characteristics of hyperspectral images based on text semantic features. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a hyperspectral image segmentation method based on fusion point cueing and Markov diffusion provided by an embodiment of the present invention;

[0028] Figure 2 This is a block diagram of a hyperspectral image segmentation device based on fusion point cueing and Markov diffusion provided in an embodiment of the present invention;

[0029] Figure 3 This is a schematic diagram of the structure of a hyperspectral image segmentation device provided in an embodiment of the present invention. Detailed Implementation

[0030] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0031] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0032] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0033] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0034] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0035] This invention provides a hyperspectral image segmentation method based on fusion point cues and Markov diffusion. This method can be implemented using a hyperspectral image segmentation device, which can be a terminal or a server. Figure 1 The flowchart shown is for a hyperspectral image segmentation method based on fusion point cues and Markov diffusion. The processing flow of this method may include the following steps:

[0036] S1. Image acquisition is performed based on a hyperspectral sensor to obtain initial hyperspectral data; based on principal component analysis and spectral angle matching algorithm, spectral-spatial dimensionality reduction is performed on the initial hyperspectral data to obtain spatial block features.

[0037] Optionally, based on principal component analysis and spectral angle matching algorithms, spectral-spatial dimensionality reduction is performed on the initial hyperspectral data to obtain spatial block features, including:

[0038] Based on the initial hyperspectral data, principal component analysis was used to perform spectral dimensionality reduction to obtain principal component spectral data.

[0039] Based on the principal component spectral data, a spectral angle matching algorithm is used to screen bands and obtain key band spectral data.

[0040] Three-dimensional convolution processing is performed on key band spectral data to obtain local spectral-spatial features;

[0041] The local spectral-spatial features are spatially divided, and classification labels and location codes are added to obtain spatial block features.

[0042] In one feasible implementation, after obtaining the hyperspectral image data, it is necessary to perform dimensionality reduction processing on the data to improve computational efficiency while preserving as much useful information as possible. The initial size of the hyperspectral data is X∈R. h×w×c Where R is the set of real numbers in the initial data; h is the height of the initial hyperspectral data; w is the width of the initial hyperspectral data; and c is the number of channels in the initial hyperspectral data. The first 20 principal components were extracted using principal component analysis.

[0043] For X∈R h×w×c Each band is mean-centered to obtain the covariance matrix. Solve for the eigenvalues ​​λ of C i and the corresponding eigenvector u i After sorting by eigenvalues, the first 20 principal components are selected. The data can then be represented as X′=X·U, where U=[u1,u2,...,u 20 ]∈R c×20 The corresponding principal component projection matrix, after principal component analysis, yields a data size of Y′∈R. h×w×20 Next, a spectral angle matching algorithm was used to filter 10 key bands. Spectral similarity was quantified by calculating the angle between spectral vectors, given two spectral vectors x, y∈R. 20 The formula for calculating the spectral angle is as follows (1):

[0044]

[0045] Where x·y represents the vector dot product, and ‖·‖ represents the L2 norm.

[0046] For the 20 preprocessed principal component bands, the average spectral angle between each band and the spectra of typical land cover in the reference spectral library was calculated. The 10 bands with the smallest average spectral angle were selected as key bands, and finally, the size Z∈R was obtained. h×w×10 The key band features are consistent with the number of 10 categories, and the dimensionality is matched with the subsequent segmentation task.

[0047] After undergoing 3D convolution with a kernel size of 3×3×3 and a stride of 1, local spectral-spatial features are extracted, and the output size becomes J∈R. h×w×32 ;

[0048] Divide the spatial dimension of h×w into There are 3 blocks, each corresponding to K∈R. 16×16×32 The feature dimension, the sequence length is Total input size is After adding category tags and location codes, the resulting size is The spatial block features are used as inputs for subsequent feature extraction and coarse-grained segmentation mask generation modules.

[0049] S2. Based on the preset uniformly distributed points, according to the spatial block characteristics, the spectral-spatial feature adaptation module is used to perform coarse segmentation guided by point prompts to obtain a coarse-grained segmentation mask.

[0050] In one feasible implementation, this invention introduces a point cueing mechanism to achieve coarse segmentation in open-domain scenes, performing coarse segmentation on hyperspectral images and quickly generating preliminary masks to accelerate segmentation. To address the challenge of processing multidimensional data information in hyperspectral images, a spectral-spatial feature adaptation module is specifically designed. This adapter, through innovative network structures and algorithms, enhances the ability to capture semantic relationships between pixels in hyperspectral images and accurately identifies category differences between regions.

[0051] The spectral-spatial feature adaptation module includes a feature extraction module and a mask processing module.

[0052] The feature extraction module includes a 12-layer encoder; the encoder includes a normalization layer, a multi-head attention layer, a normalization processing layer, and a multi-layer perceptron layer;

[0053] The masking module includes a dot hint encoder and a mask decoder; the dot hint encoder includes a coordinate encoding layer and a feature adaptation and fusion layer; the feature adaptation and fusion layer includes a basic embedding layer, a first fully connected layer, a residual connection layer, and a second fully connected layer.

[0054] In one feasible implementation, the spectral-spatial feature adaptation module in this invention includes a feature extraction module and a masking module. The feature extraction module adopts a 12-layer encoder structure, with each layer sequentially executing normalization processing, multi-head attention mechanism, residual fusion, re-normalization processing, and multilayer perceptron. Let the input feature of the l-th layer of the encoder be X. l The complete processing formula for each layer is as follows:

[0055]

[0056] Where LN(·) represents the layer normalization function, MSA(·) represents the multi-head attention mechanism, and MLP(·) represents the multilayer perceptron. Represents element-wise residual addition; X lis the input feature of the l-th layer.

[0057] The standardization process normalizes the input features, while the multi-head attention mechanism captures the long-distance dependencies of the spectral-spatial dimensions in parallel by using eight attention heads while keeping the 8192-dimensional feature dimension constant.

[0058] After multi-head attention calculation is completed, the original input features are combined with the attention calculation results through residual fusion, and then standardized again.

[0059] The multilayer perceptron first increases the feature dimension to 32,768 dimensions through linear transformation, and then reduces it back to 8,192 dimensions through linear transformation to enrich the feature representation capability.

[0060] Optionally, based on preset uniformly distributed points and spatial block characteristics, a spectral-spatial feature adaptation module is used to perform point-guided coarse segmentation to obtain a coarse-grained segmentation mask, including:

[0061] Hierarchical self-attention encoding is performed based on spatial block features to obtain global classification labels and local spatial labels; spectral image embedding is constructed based on global classification labels and local spatial labels.

[0062] Based on spectral image embedding and preset basic semantic embedding, point coding processing is performed on preset uniformly distributed points to obtain point cue embedding;

[0063] Based on the positional and semantic constraints of point cue embedding, fusion processing is performed on spectral image embedding to obtain fused embedding;

[0064] Perform mask decoding on the fused embedding to obtain candidate masks;

[0065] Based on spectral image embedding and point cue embedding, the confidence of the mask is calculated according to the candidate mask.

[0066] Based on a preset confidence threshold, the confidence of the mask is filtered to obtain the confidence filtering results; according to the confidence filtering results, the corresponding mask is selected from the candidate masks to obtain the filtered mask;

[0067] Based on the mask confidence level, the bilinear interpolation method is used to refine the mask according to the candidate mask and the filtered mask, so as to obtain a coarse-grained segmentation mask.

[0068] In one feasible implementation, after layer-by-layer processing by a 12-layer encoder, two types of features are finally output: one is a global classification label with a dimension of 8192, which serves as the global feature; the other is... Each spatial marker has a dimension of 8192, and each spatial marker corresponds to a block feature of size 16×16, preserving local spatial details.

[0069] Point cues are constructed by sampling uniformly distributed points at preset fixed intervals. The coordinates of these points are input into a point cue encoder, and after position encoding and learnable parameter transformation, a point cue embedding that can be fused with image features is generated.

[0070] The spectral image, composed of classification and spatial labels, is embedded and fused with point cue embedding. Element-wise addition and attention mechanisms are used to integrate global and local image features with the positional and semantic constraints of the point cue. The fused features are then input into a mask decoder, which generates an initial candidate mask based on this input.

[0071] Two confidence scores are performed, using an internal scoring head combined with point cues and image content to evaluate the matching degree and quality of candidate masks. Low-quality masks are filtered out based on a preset score threshold, while high-confidence results are retained.

[0072] A channel-wise multiplication fusion method is employed: effective features are compressed to 10 dimensions (consistent with the number of channels in the initial mask) using a 1×1 convolution, and then multiplied element-wise with the initial mask to output a fused low-resolution feature mask. Bilinear interpolation is used for single-pass upsampling. The output mask, at its original resolution, is then verified, with dimensions perfectly matching the input hyperspectral image. Based on the obtained effective features, the confidence level of the initial mask is verified, low-quality regions are filtered out, and high-confidence results are retained to generate the final coarse-grained mask.

[0073] Finally, the generated size is B∈R h×w×10 The coarse-grained segmentation mask provides a scale and channel number-adapted coarse-grained segmentation mask for subsequent segmentation tasks, realizing the conversion from encoder features to target mask, which is used as input for subsequent models.

[0074] Optionally, based on spectral image embedding and preset basic semantic embedding, point coding processing is performed on preset uniformly distributed points to obtain point cue embedding, including:

[0075] Spatial dimension location encoding is performed on the preset uniformly distributed points to obtain the location encoding vector;

[0076] The position encoding vector and the preset basic semantic embedding are concatenated and processed using a linear transformation to obtain the first coordinate-semantic hybrid embedding.

[0077] Based on spectral image embedding, the feature distribution of the first coordinate-semantic hybrid embedding is fine-tuned to obtain the second coordinate-semantic hybrid embedding.

[0078] The point cue embedding is obtained by adding elements together based on the first coordinate-semantic hybrid embedding and the second coordinate-semantic hybrid embedding.

[0079] In one feasible implementation, during the generation of the point hint embedding, positional encoding is used to transform the spatial coordinate information of the point hint into a vector representation that can be fused with image features, thereby preserving the semantic constraints of the point hint's spatial location. Let the spatial coordinates of the point hint be (x... p y p ), where x p Let y be the normalized coordinate of the point in the width direction of the image. p These are normalized coordinates in the height direction. For coordinate x... p and y p Perform position encoding separately to generate a position vector of dimension dpos for position coordinates p∈{x} p y p The positional encoding of the i-th dimension is calculated as follows: (3) and (4):

[0080]

[0081] Where i is the dimension index, and its value range is... Using the above formula, the spatial coordinates (x, y) of the point prompt are obtained. p ,y p The vector is converted into two position vectors with dimension dpos, and then concatenated to obtain the final position encoding vector. This position encoding vector is then added element-wise to the base feature vector of the point cue to generate a point cue embedding containing spatial location information.

[0082] In the dot cue encoder, the input is the coordinates of uniformly distributed points. The output is the dot cue embedding containing spatial location information; the input is the normalized coordinates of uniformly distributed points, calculated using formulas (3) and (4), and the spatial coordinates (x, y, z) of the dot cue are... p ,y p The model is converted into two location vectors with a dimension of dpos. These two dpos location vectors are then concatenated into a single location encoding vector with a dimension of 2dpos. Learnable parameters are used to generate a base vector adapted to the semantics of hyperspectral ground features. This base vector is then fused with the location encoding vector to increase its dimensionality. A basic semantic embedding is generated, with parameters optimized during model training to gradually adapt to the semantic distribution of ground features in the hyperspectral image. A linear fully connected network with no activation function ensures compatibility between the base embedding and the location information.

[0083] In the first fully connected layer, a multilayer perceptron (MLP) is used, employing a ReLU activation function for nonlinear transformation to enrich feature representation and adapt to the high-dimensional feature space of hyperspectral images. In the second fully connected layer, the MLP is used to fine-tune the feature distribution, ensuring consistency between the point cue embedding and the image feature distribution and avoiding feature shift during fusion. The output of the first MLP layer is used as a residual term and added element-wise with the output of the second MLP layer to avoid gradient vanishing and ensure the integrity of features based on classless embeddings and coordinate information.

[0084] S3. Based on the cross-dual attention mechanism, according to the initial hyperspectral data and the preset target ground object structured text, the multimodal fusion module is used to perform text-image feature fusion to obtain multimodal features;

[0085] Optionally, based on a cross-dual attention mechanism, a multimodal fusion module is used to perform text-image feature fusion based on the initial hyperspectral data and the pre-defined structured text of the target land cover to obtain multimodal features, including:

[0086] Based on the pre-trained BERT-Base model, feature extraction is performed on the pre-defined target ground object structured text to obtain text features;

[0087] Hyperspectral features are obtained by extracting features from the initial hyperspectral data;

[0088] Based on hyperspectral features, a spectral-guided text enhancement attention calculation is performed according to text features to obtain the first attention value;

[0089] Based on text features, a second attention value is obtained by calculating spectral enhancement attention guided by text features and based on hyperspectral features.

[0090] Based on the first attention value and the second attention value, feature integration is performed according to the hyperspectral features to obtain multimodal features.

[0091] In one feasible implementation, the present invention introduces a text semantic prior constraint mechanism. This mechanism, through the design of a cross-modal alignment module and a conditional generation network, achieves deep interaction between text semantic information and hyperspectral image features, providing accurate semantic guidance for the segmentation task. By using text semantic priors to compensate for the lack of supervision information, it achieves cross-modal consistency constraints between "semantic description - spectral features - segmentation results".

[0092] Using pre-defined structured text describing key features of target features, text embedding vectors T∈R are extracted using a pre-trained language model BERT-Base. L×dWhere L is the length of the text sequence and d is the dimension of the hidden layer. Additionally, a bidirectional cross-attention module is designed to achieve semantic alignment between the text and spectral features.

[0093] Spectral-guided text enhancement: Hyperspectral feature map X∈R H×W×C Generate query matrix Q using self-attention. s Text embedding T as key K t Sum V t The calculation process is as follows: (5)

[0094]

[0095] Text-guided spectral enhancement: Text embedding T generates query Q t Hyperspectral feature X as bond K s Sum V s The calculation process is as follows: (6)

[0096]

[0097] The bidirectional attention results are concatenated with the original features and then integrated through residual connections, as shown in equation (7):

[0098] X fusion =X+MLP([A s→t A t→s ]) (7);

[0099] The aligned multimodal features X fusion As the conditional input c of the Markov diffusion model's reverse process, in each diffusion step t, the conditional noise prediction network ∈ θ (x t The image signal is gradually recovered using textual semantic constraints, and the denoising process is guided by text embedding and attention mapping of spectral channels. i，j =Attention(T) i X j This enhances the sensitivity to key spectral features.

[0100] When processing text, the BERT-Base model first converts the input text into a fixed-format token sequence. The length of the output embedding vector is the same as the length of the input token sequence, which is L in the patent. The hidden layer dimension d = 768. The final result is an L×d matrix.

[0101] S4. Based on the initial hyperspectral data, use the U-net encoder to extract multi-scale features and obtain multi-scale features;

[0102] In one feasible implementation, the U-Net encoder is used to downsample the hyperspectral cube in this invention, thereby enabling the simultaneous capture of multi-scale micro-targets, texture details, and a wide range of semantic context, providing complementary and robust discrimination criteria for subsequent segmentation.

[0103] S5. A symmetric encoder-decoder convolutional network based on the Markov diffusion model performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation mask, multimodal features and multi-scale features, and extracts denoised hyperspectral features and high-order fusion mask.

[0104] Optionally, a symmetric encoder-decoder convolutional network based on a Markov diffusion model performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation masks, multimodal features, and multi-scale features, and extracts denoised hyperspectral features and high-order fusion masks, including:

[0105] Based on coarse-grained segmentation masks, forward diffusion noise is applied to the initial hyperspectral data to obtain noisy hyperspectral data;

[0106] Based on multimodal and multiscale features, back-diffusion denoising is performed on noisy hyperspectral data to obtain denoised hyperspectral data.

[0107] Feature extraction is performed on the denoised hyperspectral data to obtain denoised hyperspectral features; channel separation is performed on the denoised hyperspectral features to obtain denoised hyperspectral features and a high-order fusion mask.

[0108] In one feasible implementation, the original hyperspectral features are jointly noise-added with a coarse-grained segmentation mask, and the data is gradually mapped to a Gaussian distribution during diffusion. In the reverse denoising process, the original features are gradually recovered using a symmetric encoder-decoder convolutional network architecture.

[0109] In the forward joint denoising process, a diffusion prior distribution is constructed. After receiving the original hyperspectral features and a coarse-grained segmentation mask, the encoder performs a joint denoising operation. Through multiple iterations, the joint features are gradually mapped to a Gaussian distribution, completing the forward diffusion process and providing a prior space for the reverse denoising. The original hyperspectral features are F∈R. h×w×c The coarse-grained segmentation mask is M∈R h×w×10 The joint feature Z0 = concat[F, M]. The forward diffusion process adds Gaussian noise progressively through T iterations, and the joint feature Z0 at step t is... t The calculation process is as follows: (8)

[0110]

[0111] Where, α t ∈(0,1) represents the preset noise control parameters. This is a standard Gaussian noise vector. When t = T, That is, the joint features are completely mapped to a Gaussian distribution, providing a prior distribution for inverse denoising.

[0112] In the reverse denoising stage, the symmetric encoder-decoder convolutional network performs reverse iterations to gradually eliminate noise and reconstruct hyperspectral features close to the original distribution, providing feature input for the final segmentation. The reverse diffusion process removes Gaussian noise. Starting from this point, noise is gradually eliminated through T iterations to reconstruct the joint features. The denoising update in step t is as follows (9):

[0113]

[0114] in, For cumulative noise parameters, ∈ θ (·) represents the noise prediction network. For sampling noise, σ t The noise standard deviation is defined as follows (10):

[0115]

[0116] By iterating to t=0, the output Z′0≈Z0, which is the reconstructed joint feature, is used to separate the recovered hyperspectral feature F′ to provide accurate input for the final segmentation.

[0117] The forward diffusion process gradually adds noise to the hyperspectral image, making the image data gradually approach a Gaussian distribution, while the reverse diffusion process learns the mapping relationship from noise to the original image. By combining the two, the mask obtained from coarse segmentation can be fully utilized to extract spectral features in the hyperspectral image, significantly enhancing the ability to capture image context information and greatly improving the detection accuracy of tiny details in high-resolution hyperspectral images.

[0118] S6. Perform multi-scale fine segmentation based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data; based on the cross-entropy loss function, optimize the inverse parameters of the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric encoder-decoder convolutional network according to the initial hyperspectral data and segmentation prediction data to obtain optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network;

[0119] In one feasible implementation, the input features F of the fine segmentation head are formed through channel splicing and attention fusion. in Using convolutional layers to apply F inChannel compression and spatial feature extraction are performed to enhance the boundaries and category characteristics of ground features. The feature map size is gradually increased through upsampling operations, restoring the resolution from low-resolution features to that of the original hyperspectral image while preserving multi-scale features.

[0120] The final convolutional layer compresses the feature channels to 10, the number of land cover categories, and then performs softmax activation to obtain the segmentation probability mask. For each spatial location (i, j) This represents the probability that the location belongs to the k-th type of land cover.

[0121] The final segmentation result output by the decoder at the end of the model. The true value of the segmentation Y and the annotation gt The error is calculated using the cross-entropy loss, as shown in equation (11):

[0122]

[0123] This loss is applied to the overall model for global optimization and backpropagates to the entire chain, including the Unet encoder, symmetric encoder-decoder convolutional network, etc., so that the model forms a cooperative constraint between coarse segmentation alignment and fine segmentation accuracy, thereby improving the robustness and accuracy of the segmentation task.

[0124] S7. Obtain target hyperspectral data; based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder and optimized symmetric encoder-decoder convolutional network, perform image segmentation prediction based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

[0125] In one feasible implementation, quantitative results from this invention on three publicly available datasets—the Indian Pines AVIRIS Hyperspectral Reflectance Dataset, the Pavia University AVIRIS Hyperspectral Reflectance Dataset, and the Salinas Valley AVIRIS Hyperspectral Reflectance Dataset—show that, compared to existing traditional methods such as Support Vector Machine (SVM) and U-shaped Convolutional Neural Network (UNet) and deep learning methods such as 3D Convolutional Neural Network (3D-CNN) and Transformer-U-shaped Convolutional Neural Network (Transformer-U-shaped Convolutional Neural Network)… The Transformer-Unet network improves overall accuracy (OA) by an average of 2.2–5.5 percentage points, boundary F1 scores by an average of 2.8–5.1 percentage points, and training time by an average of 10%–28%. It effectively solves the problems of "low efficiency in high-dimensional processing, insufficient spatial-spectral fusion, and lack of semantic priors" in existing technologies, meets the high-precision and high-efficiency segmentation requirements of hyperspectral remote sensing images, and can provide technical support for fields such as environmental protection and resource management.

[0126] This invention proposes a hyperspectral image segmentation method based on point cueing and Markov diffusion. It achieves coarse-grained feature extraction through point cueing, addressing the problems of low efficiency and feature redundancy in high-dimensional data processing. The spatial semantic constraints of point cueing adapt to the challenges of hyperspectral annotation. Leveraging the Markov forward and reverse diffusion processes of the diffusion model, it deeply fuses spatial and spectral features, enhancing the capture of subtle details and optimizing mixed pixel processing and noise robustness. A text-conditional information-driven multimodal fusion mechanism is introduced to compensate for the lack of semantic priors in traditional methods. Text-image cross-modal interaction provides semantic guidance for segmentation, improving the ability to distinguish similar features and generalize to complex scenes, ultimately achieving high-precision semantic segmentation of hyperspectral remote sensing images. This invention is a highly efficient and robust hyperspectral image segmentation method that fully considers the characteristics of hyperspectral images based on text semantic features.

[0127] Figure 2 This is a block diagram of a hyperspectral image segmentation device based on fusion point cues and Markov diffusion, provided in an embodiment of the present invention. This device is used for hyperspectral image segmentation based on fusion point cues and Markov diffusion. (Refer to...) Figure 2 The device includes an initial data dimensionality reduction module 210, a spectral-spatial feature adaptation module 220, a multimodal feature fusion module 230, a multi-scale feature extraction module 240, a feature diffusion reconstruction module 250, a model parameter optimization module 260, and a target image segmentation module 270. Among them:

[0128] The initial data dimensionality reduction module 210 is used to acquire initial hyperspectral data by image acquisition based on a hyperspectral sensor; and to perform spectral-spatial dimensionality reduction processing on the initial hyperspectral data based on principal component analysis and spectral angle matching algorithm to obtain spatial block features.

[0129] The spectral-spatial feature adaptation module 220 is used to perform coarse segmentation guided by point prompts based on preset uniformly distributed points and spatial block features to obtain a coarse-grained segmentation mask.

[0130] The multimodal feature fusion module 230 is used to perform text-image feature fusion based on the cross-dual attention mechanism, according to the initial hyperspectral data and the preset target ground object structured text, to obtain multimodal features.

[0131] The multi-scale feature extraction module 240 is used to extract multi-scale features from the initial hyperspectral data using the U-net encoder to obtain multi-scale features.

[0132] The feature diffusion reconstruction module 250 is used in a symmetric encoder-decoder convolutional network based on a Markov diffusion model. It performs diffusion reconstruction based on hyperspectral initial data, coarse-grained segmentation mask, multimodal features and multi-scale features, and extracts denoised hyperspectral features and high-order fusion mask.

[0133] The model parameter optimization module 260 is used to perform multi-scale fine segmentation based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data. Based on the cross-entropy loss function, the module optimizes the parameters of the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric encoder-decoder convolutional network according to the initial hyperspectral data and segmentation prediction data, resulting in optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network.

[0134] The target image segmentation module 270 is used to acquire target hyperspectral data; based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder and optimized symmetric encoder-decoder convolutional network, it performs image segmentation prediction based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

[0135] Optionally, the initial data dimensionality reduction module 210 is further used for:

[0136] Based on the initial hyperspectral data, principal component analysis was used to perform spectral dimensionality reduction to obtain principal component spectral data.

[0137] Based on the principal component spectral data, a spectral angle matching algorithm is used to screen bands and obtain key band spectral data.

[0138] Three-dimensional convolution processing is performed on key band spectral data to obtain local spectral-spatial features;

[0139] The local spectral-spatial features are spatially divided, and classification labels and location codes are added to obtain spatial block features.

[0140] The spectral-spatial feature adaptation module includes a feature extraction module and a mask processing module.

[0141] The feature extraction module includes a 12-layer encoder; the encoder includes a normalization layer, a multi-head attention layer, a normalization processing layer, and a multi-layer perceptron layer;

[0142] The masking module includes a dot hint encoder and a mask decoder; the dot hint encoder includes a coordinate encoding layer and a feature adaptation and fusion layer; the feature adaptation and fusion layer includes a basic embedding layer, a first fully connected layer, a residual connection layer, and a second fully connected layer.

[0143] Optionally, the spectral-spatial feature adaptation module 220 is further used for:

[0144] Hierarchical self-attention encoding is performed based on spatial block features to obtain global classification labels and local spatial labels; spectral image embedding is constructed based on global classification labels and local spatial labels.

[0145] Based on spectral image embedding and preset basic semantic embedding, point coding processing is performed on preset uniformly distributed points to obtain point cue embedding;

[0146] Based on the positional and semantic constraints of point cue embedding, fusion processing is performed on spectral image embedding to obtain fused embedding;

[0147] Perform mask decoding on the fused embedding to obtain candidate masks;

[0148] Based on spectral image embedding and point cue embedding, the confidence of the mask is calculated according to the candidate mask.

[0149] Based on a preset confidence threshold, the confidence of the mask is filtered to obtain the confidence filtering results; according to the confidence filtering results, the corresponding mask is selected from the candidate masks to obtain the filtered mask;

[0150] Based on the mask confidence level, the bilinear interpolation method is used to refine the mask according to the candidate mask and the filtered mask, so as to obtain a coarse-grained segmentation mask.

[0151] Optionally, the spectral-spatial feature adaptation module 220 is further used for:

[0152] Spatial dimension location encoding is performed on the preset uniformly distributed points to obtain the location encoding vector;

[0153] The position encoding vector and the preset basic semantic embedding are concatenated and processed using a linear transformation to obtain the first coordinate-semantic hybrid embedding.

[0154] Based on spectral image embedding, the feature distribution of the first coordinate-semantic hybrid embedding is fine-tuned to obtain the second coordinate-semantic hybrid embedding.

[0155] The point cue embedding is obtained by adding elements together based on the first coordinate-semantic hybrid embedding and the second coordinate-semantic hybrid embedding.

[0156] Optionally, the multimodal feature fusion module 230 is further used for:

[0157] Based on the pre-trained BERT-Base model, feature extraction is performed on the pre-defined target ground object structured text to obtain text features;

[0158] Hyperspectral features are obtained by extracting features from the initial hyperspectral data;

[0159] Based on hyperspectral features, a spectral-guided text enhancement attention calculation is performed according to text features to obtain the first attention value;

[0160] Based on text features, a second attention value is obtained by calculating spectral enhancement attention guided by text features and based on hyperspectral features.

[0161] Based on the first attention value and the second attention value, feature integration is performed according to the hyperspectral features to obtain multimodal features.

[0162] Optionally, the feature diffusion reconstruction module 250 is further used for:

[0163] Based on coarse-grained segmentation masks, forward diffusion noise is applied to the initial hyperspectral data to obtain noisy hyperspectral data;

[0164] Based on multimodal and multiscale features, back-diffusion denoising is performed on noisy hyperspectral data to obtain denoised hyperspectral data.

[0165] Feature extraction is performed on the denoised hyperspectral data to obtain denoised hyperspectral features; channel separation is performed on the denoised hyperspectral features to obtain denoised hyperspectral features and a high-order fusion mask.

[0166] This invention proposes a hyperspectral image segmentation method based on point cueing and Markov diffusion. It achieves coarse-grained feature extraction through point cueing, addressing the problems of low efficiency and feature redundancy in high-dimensional data processing. The spatial semantic constraints of point cueing adapt to the challenges of hyperspectral annotation. Leveraging the Markov forward and reverse diffusion processes of the diffusion model, it deeply fuses spatial and spectral features, enhancing the capture of subtle details and optimizing mixed pixel processing and noise robustness. A text-conditional information-driven multimodal fusion mechanism is introduced to compensate for the lack of semantic priors in traditional methods. Text-image cross-modal interaction provides semantic guidance for segmentation, improving the ability to distinguish similar features and generalize to complex scenes, ultimately achieving high-precision semantic segmentation of hyperspectral remote sensing images. This invention is a highly efficient and robust hyperspectral image segmentation method that fully considers the characteristics of hyperspectral images based on text semantic features.

[0167] Figure 3 This is a schematic diagram of the structure of a hyperspectral image segmentation device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, the hyperspectral image segmentation device may include the above-mentioned Figure 2 The illustrated hyperspectral image segmentation apparatus is based on fusion point cues and Markov diffusion. Optionally, the hyperspectral image segmentation apparatus 310 may include a first processor 2001.

[0168] Optionally, the hyperspectral image segmentation device 310 may also include a memory 2002 and a transceiver 2003.

[0169] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0170] The following is combined Figure 3 A detailed description of each component of the hyperspectral image segmentation device 310 is provided below:

[0171] The first processor 2001 is the control center of the hyperspectral image segmentation device 310. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0172] Optionally, the first processor 2001 can perform various functions of the hyperspectral image segmentation device 310 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0173] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 3 CPU0 and CPU1 are shown in the diagram.

[0174] In a specific implementation, as one example, the hyperspectral image segmentation device 310 may also include multiple processors, for example... Figure 3 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, "processor" can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0175] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0176] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or may exist independently, and may be connected via the interface circuit of the hyperspectral image segmentation device 310. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0177] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0178] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 3 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function.

[0179] Optionally, the transceiver 2003 can be integrated with the first processor 2001, or it can exist independently and be connected to the interface circuit of the hyperspectral image segmentation device 310. Figure 3 (Not shown in the figure) is coupled to the first processor 2001, and the embodiments of the present invention do not specifically limit this.

[0180] It should be noted that, Figure 3 The structure of the hyperspectral image segmentation device 310 shown does not constitute a limitation on the router. Actual hyperspectral image segmentation devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0181] Furthermore, the technical effect of the hyperspectral image segmentation device 310 can be referred to the technical effect of the hyperspectral image segmentation method based on fusion point cueing and Markov diffusion described in the above method embodiments, and will not be repeated here.

[0182] It should be understood that the first processor 2001 in this embodiment of the invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor.

[0183] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0184] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0185] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0186] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0187] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0188] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0189] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0190] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0191] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0192] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0193] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0194] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A hyperspectral image segmentation method based on fusion point cues and Markov diffusion, characterized in that, The method includes: Image acquisition is performed using a hyperspectral sensor to obtain initial hyperspectral data; based on principal component analysis and spectral angle matching algorithm, spectral-spatial dimensionality reduction is performed on the initial hyperspectral data to obtain spatial block features. Based on the preset uniformly distributed points, and according to the spatial block characteristics, the spectral-spatial feature adaptation module is used to perform coarse segmentation guided by point prompts to obtain a coarse-grained segmentation mask. Based on the cross-dual attention mechanism, text-image feature fusion is performed using a multimodal fusion module based on the initial hyperspectral data and the pre-defined target ground cover structured text to obtain multimodal features; Based on the initial hyperspectral data, multi-scale feature extraction was performed using the U-net encoder to obtain multi-scale features; A symmetric encoder-decoder convolutional network based on a Markov diffusion model performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation masks, multimodal features, and multi-scale features, and extracts denoised hyperspectral features and high-order fusion masks. Multi-scale fine segmentation is performed based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data. Based on the cross-entropy loss function, the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric codec convolutional network are optimized by inverse parameters according to the initial hyperspectral data and segmentation prediction data. This results in optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric codec convolutional network. Acquire target hyperspectral data; based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network, perform image segmentation prediction based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

2. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 1, characterized in that, The method based on principal component analysis and spectral angle matching algorithm performs spectral-spatial dimensionality reduction processing on the initial hyperspectral data to obtain spatial block features, including: Based on the initial hyperspectral data, principal component analysis was used to perform spectral dimensionality reduction to obtain principal component spectral data. Based on the principal component spectral data, a spectral angle matching algorithm is used to screen bands and obtain key band spectral data. Three-dimensional convolution processing is performed on key band spectral data to obtain local spectral-spatial features; The local spectral-spatial features are spatially divided, and classification labels and location codes are added to obtain spatial block features.

3. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 1, characterized in that, The spectral-spatial feature adaptation module includes a feature extraction module and a masking module; The feature extraction module includes a 12-layer encoder; the encoder includes a normalization layer, a multi-head attention layer, a normalization processing layer, and a multi-layer perceptron layer. The mask processing module includes a dot prompt encoder and a mask decoder; The point prompt encoder includes a coordinate encoding layer and a feature adaptation and fusion layer; The feature adaptation fusion layer includes a basic embedding layer, a first fully connected layer, a residual connection layer, and a second fully connected layer.

4. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 1, characterized in that, Based on preset uniformly distributed points, and according to spatial block characteristics, a spectral-spatial feature adaptation module is used to perform point-guided coarse segmentation to obtain a coarse-grained segmentation mask, including: Hierarchical self-attention encoding is performed based on spatial block features to obtain global classification labels and local spatial labels; spectral image embedding is constructed based on global classification labels and local spatial labels. Based on spectral image embedding and preset basic semantic embedding, point coding processing is performed on preset uniformly distributed points to obtain point cue embedding; Based on the positional and semantic constraints of point cue embedding, fusion processing is performed on spectral image embedding to obtain fused embedding; Perform mask decoding on the fused embedding to obtain candidate masks; Based on spectral image embedding and point cue embedding, the confidence of the mask is calculated according to the candidate mask. Based on a preset confidence threshold, the confidence of the mask is filtered to obtain the confidence filtering results; according to the confidence filtering results, the corresponding mask is selected from the candidate masks to obtain the filtered mask; Based on the mask confidence level, the bilinear interpolation method is used to refine the mask according to the candidate mask and the filtered mask, so as to obtain a coarse-grained segmentation mask.

5. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 4, characterized in that, The method based on spectral image embedding and preset basic semantic embedding performs point encoding processing on preset uniformly distributed points to obtain point cue embedding, including: Spatial dimension location encoding is performed on the preset uniformly distributed points to obtain the location encoding vector; The position encoding vector and the preset basic semantic embedding are concatenated and processed using a linear transformation to obtain the first coordinate-semantic hybrid embedding. Based on spectral image embedding, the feature distribution of the first coordinate-semantic hybrid embedding is fine-tuned to obtain the second coordinate-semantic hybrid embedding. The point cue embedding is obtained by adding elements together based on the first coordinate-semantic hybrid embedding and the second coordinate-semantic hybrid embedding.

6. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 1, characterized in that, The cross-dual attention mechanism, based on initial hyperspectral data and pre-defined structured text of target land features, uses a multimodal fusion module to perform text-image feature fusion to obtain multimodal features, including: Based on the pre-trained BERT-Base model, feature extraction is performed on the pre-defined target ground object structured text to obtain text features; Hyperspectral features are obtained by extracting features from the initial hyperspectral data; Based on hyperspectral features, a spectral-guided text enhancement attention calculation is performed according to text features to obtain the first attention value; Based on text features, a second attention value is obtained by calculating spectral enhancement attention guided by text features and based on hyperspectral features. Based on the first attention value and the second attention value, feature integration is performed according to the hyperspectral features to obtain multimodal features.

7. The hyperspectral image segmentation method based on fusion point cues and Markov diffusion according to claim 1, characterized in that, The symmetric encoder-decoder convolutional network based on the Markov diffusion model performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation masks, multimodal features, and multi-scale features, and extracts denoised hyperspectral features and high-order fusion masks, including: Based on coarse-grained segmentation masks, forward diffusion noise is applied to the initial hyperspectral data to obtain noisy hyperspectral data; Based on multimodal and multiscale features, back-diffusion denoising is performed on noisy hyperspectral data to obtain denoised hyperspectral data. Feature extraction is performed on the denoised hyperspectral data to obtain denoised hyperspectral features; channel separation is performed on the denoised hyperspectral features to obtain denoised hyperspectral features and a high-order fusion mask.

8. A hyperspectral image segmentation apparatus based on fusion point cues and Markov diffusion, wherein the hyperspectral image segmentation apparatus based on fusion point cues and Markov diffusion is used to implement the hyperspectral image segmentation method based on fusion point cues and Markov diffusion as described in any one of claims 1-7, characterized in that, The device includes: The initial data dimensionality reduction module is used to acquire initial hyperspectral data based on image acquisition using a hyperspectral sensor; based on principal component analysis and spectral angle matching algorithm, it performs spectral-spatial dimensionality reduction processing on the initial hyperspectral data to obtain spatial block features. The spectral-spatial feature adaptation module is used to perform coarse segmentation guided by point prompts based on preset uniformly distributed points and spatial block characteristics, thereby obtaining a coarse-grained segmentation mask. The multimodal feature fusion module is used to perform text-image feature fusion based on the cross-dual attention mechanism, according to the initial hyperspectral data and the preset target ground cover structured text, to obtain multimodal features; The multi-scale feature extraction module is used to extract multi-scale features from the initial hyperspectral data using the U-net encoder to obtain multi-scale features. The feature diffusion reconstruction module is used in a symmetric encoder-decoder convolutional network based on a Markov diffusion model. It performs diffusion reconstruction based on initial hyperspectral data, coarse-grained segmentation mask, multimodal features and multi-scale features, and extracts denoised hyperspectral features and high-order fusion mask. The model parameter optimization module is used to perform multi-scale fine segmentation based on hyperspectral features and high-order fusion masks to obtain segmentation prediction data. Based on the cross-entropy loss function, the module performs inverse parameter optimization on the spectral-spatial feature adaptation module, multimodal fusion module, U-net encoder, and symmetric encoder-decoder convolutional network according to the initial hyperspectral data and segmentation prediction data, resulting in optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network. The target image segmentation module is used to acquire target hyperspectral data. Based on the optimized spectral-spatial feature adaptation module, optimized multimodal fusion module, optimized U-net encoder, and optimized symmetric encoder-decoder convolutional network, image segmentation prediction is performed based on the hyperspectral data to be segmented to obtain target segmentation prediction data.

9. A hyperspectral image segmentation device, characterized in that, The hyperspectral image segmentation device includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium contains program code that can be invoked by a processor to execute the method as described in any one of claims 1 to 7.

Images