Hyperspectral oil spill identification method and device based on space-spectrum-frequency domain joint representation

Abstract

The application discloses a hyperspectral oil spill identification method and device based on space-spectrum-frequency joint representation, and relates to the technical field of hyperspectral remote sensing image processing and marine environment monitoring. The method comprises the following steps: performing scene-based standard normal variable normalization processing and band selection on hyperspectral cube oil product data; synchronously feeding the selected key band data into a ViT space-spectrum feature extraction branch, a frequency domain feature learning branch and a global semantic feature extraction branch. The frequency domain feature, the semantic feature and the space-spectrum feature are combined, and the obtained spliced feature is input into a gated channel attention module to obtain optimized discriminative features. The optimized discriminative features are input into a classification output end, and based on a mask and a confidence decision mechanism, an oil spill identification result is output. By using the application, pixel-level oil product high-precision classification can be realized under different scene, thickness and water body conditions, the false alarm rate is effectively reduced, and reliable support is provided for marine oil spill monitoring and emergency response.

Description

Technical Field

[0001] This invention relates to the fields of hyperspectral remote sensing image processing and marine environmental monitoring technology, and in particular to a hyperspectral oil spill identification method and device with joint spatial-spectral-frequency domain characterization. Background Technology

[0002] Hyperspectral imaging technology can acquire spectral reflectance information of targets within a continuous narrow band, and each pixel can form a spectral curve containing material information. Different oils exhibit different reflectance intensities, absorption characteristics, and curve shapes in specific bands due to their different chemical compositions, providing a data foundation for oil and water background identification.

[0003] However, hyperspectral oil spill identification is significantly affected by scene factors. For the same type of oil, the spectrum may shift and change shape under different times, water backgrounds, oil film thicknesses, or lighting conditions. Models are prone to mislearning scene differences as class differences, leading to decreased cross-scene classification performance. Hyperspectral data contains hundreds of continuous bands, with strong correlations between adjacent bands; inputting all of them introduces redundancy and increases computational burden. Existing methods such as principal component analysis and correlation coefficient screening focus on dimensionality reduction efficiency or inter-class differences, rarely considering cross-scene drift, noise stability, and inter-band redundancy simultaneously.

[0004] Among existing classification methods, the threshold of the spectral index method depends on scene conditions and has limited adaptability; support vector machines and random forests rely on artificial features and are difficult to uncover complex nonlinear relationships; convolutional neural networks can extract local textures, but are insufficient for modeling long-range band dependencies. In recent years, although visual Transformers can enhance feature associations through self-attention, existing methods mostly encode the entire band directly, lacking stable band constraints for oil spill identification, and mainly rely on raw spectral domain features, failing to utilize trends and energy distributions in the frequency domain. In addition, global conditions such as sensor status and observation environment can provide scene constraints for classification, but existing methods do not incorporate their structured encoding into the learning process.

[0005] The existing technology has the following problems: (1) Existing band screening methods are not stable enough. They do not take into account cross-scenario drift, noise and inter-band redundancy of similar oil products in the evaluation, and the discrimination ability decreases after the scenario changes.

[0006] (2) Existing methods are not capable of expressing and fusing multidimensional features. They often use simple splicing or fixed weight fusion, making it difficult to dynamically judge the reliability of each feature according to the scene conditions.

[0007] (3) Existing methods lack structured encoding of scene information, which can easily lead to mislearning scene differences as category differences, resulting in a decline in cross-scene generalization performance. Summary of the Invention

[0008] To address the technical problems of insufficient stability in band selection, lack of multi-dimensional feature expression and fusion capabilities, and limited cross-scenario adaptability in existing technologies, this invention provides a hyperspectral oil spill identification method and apparatus with joint spatial-spectral-frequency domain characterization. The technical solution is as follows:

[0009] On the one hand, a hyperspectral oil spill identification method with joint spatial-spectral-frequency domain characterization is provided. This method is implemented by a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization, and includes: S1. Obtain the image data to be identified and the trained oil spill identification model. The image data to be identified is hyperspectral cubic oil data. The oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end. S2. Perform scene-based standard normalization on the hyperspectral cubic oil data, and select bands on the normalized hyperspectral data to obtain key band data. S3. Simultaneously feed the key band data into the ViT spatial spectral feature extraction branch, the frequency domain feature learning branch, and the global semantic feature extraction branch. Obtain spatial spectral features based on the ViT spatial spectral feature extraction branch, obtain frequency domain features based on the frequency domain feature learning branch, and obtain semantic features based on the global semantic feature extraction branch. S4. Combine the frequency domain features with the semantic features, concatenate the generated auxiliary expression with the spatial spectrum features, and input the resulting concatenated features into the gated channel attention module to obtain the optimized discriminative features; S5. Input the optimized discrimination features into the classification output terminal, and output the oil spill identification result based on the mask and confidence decision mechanism.

[0010] On the other hand, a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization is provided. This device is applied to the hyperspectral oil spill identification method with joint spatial-spectral-frequency domain characterization. The device includes: The acquisition unit is used to acquire the image data to be identified and the trained oil spill identification model. The image data to be identified is hyperspectral cubic oil data. The oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end. The preprocessing unit is used to perform scene-based standard normalization of hyperspectral cubic oil data, and to select bands from the normalized hyperspectral data to obtain key band data. The feature extraction unit is used to synchronously send key band data into the ViT spatial spectral feature extraction branch, the frequency domain feature learning branch, and the global semantic feature extraction branch. Based on the ViT spatial spectral feature extraction branch, spatial spectral features are obtained; based on the frequency domain feature learning branch, frequency domain features are obtained; and based on the global semantic feature extraction branch, semantic features are obtained. The fusion unit is used to combine frequency domain features with semantic features, concatenate the generated auxiliary representation with spatial spectral features, and input the resulting concatenated features into the gated channel attention module to obtain optimized discriminative features; The identification unit is used to input optimized discrimination features into the classification output end and output oil spill identification results based on the mask and confidence decision mechanism.

[0011] On the other hand, a hyperspectral oil spill identification device with spatial-spectral-frequency domain joint characterization is provided. The hyperspectral oil spill identification device with spatial-spectral-frequency domain joint characterization includes: a processor; a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, any one of the methods of hyperspectral oil spill identification with spatial-spectral-frequency domain joint characterization as described above is implemented.

[0012] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the at least one instruction being loaded and executed by a processor to implement any of the above-described hyperspectral oil spill identification methods with joint spatial-spectral-frequency domain characterization.

[0013] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: This invention employs scene-level SNV normalization to eliminate cross-scene spectral shifts. It uses class separability as a positive objective and cross-scene drift and noise sensitivity as penalty terms for band selection, and reduces redundancy through correlation constraints. This identifies key bands that maintain discriminative ability across multiple scenes, reducing computational load and improving feature reliability. Furthermore, this invention jointly extracts spatial spectral features, frequency domain features, and global semantic features. Frequency domain features are extracted using one-dimensional FFT to obtain the frequency energy distribution of spectral lines. These features are then concatenated with semantic features to generate an auxiliary representation, which is then concatenated with spatial spectral features and adaptively modulated by a gated channel attention module, improving recognition accuracy in complex environments. Additionally, this invention uses scene metadata to extract semantic vectors through a pre-trained large language model as global conditional constraints, ensuring the model's accuracy does not collapse due to spectral shifts in unseen scenes, significantly improving cross-scene generalization ability. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart of a hyperspectral oil spill identification method based on a joint spatial-spectral-frequency domain characterization provided by an embodiment of the present invention; Figure 2 This is an architecture diagram of a hyperspectral oil spill identification method based on a joint spatial-spectral-frequency domain characterization provided by an embodiment of the present invention; Figure 3 This is a diagram of a Transformer encoder cross-layer fusion architecture provided in an embodiment of the present invention; Figure 4 This is a block diagram of a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization provided in an embodiment of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0017] In embodiments of the present invention, words such as "exemplarily," "for example," etc., are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary" in the present invention should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the word "exemplary" is intended to present the concept in a concrete manner. Furthermore, in embodiments of the present invention, the meaning expressed by "and / or" can be both, or either one.

[0018] In the embodiments of this invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent. Similarly, the terms "of," "corresponding (relevant)," and "corresponding" may sometimes be used interchangeably. It should be noted that, without emphasizing the distinction between them, their intended meanings are consistent.

[0019] In this embodiment of the invention, sometimes a subscript such as W1 may be written in a non-subscript form such as W1. When the difference is not emphasized, the meaning they express is the same.

[0020] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0021] This invention provides a hyperspectral oil spill identification method using a joint spatial-spectral-frequency domain characterization approach. This method can be implemented using a hyperspectral oil spill identification device that utilizes this joint spatial-spectral-frequency domain characterization approach, which can be a terminal or a server. This embodiment proposes a hyperspectral oil spill identification network using a joint spatial-spectral-frequency domain characterization approach. This architecture aims to fully utilize multimodal data and prior knowledge of the scenario to solve the cross-scenario generalization problem of oil classification in complex marine environments. Figure 1 The flowchart of the hyperspectral oil spill identification method with joint spatial-spectral-frequency domain characterization is shown below. Figure 2 The diagram shows the architecture of a hyperspectral oil spill identification method that uses a combined spatial-spectral-frequency domain characterization. The processing flow of this method may include the following steps:

[0022] S1. Obtain the image data to be identified and the trained oil spill identification model.

[0023] The image data to be identified is hyperspectral cubic oil data, and the oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end.

[0024] In one feasible implementation, let the original hyperspectral data cube be... Where H and W are the spatial height and width, respectively, and L is the original number of bands; the spatial neighborhood is obtained with the center pixel to be classified as the center, and the original spectral vector is obtained from the same pixel.

[0025] Optionally, the training process for the oil spill detection model includes: S11. Obtain the training dataset. The training dataset contains hyperspectral cubic oil data from multiple collection scenarios and corresponding pixel-level category labels. It is divided into training set, validation set, and test set according to the collection scenario to ensure that samples from the same scenario do not appear in different sets.

[0026] S12. Input the samples in the training set into the initial oil spill identification model in batches, and output the predicted probability distribution of each pixel belonging to each category.

[0027] S13. Based on the predicted probability distribution and the corresponding true labels, the classification loss value is calculated using the cross-entropy loss function that combines class weights. The class weights are calculated based on the reciprocal frequency of each class of samples in the training set.

[0028] S14. With classification loss as the optimization objective, a cosine annealing learning rate scheduling strategy with a preheating mechanism is adopted. The learnable parameters of each branch and fusion module in the oil spill identification model are updated synchronously through backpropagation until the model converges.

[0029] S15. During training, early model stopping and hyperparameter adjustment are performed based on the classification accuracy on the validation set. After model convergence, the trained oil spill identification model is evaluated using a test set. The oil spill identification model that passes the evaluation is determined as the well-trained oil spill identification model. The evaluation uses overall classification accuracy, average classification accuracy, and Kappa coefficient as quantitative evaluation indicators.

[0030] In one feasible implementation, to comprehensively evaluate the model's cross-scene generalization ability in complex real-world environments, this invention abandons the pixel-by-pixel random sampling strategy commonly used in traditional hyperspectral classification for data partitioning. During the training phase, training, validation, and test sets are strictly divided according to scene, ensuring that pixels that are spatially highly adjacent and have highly similar spectral features in the same acquisition scene do not simultaneously leak into the training and test sets. This prevents the network from obtaining artificially high classification accuracy by "rote memorizing" scene background information, objectively reflecting the model's true recognition performance in unseen scenes.

[0031] Considering that the number of background water pixels in actual marine oil spill scenarios is usually far greater than the number of target oil pixels, and that the difficulty of obtaining samples for different oil categories (such as heavy oil, diesel, and gasoline) varies, a severe long-tail imbalance problem exists in the sample distribution. This invention uses a weighted cross-entropy loss function combined with class weights as the optimization objective, supplemented by L2 regularization to prevent overfitting during cross-scenario training. Assuming the current batch contains N pixels, the overall classification loss function is... The calculation formula is:

[0032] in, C Total number of categories; The class weights are calculated based on the reciprocal frequencies of each type of sample in the training set, and are used to increase the penalty weight of rare classes in the loss function; The i-th pixel's true label One-Hot encoding of the... c One component; λ represents the probability value predicted by the network in the aforementioned classification output strategy; λ is the weight decay coefficient. This represents the sum of squares of all learnable parameters in the network.

[0033] Regarding the training process, this invention employs the AdamW optimizer for end-to-end joint training of the multi-branch network. This is achieved through classification loss... The generated gradients are propagated backwards, synchronously updating the gating modulation parameters, frequency domain MLP mapping parameters, semantic LLM dimensionality reduction layer, and ViT spatial spectrum encoder parameters. For learning rate scheduling, a cosine annealing strategy with a warm-up mechanism is employed: a very small learning rate is used for warm-up in the early stages of training to prevent drastic gradient fluctuations caused by the initial concatenation of multimodal features from corrupting the pre-trained semantic weights; subsequently, the learning rate decays smoothly along a cosine curve, guiding the model to find the optimal parameter solution in the complex loss hyperplane composed of multi-source features until the network converges.

[0034] Finally, this invention defines a multi-dimensional quantitative evaluation process. In independent test sets, the overall accuracy (OA), average accuracy (AA), and Kappa coefficient, commonly used in hyperspectral remote sensing image classification, are employed as the main quantitative evaluation indicators. The overall accuracy evaluates the model's macroscopic classification accuracy across all pixels in the scene; the average accuracy primarily reflects the model's ability to identify easily confused and rare oil product categories at a microscopic level; and the Kappa coefficient measures the overall consistency between the classification results and the actual distribution of ground features. Furthermore, by combining a confusion matrix, the misclassification patterns of similar oil products (such as crude oil and heavy oil) under cross-scene conditions are further analyzed to comprehensively verify the generalization ability and application reliability of this invention.

[0035] S2. The hyperspectral cubic oil data is normalized according to the scenario-based standard normal variables. The normalized hyperspectral data is then subjected to band selection to obtain key band data.

[0036] In one feasible implementation, the network head receives the raw hyperspectral cube data. First, scene normalization is used to eliminate the overall spectral drift caused by different acquisition batches and lighting conditions. Then, the stable band selection module selects the most discriminative and noise-resistant key band data based on a comprehensive consideration of class separability, cross-scene perturbation and frequency band redundancy, providing high-quality input for subsequent feature learning.

[0037] During the band selection phase, this invention does not rely solely on "large variance in a single band" or "large differences in categories within a single band," but rather follows a joint screening principle of "strong class separability, low scene disturbance, low noise impact, and minimal band redundancy." Specifically, for each candidate band, the mean differences of different oil product categories on that band are first examined. If different categories exhibit significantly different response levels in that band, then that band has high class discrimination value; conversely, if multiple categories have similar responses in that band, then even if the band itself varies considerably, it is not suitable as a key band for oil product classification.

[0038] Optionally, the specific operation steps of S2 may include the following S21-S25: S21. For each scene included in the hyperspectral cubic oil data, determine the overall mean of all band data of all effective pixels, determine the standard deviation of the spectral data of all effective pixels based on the overall mean, and standardize each pixel based on the overall mean and standard deviation to obtain standardized hyperspectral data.

[0039] In one feasible implementation, the present invention first performs scene-based Standard Normal Variate (SNV) normalization on the raw hyperspectral data. Since hyperspectral oil data typically consists of multiple independent acquisition scenes, these scenes are easily affected by factors such as illumination conditions, sensor response differences, background water conditions, oil film thickness variations, and observation geometry. This can cause the same oil product to exhibit overall reflectance intensity shifts or spectral amplitude scale changes under different scenes, thereby reducing the model's ability to learn the intrinsic spectral features of the oil product. To reduce spectral drift caused by non-target factors between scenes, the present invention employs a scene-level SNV normalization strategy. Specifically, it uses a single acquisition scene as the statistical unit, calculates the global mean and standard deviation using the spectral responses of all valid pixels within that scene, and then performs uniform standardization processing on the spectra of all pixels within the scene based on these statistics.

[0040] Let the first s There are a total of [number] scenarios Each of the following is a valid cell, and each cell contains [number] valid cells. L One spectral band. n The original spectrum of each pixel is represented as follows: ,in L This represents the number of raw bands in the hyperspectral data. First, the overall mean of all band data for all valid pixels is calculated within each scene:

[0041] in, Indicates the first s The average spectral response level of each scene.

[0042] Then the standard deviation of the spectral data of all valid pixels in the scene was calculated. : Based on the above scene-level statistical parameters, the first [unclear] for each pixel in this scene i Standardize each band: in, This represents the spectral value after scene-level SNV processing. A minimal constant introduced to prevent numerical instability caused by excessively small standard deviation.

[0043] Unlike traditional pixel-by-pixel SNV methods, scene-level SNV does not calculate the mean and standard deviation independently for each pixel spectrum. Instead, it shares a unified normalization parameter within the same acquisition scene. Therefore, this method effectively reduces the overall spectral shift caused by variations in illumination intensity, sensor gain, and environmental conditions between different acquisition scenes, while preserving the relative spectral response differences between different oil types within the same scene. This allows subsequent classification models to focus more on spectral shape changes and band response characteristics related to oil type. After scene-level SNV processing, standardized hyperspectral data is obtained. This standardized data serves as input for subsequent band selection, feature extraction, and oil classification networks.

[0044] S22. Based on standardized hyperspectral data, for each candidate band, determine the average response of each category of samples in the candidate band and the overall average response of all samples in the candidate band, and calculate the inter-class dispersion of the categories based on the average response and the overall average response.

[0045] In one feasible implementation, let the first c Class of samples in the first i The average response across each band is All samples in the first i The overall average response across each band is Then, class separability can be determined by the degree of dispersion between classes. express: in, C For the number of categories, For the first c Number of samples per class N This represents the total number of samples. The larger the value, the more distinct the categories are in the [number of]th [item]. iThe more significant the difference in mean values ​​across different bands, the more effective that band is in distinguishing between different oil products and water backgrounds.

[0046] S23. For each candidate band, determine the mean offset of the same category of samples in different scenarios, and calculate the scenario disturbance based on the mean offset.

[0047] In one feasible implementation, high inter-class differences alone are insufficient to guarantee the stability and effectiveness of a band. For the same oil product, if its response varies significantly across different sampling scenarios, the band may be more sensitive to illumination, background water, sampling batch, or oil film state, rather than the oil product category itself. Therefore, this invention further calculates the mean shift of similar samples under different scenarios to describe the scenario sensitivity of the band. Let the... c Class in s The first scenario, the first i The mean value across each band is Then the scene disturbance amount It can be represented as:

[0048] in, Indicates the first c Class in s Number of samples in each scenario Indicates containing the first c The number of scenarios for each type of sample. The smaller the value, the more consistent the response of that band is across different scenarios for the same category, and therefore the band is more robust to changes in the scenario.

[0049] S24. For each candidate band, calculate the noise disturbance based on the degree of dispersion of similar samples relative to the scene mean within the same scene.

[0050] In one feasible implementation, if a certain wavelength band still exhibits significant random fluctuations within samples of the same type and scene, then that band may be significantly affected by sensor noise, local reflection anomalies, or pixel mixing. The noise perturbation can be constructed using the dispersion of samples of the same type and scene relative to their scene mean. :

[0051] in, The larger the value, the more susceptible the band is to random disturbances or anomalous responses, and its priority should be reduced in band selection.

[0052] S25. Calculate the stability separability score of each candidate band based on the inter-class dispersion, scene disturbance, and noise disturbance. Sort each candidate band according to its stability separability score. Calculate the correlation distance between any two candidate bands. Perform redundancy removal analysis on the sorted candidate bands based on the correlation distance. Select bands whose stability separability scores meet the preset conditions from the redundancy removal candidate bands to form a key band set.

[0053] In one feasible implementation, after obtaining class separability, scene perturbation, and noise perturbation, the present invention calculates a stable separability score for each band: in, and These are non-negative weights used to adjust the penalty level for scene disturbances and noise disturbances in band selection. This represents a constant introduced to prevent the denominator from being zero. The core idea of ​​this scoring system is to prioritize bands that can significantly differentiate between different oil product categories, remain stable across different scenarios, and are insensitive to random disturbances. Bands with higher scores are not simply those that exhibit the most dramatic changes, but rather those that are most valuable to oil product categories and possess cross-scenario robustness.

[0054] After sorting the candidate bands, this invention does not directly select the first one. K Instead of selecting the highest-scoring band, further redundancy removal analysis is performed. Since there is often strong correlation between consecutive hyperspectral bands, retaining multiple high-scoring bands that represent approximate information would increase input dimensionality without significantly improving discriminative power. For any two candidate bands... i and j Calculate the correlation distance between their spectral responses:

[0055] in, The Pearson correlation coefficient is used. and They represent the first i and the j The response sequence of each band on all training samples. When the absolute correlation coefficient of two candidate bands is greater than a preset threshold, it indicates that the information provided by the two bands is highly overlapping. In this invention, only the band with a higher stability and separability score is retained; or, adjacent highly correlated bands are aggregated into continuous band groups, and then representative bands with high scores, good intra-group centrality, and stable fluctuations are selected from each band group.

[0056] Finally, the key band set was obtained. B : in,K The number of original bands, the complexity of the classification task, and the performance on the validation set can be determined. SelectTopK ( Score (i) ) indicates in Score (i) Before selection K This key band set retains the most discriminative spectral information among different oil types and water backgrounds, while reducing the impact of scene variations, measurement noise, and redundancy of adjacent bands on subsequent models. The selected key band spectral lines will serve as important inputs for subsequent model branches.

[0057] S3. The keyband data is fed into the ViT spatial spectral feature extraction branch, the equally spaced reconstructed spectral lines determined based on the keyband set are fed into the frequency domain feature learning branch, and the scene metadata corresponding to the current observation is fed into the global semantic feature extraction branch. Spatial spectral features are obtained based on the ViT spatial spectral feature extraction branch, frequency domain features are obtained based on the frequency domain feature learning branch, and semantic features are obtained based on the global semantic feature extraction branch.

[0058] In one feasible implementation, the initial spectral data is used to perform in-depth analysis of oil pixels from different information dimensions.

[0059] Optionally, the specific operation steps of S3 may include the following S31-S33: S31. Based on the ViT spatial spectral feature extraction branch, a spectral sliding window operation is performed on the spatial neighborhood pixels to jointly flatten the extracted three-dimensional local spatial spectral blocks. The jointly flattened spatial spectral blocks are then input into the ViT network, which includes a transformer encoding and a cross-layer fusion mechanism, to obtain spatial spectral features.

[0060] In one feasible implementation, this embodiment sets up a ViT spatial spectral feature extraction branch to extract spatial texture, local spectral shape, and cross-band correlation features from the central pixel and its neighborhood. Unlike the conventional method of directly inputting the fixed spatial block into the Transformer after flattening it as a whole, the first innovation of this branch is the adoption of a spatial spectral token construction method of "fixed spatial neighborhood and sliding spectral window".

[0061] Optionally, the specific operation steps of S31 may include the following S311-S316: S311. Using the pixel to be classified as the center, extract a portion of size [size missing]. P × P The spatial neighborhood is defined for each band in the selected set of key bands, and a spatial neighborhood is constructed along the spectral dimension centered on the current band. r Local empty spectral blocks of adjacent bands, which simultaneously contain local spatial context and local continuous spectral band information.

[0062] In one feasible implementation, each token no longer corresponds to only a spatial image patch, but corresponds to a three-dimensional spatial spectral unit that simultaneously contains local spatial context and local continuous spectral bands.

[0063] Let the center band be the th i Each retained band, and its corresponding empty spectral block, can be represented as: in, The coordinates within the spatial neighborhood are ( u , v The pixel in the first position j Normalized response values ​​for each band. When the window exceeds the beginning or end of the spectral sequence, boundary replication padding is used to ensure that each token has a consistent spectral length.

[0064] Therefore, each token retains the two-dimensional spatial structure surrounding the central pixel, capable of describing the internal texture of oil film patches, mixed pixels at the oil-water interface, and local reflectance variations. Simultaneously, each token contains a local continuous spectral band centered on the target band, preserving absorption valleys, spectral line slopes, peak and valley positions, and information on the coordinated changes of adjacent bands. Thus, this invention does not simply extract and fuse spatial and spectral information separately, but rather establishes a coupling relationship between local spatial structure and local spectral shape during the token construction stage. This token setting enhances the model's adaptability to variations in oil film thickness, edge mixing, and background water disturbances.

[0065] S312. Flatten each local empty spectral block in a preset order, convert it into a d-dimensional embedding vector through linear mapping, add a learnable classification token to the beginning of the embedding vector sequence, and superimpose a learnable position code to obtain the input sequence of the transformer encoder.

[0066] In one feasible implementation, each spatial spectral block is flattened in a preset order and converted into a d-dimensional embedding vector via linear mapping. Assuming the keyband set B contains K keybands, the aforementioned spatial spectral sequence is formed with each pixel and each band as the center, thus ultimately forming K spatial spectral tokens. A learnable classification token is added to the beginning of the token sequence, and a learnable positional encoding is superimposed to obtain the input to the Transformer encoder. The positional encoding is used to maintain the relative order of the spatial spectral tokens in the sequence, enabling the network to identify the band positions corresponding to different tokens and their spatial spectral relationships.

[0067] in, Indicates the first i The d-dimensional embedding vector is obtained by flattening and linearly mapping the local spatial spectral block formed by the central pixels. Represents a learnable category token. This represents a learnable location code. This represents the initial token sequence input to the Transformer encoder. This indicates concatenation along the token sequence dimension.

[0068] S313. Input the input sequence sequentially into the transformer encoder, which consists of a multi-head self-attention module and a feedforward network module. Calculate the attention weights between each spatial spectrum token through the multi-head self-attention mechanism. Sum the value vectors of each spatial spectrum token according to the attention weights to obtain the current layer output features that integrate the long-range dependencies between different spatial spectrum tokens.

[0069] In one feasible implementation, the token sequence is then sequentially input into a Transformer encoder consisting of a multi-head self-attention module and a feedforward network module. The multi-head self-attention mechanism can adaptively establish dependencies between local spatial regions, different spectral bands, and more distant bands based on the correlation between different spatial spectral tokens, thereby enhancing the ability to express subtle differences in oil product categories. The encoder employs layer normalization and residual connections to ensure the stability of the deep network training process.

[0070] S314. The output features of the current layer are concatenated with the historical output features of the two layers in between along the feature dimension. The concatenated features are then fused with cross-layer information through a learnable convolutional mapping to obtain cross-layer fused features.

[0071] To avoid the gradual weakening of shallow details caused by traditional Transformers using only the output of the last layer, this invention innovatively sets up a cross-layer Transformer information fusion mechanism. Starting from the preset encoding layer, the input features of the current layer are concatenated with the historical features of the previous layer, and then feature compression, weight recalibration, and dimension restoration are performed through learnable convolutional fusion units. This process allows local edges, fine-grained textures, and short-range spectral variations extracted by the shallow network to directly participate in deep semantic expression, while the global correlation information obtained by the deep network inversely enhances the discriminative power of local features.

[0072] For example, the l Layer-to-layer integration, such as Figure 3 As shown, the cross-layer fusion feature can be represented as: in, For the input features of the current coding layer, For historical output features at intervals of two layers, This indicates concatenation along the feature dimension. This represents a learnable convolutional mapping used for cross-layer information fusion. This cross-layer fusion is not a simple superposition of outputs from different layers, but rather an adaptive selection of the contribution ratio between shallow local information and deep abstract information through learnable parameters.

[0073] S315. Use the cross-layer fusion features as the input sequence for the next coding layer, and return to execute S313 to S315 until all coding layers have been traversed.

[0074] S316. After multi-layer transformer encoding and cross-layer fusion, the final output corresponding to the classification token is taken as the spatial spectral feature.

[0075] In one feasible implementation, after multi-layer Transformer encoding and cross-layer fusion, the final output corresponding to the classification token is taken as the spatial spectral feature: in, For the number of layers in the Transformer encoder, The spatial spectral features of the central pixel are used. These features simultaneously reflect the local spatial distribution of oil targets, continuous changes in adjacent bands, cross-spectral dependencies, and spatial spectral semantic information at different levels. They also serve as the basic input for subsequent frequency domain feature learning, empirical auxiliary representation, and multi-branch fusion classification.

[0076] S32. Based on the frequency domain feature learning branch, the effective spectral range for frequency domain analysis is determined according to the key band set. The original standardized continuous spectrum of the center pixel within the effective spectral range is extracted and resampled at equal intervals. One-dimensional fast Fourier transform is performed on the obtained reconstructed spectrum lines at equal intervals to calculate the normalized power spectrum and perform multi-scale frequency band energy encoding to characterize the overall trend and local fluctuation of the oil spectrum lines, thereby obtaining the frequency domain features of a single pixel.

[0077] In one feasible implementation, to compensate for the shortcomings of the spatial spectral feature branch, which mainly focuses on spatial neighborhood structure and local spectral band correlation, this embodiment of the invention sets up a frequency domain feature learning branch to model the overall fluctuation pattern, local peak and valley variations, and spectral structure at different scales of the single-pixel spectral curve. The class differences contained in the oil product spectral curve are not only reflected in the reflectance level of a certain band, but also in the overall trend of the spectral lines, the broad absorption structure, local absorption peaks, and the differences in the rate of change between adjacent bands. After mapping the spectral curve to the frequency domain, the above-mentioned spectral differences can be characterized from the perspective of energy distribution, providing information complementary to the spatial spectral features for oil product classification.

[0078] Optionally, the specific operation steps of S32 may include the following S321-S325: S321. For the center pixel, based on the key band set obtained through stable and separable band selection, the effective spectral range for frequency domain analysis is determined, and the original normalized continuous spectral response within the effective spectral range is extracted. Then, based on the actual wavelength coordinates corresponding to the original normalized continuous spectral response, an equally spaced resampling grid is constructed, and equally spaced reconstructed spectral lines guided by the key bands are obtained through interpolation.

[0079] In one feasible implementation, for the center pixel, a spectral vector arranged in the original wavelength order is obtained after stable and separable band selection. Considering that the key bands may be discontinuous or unevenly spaced along the original wavelength axis, this invention does not directly use the spectral vector for the Fast Fourier Transform. Instead, it uses the minimum center wavelength of the key band set. and maximum center wavelength Determine the effective spectral range. Extract the original normalized continuous spectrum of the center pixel within this range, and construct an equally spaced sampling grid of length M according to the actual wavelength coordinates. Obtain the equally spaced reconstructed spectral lines through linear interpolation. This serves as the input for the subsequent one-dimensional fast Fourier transform. The resampling length M is determined based on the wavelength span of the effective spectral range and the preset resampling interval. Determined, that is .

[0080] S322. Perform a one-dimensional real fast Fourier transform on the equally spaced reconstructed spectral lines to obtain the complex spectral coefficients at each frequency position, and calculate the power spectral value corresponding to each frequency position.

[0081] In one feasible implementation, a one-dimensional real-valued fast Fourier transform is performed on the equally spaced reconstructed spectral lines to obtain the complex spectral coefficients at each frequency position, and then their power spectra are calculated. The calculation form is as follows:

[0082] in, In the equally spaced reconstructed spectral lines, the first... n The response value at each resampling location, For the first k spectral coefficients at each frequency position, This represents the corresponding power spectrum value. Since the input spectrum is a real number sequence, a real-number fast Fourier transform can be used in actual calculations to retain only the non-negative frequency components, thereby reducing the computational load.

[0083] S323. Remove the zero-frequency component and perform L1 normalization on the power spectrum values ​​of the remaining non-negative frequency components to obtain the normalized power spectrum vector.

[0084] In one feasible implementation, the zero-frequency component mainly reflects the overall mean or overall brightness level of the spectral curve, and is easily affected by changes in light intensity, oil film thickness, and overall background reflectivity. To make the network more attentive to the shape and relative changes in spectral structure, this invention removes the zero-frequency component and performs L1 normalization on the remaining power spectrum.

[0085] in, This refers to the number of non-negative frequency components retained after removing zero-frequency terms. For the normalized power spectrum, To prevent extremely small constants with a denominator of zero, normalization is used. The frequency domain characteristics between different pixels mainly reflect the relative energy distribution of each frequency component, rather than the absolute signal strength, which helps to reduce the impact of cross-scene brightness variations and overall amplitude scaling.

[0086] S324. Divide the normalized power spectrum vector into low-frequency, mid-frequency and high-frequency intervals according to the frequency index, and calculate the sum of the power spectrum values ​​in each frequency interval to obtain the low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio.

[0087] In one feasible implementation, to enable the network to simultaneously obtain spectral shape change information at different scales, this invention divides the normalized power spectrum into low-frequency, mid-frequency, and high-frequency intervals according to frequency index, and calculates the energy proportion of each frequency band. The low-frequency components mainly characterize the overall trend of the spectral lines, the broad absorption structure, and the slowly changing reflectance background; the mid-frequency components mainly characterize the relatively stable spectral shape curvature and gradual differences between different oil products; and the high-frequency components reflect local peaks and valleys, abrupt edge changes, fine-grained spectral shape fluctuations, and residual perturbations. The energy percentage of each frequency band can be expressed as:

[0088] in, Show the first The set of frequency indices contained in each frequency range This indicates the normalized energy percentage of the frequency band. The boundaries of the frequency range can be preset based on the spectrum length or determined through validation results from the training set.

[0089] S325. The normalized power spectral vector is concatenated with the low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio. The concatenated features are then input into a multilayer perceptron for nonlinear encoding to obtain frequency domain features.

[0090] In one feasible implementation, a fine-grained normalized power spectral vector is combined with low-frequency energy characteristics. Mid-frequency energy characteristics High-frequency energy characteristics The data are concatenated and then input into a multilayer perceptron for nonlinear encoding to obtain frequency domain features. : in, Normalized power spectral vector. This network can adaptively learn the contribution of different frequency components to oil categories based on training data, without fixing the energy rules of a certain frequency band to an artificial threshold. The resulting frequency domain features. With ViT spatial spectral characteristics They complement each other: the former focuses on describing the energy distribution and oscillation structure of single-pixel spectral lines at different frequency scales, while the latter focuses on describing the spatial neighborhood texture and the coupling relationship of local spectral segments. Together, they improve the separability of different oil categories in complex scenarios.

[0091] S33. Based on the global semantic feature extraction branch, extract the scene metadata of the current observation scene, convert the extracted information into natural language prompts through structured prompt encoding, and input it into the pre-trained large language model to extract semantic features with global constraints.

[0092] In one feasible implementation, to address the spectral shift problem arising from different acquisition batches, background environments, and observation conditions of hyperspectral oil data, this invention innovatively introduces a global semantic feature extraction branch based on a Large Language Model (LLM). This branch aims to transform unstructured prior information about the scene into computable structured high-dimensional semantic vectors, providing global conditional constraints for subsequent feature fusion, thereby improving the model's generalization ability in complex cross-scene environments.

[0093] Optionally, the specific operation steps of S33 may include the following S331-S334: S331. Extract the global metadata of the current hyperspectral image. The global metadata includes sensor type and band range, illumination and weather conditions, background water type, and candidate oil categories included in the current task.

[0094] In one feasible implementation, since global information such as sensor status, lighting conditions, and background water turbidity is difficult to directly reflect through the spectrum or local spatial texture of a single pixel, this invention first extracts global metadata from the current hyperspectral image. Let the extracted scene metadata set be... ,in Indicates the sensor type and frequency band range. This indicates environmental conditions such as lighting and weather. Indicates the type of background water (e.g., clear seawater, turbid nearshore water, etc.). This indicates the candidate oil categories included in the current task.

[0095] S332. Using a preset natural language prompt template, the discrete variables in the global metadata are mapped into coherent scene description text.

[0096] In one feasible implementation, a preset natural language prompt template is used to map the aforementioned discrete variables into coherent text descriptions. The text mapping process can be represented as follows:

[0097] in, This represents a text template generation function. T This is the string used to describe the scene being generated. For example, the generated text could be: "This hyperspectral image was acquired by a drone equipped with a shortwave infrared sensor under strong light conditions. The background is turbid water. The classification task is to distinguish between heavy oil, diesel, and seawater."

[0098] S333. Input the scene description text into the pre-trained large language model encoder, extract the global representation vector of the last layer of the large language model, and obtain the original semantic features.

[0099] In one feasible implementation, after obtaining the structured text T, it is input into a pre-trained large language model (such as BERT or RoBERTa) for feature extraction. Since the pre-trained language model has learned rich semantic representation capabilities on massive amounts of text data and can deeply understand the contextual relationships in environmental descriptions, it can be used to extract deep scene semantics. To preserve the global syntactic and semantic information of the text, this invention extracts the global representation vector of the last layer of the LLM (e.g., the output vector of the [CLS] label or the average pooling of all word vectors) to obtain the original semantic features:

[0100] in, This refers to a large language model encoder with frozen weights or fine-tuned parameters. For dimension A high-dimensional semantic vector.

[0101] S334. Input the original semantic features into a multilayer perceptron, and project them onto a feature space that matches the spatial spectral features and frequency domain features through nonlinear mapping to obtain global semantic features.

[0102] In one feasible implementation, since the dimensionality of LLM outputs is typically high, directly performing multimodal fusion would introduce excessive computational overhead and may lead to dimensionality mismatch. Therefore, this invention utilizes a multilayer perceptron (MLP) to fuse the original semantic features... Mapping to a feature space that matches the spatial spectral features and frequency domain features yields the final global semantic features. :

[0103] in, and These are the learnable weight matrix and bias vector, respectively. Through this branch, the "acquisition environment" and "task constraints," which are originally difficult to quantify in pure visual networks, are successfully encoded as numerical features. .

[0104] S4. Combine the frequency domain features with the semantic features, concatenate the generated auxiliary expression with the spatial spectrum features, and input the resulting concatenated features into the gated channel attention module to obtain the optimized discriminative features.

[0105] In one feasible implementation, to achieve adaptive collaboration of heterogeneous features, the outputs of each branch enter a gated fusion network. First, frequency domain features reflecting microscopic physical oscillations are combined with semantic features reflecting prior macroscopic environments to generate an auxiliary expression guiding subsequent optimization. Next, this auxiliary expression is concatenated with the spatial-spectral features extracted from the main visual branch. Finally, the concatenated features are input into the gated channel attention module, which dynamically evaluates the reliability weights of each feature channel in the current scene, achieving adaptive modulation and denoising of the joint features, and outputting the final optimized discriminative features.

[0106] To address the problem in existing hyperspectral classification methods that rely on simple splicing or fixed-weight fusion of multi-source features, making it difficult to dynamically adjust the reliability of feature branches based on the current pixel's oil film boundary state, background interference level, and scene conditions, this invention designs an adaptive multi-feature fusion and feature representation optimization module. This module strictly follows the logical architecture of "auxiliary information extraction - multi-source splicing - gated adaptive optimization," fully leveraging the complementary advantages of features from different dimensions.

[0107] This invention first analyzes the frequency domain features of a single pixel. Global semantic features obtained from large language models The data is then stitched together. Frequency domain features reflect the physical oscillation patterns of spectral line energy, while global semantic features provide contextual constraints for the scene acquisition environment. This invention combines the two through a nonlinear mapping network to construct an "auxiliary spectral expression" vector that incorporates empirical semantic information. This vector acts as a priori gating signal, and its calculation process can be expressed as follows:

[0108] in, This represents the concatenation (Concat) of frequency domain features and semantic features along the channel dimension. and These are the learnable weight matrix and the bias, respectively. This is a non-linear activation function (such as GELU). This step aligns the abstract scene semantics with the microscopic frequency physics, forming a contextual prior that guides feature optimization.

[0109] In obtaining prior auxiliary features Then, it was compared with ViT spatial-spectral features containing rich local textures and cross-band dependencies. The features are then stitched together again to form a joint feature space for multimodal fusion: At this point, joint features Although it includes spatial, spectral, frequency, and global scene semantic information, the reliability of information in each dimension is not equal under different interference conditions, and direct classification is easily affected by weakly correlated features.

[0110] Therefore, to achieve adaptive fusion, this invention introduces an adaptive gated channel attention mechanism for feature representation optimization. This mechanism utilizes joint features. It autonomously learns the "reliability weights" of different feature channels in the current scenario and uses them to recalibrate the features.

[0111] First, a dimensionally consistent dynamic weight mask is generated using a multilayer perceptron. : in, , , , For network parameters, The Sigmoid activation function maps the weights of each channel to the (0,1) interval, representing the reliability of the feature channel under the current oil film state and background conditions.

[0112] Subsequently, the mapped joint features are modulated using element-wise multiplication with this gated weight, and spatial spectral features are introduced. As residual connections, they prevent the loss of backbone visual information during the optimization process: in, This is the Hadama product (element-wise multiplication). , The linear layer parameters are for feature dimension recovery. This refers to the final discriminative features after multi-feature fusion and expression optimization.

[0113] Through the aforementioned optimization mechanism of "feature concatenation—weight evaluation—dynamic modulation," the network can adaptively determine which channels in the current spatial spectrum features are noise-contaminated redundant information and which are key discriminative information based on the scenario conditions provided by LLM and the frequency domain state provided by FFT, thereby completing the adaptive optimization of feature representation. Optimized features The data will be directly input into the Classification Head for final oil classification output.

[0114] S5. Input the optimized discrimination features into the classification output terminal, and output the oil spill identification result based on the mask and confidence decision mechanism.

[0115] In one feasible implementation, the discriminative features are optimized and input into the classification output. First, the high-dimensional features are mapped to the category space through the classification head, and the category probability distribution of each oil product is calculated. To adapt to the actual operational needs of marine oil spill monitoring, a mask and confidence decision mechanism is introduced in the final step. Invalid backgrounds are filtered out by a preset spatial mask, and low-reliability strongly mixed pixels are truncated using a confidence threshold. Finally, a high-precision pixel-level oil product category map is output.

[0116] After multi-feature fusion and feature representation optimization, the resulting fused features The system has fully integrated local spatial-spectral joint information, single-pixel frequency domain oscillation patterns, and global scene semantic constraints. The classification output strategy in this stage aims to map this high-dimensional feature vector to a specific oil product category and combine it with the actual spatial attributes of the remote sensing image for decision-making and background filtering. Specifically, it includes three steps: classification head mapping, probability transformation, and confidence mask decision-making.

[0117] Optionally, the specific operation steps of S5 may include the following S51-S54: S51. Input the optimized discriminative features into the classification head, and then map them to the category space through layer normalization and fully connected linear layers to obtain the log probability vectors of each category.

[0118] In one feasible implementation, the optimized features are... The input is fed into the Classification Header. To prevent feature distribution shift and accelerate network convergence, the features are first normalized using Layer Normalization, and then their dimensions are mapped to the class space through a fully connected linear layer to obtain the log probability vector Z of the predicted class.

[0119] S52. The Softmax function is used to normalize the logarithmic probability vector to obtain the predicted probability distribution of each pixel belonging to each category.

[0120] In one feasible implementation, the unbounded logarithmic probability is transformed into a statistically meaningful probability distribution, and the Softmax function is used to... Z Perform normalization. Calculate the... c Predicted probabilities for each category :

[0121] The resulting probability distribution vector It reflects the network's confidence level in the current pixel belonging to each category.

[0122] S53. Pre-import land and cloud non-water space masks. For invalid pixels within the space mask, output as background category.

[0123] S54. For valid water body pixels, set a confidence threshold and determine whether the maximum predicted probability is greater than or equal to the confidence threshold. If so, take the category corresponding to the maximum predicted probability as the oil spill identification result of the valid water body pixel. If not, classify the valid water body pixel into the unknown mixed category or the background category.

[0124] In one feasible implementation, in complex marine oil spill scenarios, there are often strongly mixed pixels due to ship tracks, flare reflections, or extremely thin oil films. Using only the conventional maximum probability output can easily lead to false alarms in low-confidence areas. Therefore, this invention introduces a joint decision-making strategy at the output end: "confidence threshold truncation" and "spatial effective mask." On the one hand, before classification inference, a non-water body mask (such as land / clouds) is pre-imported. Invalid pixels within the mask are directly output as background, avoiding unnecessary computational waste. On the other hand, a confidence threshold is set for effective water body pixels. Only when the maximum predicted probability is greater than or equal to this threshold is it classified as the corresponding oil category; otherwise, it is classified as "unknown mixed category" or "background," minimizing the false alarm rate of oil spills. The final output category discrimination rule can be expressed as:

[0125] in, For preset background or unknown category labels, The confidence threshold is determined based on experience or a validation set. This classification output strategy not only completes end-to-end class prediction but also effectively improves the reliability of hyperspectral classification results in actual marine monitoring operations.

[0126] This invention employs scene-level SNV normalization to eliminate cross-scene spectral shifts. It uses class separability as a positive objective and cross-scene drift and noise sensitivity as penalty terms for band selection, and reduces redundancy through correlation constraints. This identifies key bands that maintain discriminative ability across multiple scenes, reducing computational load and improving feature reliability. Furthermore, this invention jointly extracts spatial spectral features, frequency domain features, and global semantic features. Frequency domain features are extracted using one-dimensional FFT to obtain the frequency energy distribution of spectral lines. These features are then concatenated with semantic features to generate an auxiliary representation, which is then concatenated with spatial spectral features and adaptively modulated by a gated channel attention module, improving recognition accuracy in complex environments. Additionally, this invention uses scene metadata to extract semantic vectors through a pre-trained large language model as global conditional constraints, ensuring the model's accuracy does not collapse due to spectral shifts in unseen scenes, significantly improving cross-scene generalization ability.

[0127] Figure 4 This is a block diagram of a hyperspectral oil spill identification device based on a spatial-spectral-frequency domain joint characterization method provided in an embodiment of the present invention. The device is used for hyperspectral oil spill identification based on this method. (Refer to...) Figure 4 The device includes an acquisition unit 410, a preprocessing unit 420, a feature extraction unit 430, a fusion unit 440, and a recognition unit 450. Wherein:

[0128] The acquisition unit 410 is used to acquire the image data to be identified and the trained oil spill identification model. The image data to be identified is hyperspectral cubic oil data. The oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end. The preprocessing unit 420 is used to perform scene-based standard normalization processing on hyperspectral cubic oil data, and to select bands on the normalized hyperspectral data to obtain key band data. The feature extraction unit 430 is used to feed keyband data into the ViT spatial spectral feature extraction branch, feed equally spaced reconstructed spectral lines determined based on the keyband set into the frequency domain feature learning branch, and feed scene metadata corresponding to the current observation into the global semantic feature extraction branch. Spatial spectral features are obtained based on the ViT spatial spectral feature extraction branch, frequency domain features are obtained based on the frequency domain feature learning branch, and semantic features are obtained based on the global semantic feature extraction branch.

[0129] The fusion unit 440 is used to combine frequency domain features with semantic features, concatenate the generated auxiliary expression with the spatial spectrum features, and input the obtained concatenated features into the gated channel attention module to obtain optimized discriminative features; The identification unit 450 is used to input optimized discrimination features into the classification output end and output oil spill identification results based on the mask and confidence decision mechanism.

[0130] Figure 5 This is a schematic diagram of the structure of a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization provided in an embodiment of the present invention, as shown below. Figure 5 As shown, a hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization can include the above-mentioned Figure 4 The illustrated hyperspectral oil spill identification device features a combined spatial-spectral-frequency domain characterization. Optionally, the hyperspectral oil spill identification device 510 with combined spatial-spectral-frequency domain characterization may include a first processor 2001.

[0131] Optionally, the hyperspectral oil spill identification device 510, which features a combined spatial-spectral-frequency domain characterization, may also include a memory 2002 and a transceiver 2003.

[0132] The first processor 2001, memory 2002, and transceiver 2003 can be connected via a communication bus.

[0133] The following is combined Figure 5 The components of the hyperspectral oil spill identification device 510, characterized by joint spatial-spectral-frequency domain representation, are described in detail below: The first processor 2001 is the control center of the hyperspectral oil spill identification device 510, which is characterized by joint spatial-spectral-frequency domain representation. It can be a single processor or a collective term for multiple processing elements. For example, the first processor 2001 can be one or more central processing units (CPUs), application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0134] Optionally, the first processor 2001 can perform various functions of the hyperspectral oil spill identification device 510 by running or executing software programs stored in the memory 2002 and calling data stored in the memory 2002.

[0135] In a specific implementation, as one example, the first processor 2001 may include one or more CPUs, for example... Figure 5 CPU0 and CPU1 are shown in the diagram.

[0136] In a specific implementation, as one example, the hyperspectral oil spill identification device 510, which uses a combined spatial-spectral-frequency domain characterization, may also include multiple processors, for example... Figure 5 The first processor 2001 and the second processor 2004 are shown in the diagram. Each of these processors can be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor can refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0137] The memory 2002 is used to store the software program that executes the present invention, and is controlled by the first processor 2001 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0138] Optionally, the memory 2002 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 2002 may be integrated with the first processor 2001 or exist independently, and may be connected via the interface circuit of the hyperspectral oil spill identification device 510, which is characterized by a joint spatial-spectral-frequency domain. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0139] The transceiver 2003 is used to communicate with network devices or with terminal devices.

[0140] Alternatively, transceiver 2003 may include a receiver and a transmitter. Figure 5 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0141] Optionally, the transceiver 2003 can be integrated with the first processor 2001 or exist independently, and can be connected to the interface circuit of the hyperspectral oil spill identification device 510, which is characterized by joint spatial-spectral-frequency domain characteristics. Figure 5 (Not shown in the image) is coupled to the first processor 2001, and this embodiment of the invention does not specifically limit this.

[0142] It should be noted that, Figure 5 The structure of the hyperspectral oil spill identification device 510 with spatial-spectral-frequency domain joint characterization shown in the figure does not constitute a limitation on the router. Actual hyperspectral oil spill identification devices with spatial-spectral-frequency domain joint characterization may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0143] Furthermore, the technical effect of the hyperspectral oil spill identification device 510 with joint spatial-spectral-frequency domain characterization can be referred to the technical effect of the hyperspectral oil spill identification method with joint spatial-spectral-frequency domain characterization described in the above method embodiments, and will not be repeated here.

[0144] It should be understood that the first processor 2001 in the embodiments of the present invention may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or it may be any conventional processor, etc.

[0145] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0146] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0147] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0148] In this invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0149] It should be understood that, in various embodiments of the present invention, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0150] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0151] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0152] In the several embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0153] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0154] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0155] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A hyperspectral oil spill identification method using spatial-spectral-frequency domain joint characterization, characterized in that, The method includes: S1. Obtain the image data to be identified and the trained oil spill identification model. The image data to be identified is hyperspectral cubic oil data. The oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end. S2. Perform scene-based standard normalization on the hyperspectral cubic oil data, and select bands on the normalized hyperspectral data to obtain key band data. S3. Send the key band data to the ViT spatial spectral feature extraction branch, send the equally spaced reconstructed spectral lines determined based on the key band set to the frequency domain feature learning branch, and send the scene metadata corresponding to the current observation to the global semantic feature extraction branch. Obtain spatial spectral features based on the ViT spatial spectral feature extraction branch, obtain frequency domain features based on the frequency domain feature learning branch, and obtain semantic features based on the global semantic feature extraction branch. S4. Combine the frequency domain features with the semantic features, concatenate the generated auxiliary expression with the spatial spectrum features, and input the resulting concatenated features into the gated channel attention module to obtain the optimized discriminative features; S5. Input the optimized discrimination features into the classification output terminal, and output the oil spill identification result based on the mask and confidence decision mechanism.

2. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 1, characterized in that, The S2 process performs scene-based standard normalization on the hyperspectral cubic oil data, and then selects bands from the normalized hyperspectral data to obtain key band data, including: S21. For each scene included in the hyperspectral cubic oil data, determine the overall mean of all band data of all effective pixels, determine the standard deviation of the spectral data of all effective pixels based on the overall mean, and standardize each pixel based on the overall mean and standard deviation to obtain standardized hyperspectral data. S22. Based on standardized hyperspectral data, for each candidate band, determine the average response of each category of samples in the candidate band and the overall average response of all samples in the candidate band, and calculate the inter-class dispersion of the category based on the average response and the overall average response. S23. For each candidate band, determine the mean offset of the same category of samples in different scenarios, and calculate the scenario disturbance based on the mean offset. S24. For each candidate band, calculate the noise disturbance based on the degree of dispersion of similar samples relative to the scene mean within the same scene. S25. Calculate the stability separability score of each candidate band based on the inter-class dispersion, scene disturbance, and noise disturbance. Sort each candidate band according to its stability separability score. Calculate the correlation distance between any two candidate bands. Perform redundancy removal analysis on the sorted candidate bands based on the correlation distance. Select bands whose stability separability scores meet preset conditions from the redundancy-removed candidate bands to form a key band set.

3. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 1, characterized in that, S3 involves sending keyband data into the ViT spatial spectral feature extraction branch, sending the equally spaced reconstructed spectral lines determined based on the keyband set into the frequency domain feature learning branch, and sending the scene metadata corresponding to the current observation into the global semantic feature extraction branch. Spatial spectral features are obtained based on the ViT spatial spectral feature extraction branch, frequency domain features are obtained based on the frequency domain feature learning branch, and semantic features are obtained based on the global semantic feature extraction branch, including: S31. Based on the ViT spatial spectral feature extraction branch, a spectral sliding window operation is performed on the spatial neighborhood pixels. The extracted three-dimensional local spatial spectral blocks are jointly flattened. The jointly flattened spatial spectral blocks are input into the ViT network containing transformer coding and cross-layer fusion mechanism to obtain spatial spectral features. S32. Based on the frequency domain feature learning branch, the effective spectral range for frequency domain analysis is determined according to the key band set. The original normalized continuous spectrum of the center pixel within the effective spectral range is extracted and resampled at equal intervals. One-dimensional fast Fourier transform is performed on the obtained reconstructed spectrum, the normalized power spectrum is calculated and multi-scale frequency band energy encoding is performed to characterize the overall trend and local fluctuation of the oil spectrum, and the frequency domain features of a single pixel are obtained. S33. Based on the global semantic feature extraction branch, extract the scene metadata corresponding to the current observation, encode the structured scene metadata into scene description text through a preset prompt template, and input it into the pre-trained language model encoder to extract semantic features with global condition constraints.

4. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 3, characterized in that, The ViT-based spatial spectral feature extraction branch of S31 performs a spectral sliding window operation on the spatial neighborhood pixels, jointly flattens the extracted three-dimensional local spatial spectral blocks, and inputs the jointly flattened spatial spectral blocks into a ViT network containing transformer coding and cross-layer fusion mechanisms to obtain spatial spectral features, including: S311. Using the pixel to be classified as the center, extract a portion of size [size missing]. P × P The spatial neighborhood is defined for each band in the selected set of key bands, and a spatial neighborhood is constructed along the spectral dimension centered on the current band. r Local spatial spectral blocks of adjacent bands, wherein the local spatial spectral blocks simultaneously contain local spatial context and local continuous spectral band information; S312. Flatten each local empty spectral block in a preset order, convert it into a d-dimensional embedding vector through linear mapping, add a learnable classification token to the beginning of the embedding vector sequence, and superimpose a learnable position code to obtain the input sequence of the converter encoder. S313. The input sequence is sequentially input into the converter encoder composed of a multi-head self-attention module and a feedforward network module. The attention weight between each spatial spectrum token is calculated through the multi-head self-attention mechanism. The value vectors of each spatial spectrum token are weighted and summed according to the attention weight to obtain the current layer output feature that integrates the long-range dependencies between different spatial spectrum tokens. S314. The output features of the current layer are concatenated with the historical output features of the two layers in between along the feature dimension. The concatenated features are then fused with cross-layer information through a learnable convolutional mapping to obtain cross-layer fused features. S315. Use the cross-layer fusion feature as the input sequence of the next coding layer, and return to execute S313 to S315 until all coding layers are traversed. S316. After multi-layer transformer encoding and cross-layer fusion, the final output corresponding to the classification token is taken as the spatial spectral feature.

5. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 3, characterized in that, The frequency domain feature learning branch of S32 determines the effective spectral range for frequency domain analysis based on the key band set, extracts the original normalized continuous spectrum of the center pixel within the effective spectral range, and performs equal-interval resampling. A one-dimensional fast Fourier transform is performed on the obtained equally spaced reconstructed spectral lines to calculate the normalized power spectrum and perform multi-scale frequency band energy coding to characterize the overall variation trend and local fluctuation patterns of the oil spectral lines, obtaining the frequency domain features of a single pixel, including: S321. For the center pixel, the effective spectral range for frequency domain analysis is determined based on the key band set obtained by stable and separable band selection. The original normalized continuous spectral response within the effective spectral range is extracted. Based on the actual wavelength coordinates corresponding to the original normalized continuous spectral response, an equally spaced resampling grid is constructed. The equally spaced reconstructed spectral lines guided by the key band are obtained by interpolation. S322. Perform a one-dimensional real fast Fourier transform on the equally spaced reconstructed spectral lines to obtain the complex spectral coefficients at each frequency position, and calculate the power spectral value corresponding to each frequency position. S323. Remove the zero-frequency component and perform L1 normalization on the power spectrum values ​​of the remaining non-negative frequency components to obtain the normalized power spectrum vector. S324. Divide the normalized power spectrum vector into low-frequency interval, mid-frequency interval and high-frequency interval according to the frequency index, and calculate the sum of power spectrum values ​​in each frequency interval to obtain the low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio. S325. The normalized power spectrum vector is concatenated with the low-frequency energy ratio, mid-frequency energy ratio and high-frequency energy ratio. The concatenated features are then input into a multilayer perceptron for nonlinear encoding to obtain frequency domain features.

6. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 3, characterized in that, The S33 branch, based on global semantic feature extraction, extracts the currently observed scene metadata, converts the extracted information into natural language prompts via structured prompt encoding, and inputs it into a pre-trained large language model to extract semantic features with global constraints, including: S331. Extract the global metadata of the current hyperspectral image. The global metadata includes sensor type and band range, illumination and weather conditions, background water type and candidate oil categories included in the current task. S332. Using a preset natural language prompt template, the discrete variables in the global metadata are mapped into coherent scene description text; S333. Input the scene description text into the pre-trained large language model encoder, extract the global representation vector of the last layer of the large language model, and obtain the original semantic features. S334. The original semantic features are input into a multilayer perceptron and projected onto a feature space that matches the spatial spectral features and frequency domain features through nonlinear mapping to obtain global semantic features.

7. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 1, characterized in that, The training process of the oil spill identification model includes: S11. Obtain the training dataset, which contains hyperspectral cubic oil data from multiple collection scenarios and corresponding pixel-level category labels. It is divided into training set, validation set and test set according to the collection scenario to ensure that samples from the same scenario do not appear in the same set. S12. Input the samples in the training set into the initial oil spill identification model in batches, and output the predicted probability distribution of each pixel belonging to each category; S13. Based on the predicted probability distribution and the corresponding true labels, calculate the classification loss value using the cross-entropy loss function that combines category weights. The category weights are calculated based on the reciprocal frequency of each type of sample in the training set. S14. Using the classification loss value as the optimization objective, a cosine annealing learning rate scheduling strategy with a preheating mechanism is adopted to synchronously update the learnable parameters of each branch and fusion module in the oil spill identification model through backpropagation until the model converges. S15. During the training process, the model is stopped early and hyperparameters are adjusted based on the classification accuracy on the validation set. After the model converges, the oil spill identification model that has completed training is evaluated using the test set. The oil spill identification model that passes the evaluation is determined as the well-trained oil spill identification model. The evaluation uses the overall classification accuracy, average classification accuracy and Kappa coefficient as quantitative evaluation indicators.

8. The hyperspectral oil spill identification method based on spatial-spectral-frequency domain joint characterization according to claim 1, characterized in that, The S5 inputs optimized discriminative features to the classification output terminal, and outputs oil spill identification results based on a mask and confidence decision mechanism, including: S51. Input the optimized discriminant features into the classification head, and map them to the category space through layer normalization and fully connected linear layers in sequence to obtain the log probability vectors of each category; S52. The logarithmic probability vector is normalized using the Softmax function to obtain the predicted probability distribution of each pixel belonging to each category; S53. Pre-import land and cloud non-water space masks, and output invalid pixels within the space mask as background categories; S54. For effective water body pixels, set a confidence threshold and determine whether the maximum predicted probability is greater than or equal to the confidence threshold. If so, take the category corresponding to the maximum predicted probability as the oil spill identification result of the effective water body pixel; if not, classify the effective water body pixel as an unknown mixed category or a background category.

9. A hyperspectral oil spill identification device with spatial-spectral-frequency domain joint characterization, wherein the hyperspectral oil spill identification device with spatial-spectral-frequency domain joint characterization is used to implement the hyperspectral oil spill identification method with spatial-spectral-frequency domain joint characterization as described in any one of claims 1-8, characterized in that, The device includes: The acquisition unit is used to acquire the image data to be identified and the trained oil spill identification model. The image data to be identified is hyperspectral cubic oil data. The oil spill identification model includes a ViT spatial spectral feature extraction branch, a frequency domain feature learning branch, a global semantic feature extraction branch, a gated channel attention module, and a classification output end. The preprocessing unit is used to perform scene-based standard normalization of hyperspectral cubic oil data, and to select bands from the normalized hyperspectral data to obtain key band data. The feature extraction unit is used to synchronously send key band data to the ViT spatial spectral feature extraction branch, the frequency domain feature learning branch, and the global semantic feature extraction branch. Based on the ViT spatial spectral feature extraction branch, spatial spectral features are obtained; based on the frequency domain feature learning branch, frequency domain features are obtained; and based on the global semantic feature extraction branch, semantic features are obtained. The fusion unit is used to combine frequency domain features with semantic features, concatenate the generated auxiliary representation with spatial spectral features, and input the resulting concatenated features into the gated channel attention module to obtain optimized discriminative features; The identification unit is used to input optimized discrimination features into the classification output end and output oil spill identification results based on the mask and confidence decision mechanism.

10. A hyperspectral oil spill identification device with joint spatial-spectral-frequency domain characterization, characterized in that, The hyperspectral oil spill identification device, characterized by joint spatial-spectral-frequency domain representation, includes: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 8.

Images