An end-to-end inversion method for lake depth based on hyperspectral imagery and multimodal cues

Abstract

This invention provides an end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues, belonging to the field of hyperspectral remote sensing mapping technology. It retrieves hyperspectral imagery and ICESat altimeters using lake boundary vectors, preprocesses them to obtain corrected images and elevation-normalized depth values, encodes depth values ​​as triplet graticules, quantizes spatial relationships to output a signed distance field, and encodes scene vectors describing the environment. When loading enhanced data into the spectral branch of a dual-branch model to output a feature map, the scene vector, triplet graticules, and distance field are simultaneously loaded into the cues branch to inject physical rules, outputting multimodal constraint features. Based on cross-attention fusion of the two features, a depth map and an uncertainty map are output. This method solves the technical problems of existing hyperspectral imagery depth inversion techniques, such as step-by-step processes leading to progressive error accumulation, lack of heterogeneous information fusion mechanisms, and the need for full-process recalculation for cross-domain migration. It achieves the technical effect of improving inversion accuracy and timeliness.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral remote sensing mapping technology, and in particular to an end-to-end inversion method for lake water depth based on hyperspectral imagery and multimodal cues. Background Technology

[0002] The core flaw of existing technology lies in the fact that when relying solely on hyperspectral images for spectral regression, it is subject to interference from multiple factors such as seasonal changes, differences in imaging time, fluctuations in atmospheric conditions, and changes in lake water quality, resulting in systematic biases and insufficient stability in the water depth inversion results.

[0003] Specifically, the traditional step-by-step process leads to the accumulation of errors at each level, and the separation between the correction stage and the initial inversion causes a response lag, making it difficult to adapt to complex water color changes in real time. At the same time, there is a lack of a collaborative fusion mechanism for heterogeneous information such as sparse elevation points, shoreline zero-depth priors, and imaging environment descriptions, making it impossible to correct local deviations in real time through physical constraints, such as inaccurate nearshore water depth and amplified misjudgments of seasonal color shifts. Furthermore, when migrating across time phases and water areas, the entire process needs to be recalculated repeatedly, resulting in low engineering update efficiency and difficulty in meeting the dual requirements of accuracy and timeliness in actual business operations.

[0004] In summary, existing technologies for water depth inversion from hyperspectral images suffer from technical problems such as step-by-step process leading to the gradual accumulation of errors, lack of a mechanism for the collaborative fusion of heterogeneous information, and the need for full recalculation for cross-domain migration. Summary of the Invention

[0005] To address the above-mentioned deficiencies or improvement needs of existing technologies, this invention provides an end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues. This method solves the technical problems of existing technologies for depth inversion from hyperspectral images, such as step-by-step processes leading to the gradual accumulation of errors, lack of heterogeneous information fusion mechanisms, and the need for full-process recalculation for cross-domain migration.

[0006] To achieve the above objectives, this invention provides an end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues, the method comprising:

[0007] Using the lake boundary vector as the spatial constraint, after retrieving the original hyperspectral image and ICESat altimeter set from the remote sensing data platform, standardized preprocessing is performed to output corrected hyperspectral image and elevation-normalized water depth values. The elevation-normalized water depth values ​​are encoded into a triplet raster structure. Shallow water enhancement bands are retrieved from the expert experience database, and enhanced hyperspectral input data is obtained by overlaying the shallow water enhancement bands onto the corrected hyperspectral image. Spatial relationship quantization processing is performed on the corrected hyperspectral image to output signed distance field encoding. Environmental imaging description information retrieved based on a preset window period is then processed. The scene description vector is obtained by encoding the environmental prompt information. An end-to-end dual-branch model is pre-constructed, which includes a spectral extraction branch and a prompt branch. During the process of loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output a spectral feature map, the scene description vector, triplet grid structure, and signed distance field encoding are simultaneously loaded into the prompt branch to perform physical rule injection and output multimodal constraint features. The spectral feature map and multimodal constraint features are fused based on a cross-attention mechanism to output a pixel-level depth map and uncertainty map.

[0008] In one implementation, the enhanced hyperspectral input data is loaded into the spectral extraction branch to perform feature extraction, outputting a spectral feature map, and the following processing is also performed:

[0009] Convolution operations are performed on the enhanced hyperspectral input data to extract spatial features layer by layer, resulting in a basic spatial feature map. Multi-scale downsampling is then performed on the basic spatial feature map using stride convolution to construct a spatial pyramid feature. The spatial pyramid feature is then expanded into a feature sequence along the spectral dimension, and a multi-head self-attention operation is performed to model inter-band dependencies, outputting self-attention reconstructed features. Dual-path decoupling is then performed on the self-attention reconstructed features to output spatial texture features and spectral response features. Finally, the spatial texture features and spectral response features are fused to generate the spectral feature map.

[0010] In one implementation, the scene description vector, triplet grid structure, and signed distance field code are loaded into the cue branch to perform physical rule injection, outputting multimodal constraint features, and the following processing is also performed:

[0011] A pre-constructed prompt branch is included, comprising a cascaded feature encoding layer and a constraint generation layer. The scene description vector is input into the feature encoding layer to generate stable spectrum modulation parameters, wherein the stable spectrum modulation parameters are used to correct the basic spatial feature map across branches. The constraint generation layer parses the triplet grid structure to generate altimeter position constraint features. After converting the signed distance field encoding into a shoreline constraint weight map in the constraint generation layer, spatial weighted mapping is performed to generate shoreline constraint features. The stable spectrum modulation parameters, altimeter position constraint features, and shoreline constraint features are fused to generate the multimodal constraint features.

[0012] In one implementation, using the lake boundary vector as the spatial constraint, after retrieving the original hyperspectral image and ICESat altimeter set from the remote sensing data platform, standardized preprocessing is performed to output corrected hyperspectral image and elevation-normalized water depth values. The following processing is also performed:

[0013] The original hyperspectral image is radiometrically calibrated to obtain an apparent reflectance image. After atmospheric correction, the apparent reflectance image is geometrically registered with the ICESat altimeter set using a DEM, resulting in a spatially unified corrected hyperspectral image and an ICESat bathymetry set. The ICESat bathymetry set is then transformed to an elevation datum, and the elevation-normalized water depth value is output.

[0014] In one implementation, the following processing is also performed:

[0015] The triplet grid structure is updated based on the newly added measurement points to obtain the updated grid structure; the scene description vector is updated based on the newly added environmental cues to obtain the updated description vector; the updated grid structure, the updated description vector, and the signed distance field encoding are loaded into the cues branch to perform physical rule injection and output the updated constraint features; the spectral feature map and the updated constraint features are fused based on the cross-attention mechanism to dynamically update the pixel-level depth map and uncertainty map.

[0016] In one implementation, the modified hyperspectral image undergoes spatial relation quantization processing to output a signed distance field code, and the following processing is also performed:

[0017] Calculate the distributed water index based on the corrected hyperspectral image; perform water body segmentation based on the distributed water index to obtain a water body mask; generate the signed distance field code based on the water body mask.

[0018] In one implementation, the following processing is also performed:

[0019] The shoreline neighborhood zone is extracted from the water mask, wherein the shoreline neighborhood zone is identified by a signed distance field; the pixel-level depth map is post-processed and optimized based on the water mask, the shoreline neighborhood zone, and the uncertainty map to generate an underwater topographic mesh model; and isobath topology correction products and 3D visualization results are generated based on the underwater topographic mesh model.

[0020] In one embodiment, the triplet grid structure includes a measured value layer, a position mask layer, and a mass weight layer, and the scene description vector includes imaging time, solar altitude angle, solar azimuth angle, and lake surface elevation.

[0021] In one implementation, after retrieving free text annotations based on the lake boundary vector, the free text annotations are used to supplement the environmental imaging description.

[0022] One or more technical solutions provided in this invention have at least the following technical effects or advantages:

[0023] The method provided in this invention retrieves original hyperspectral images and ICESat altimeters using lake boundary vectors as constraints. After standardized preprocessing, it outputs corrected hyperspectral images and elevation-normalized water depth values. The water depth values ​​are encoded into a triplet raster structure, and shallow water enhancement bands are superimposed to generate enhanced hyperspectral input data. Signed distance field encoding is output for quantizing spatial relationships in the corrected images. Environmental imaging description information is encoded to generate scene description vectors. A dual-branch model with spectral extraction and prompting branches is constructed. When loading enhanced data into the spectral branch to output spectral feature maps, scene vectors, triplet raster structures, and distance field encodings are simultaneously loaded into the prompting branch to perform physical rule injection, outputting multimodal constraint features. Based on a cross-attention mechanism, the spectral feature maps and multimodal constraint features are fused to output pixel-level water depth maps and uncertainty maps. This achieves the technical effect of outputting pixel-level water depth maps in a single inference, avoiding step-by-step error accumulation and correction lag, and improving the accuracy and timeliness of cross-domain inversion. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 The diagram shows a flowchart of the end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues provided by the present invention.

[0026] Figure 2The diagram illustrates the process of obtaining spectral feature maps in the end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues provided by the present invention. Detailed Implementation

[0027] This invention provides an end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues, which addresses the technical problems of existing hyperspectral imagery depth inversion methods, such as step-by-step process leading to gradual error accumulation, lack of heterogeneous information fusion mechanism, and the need for full recalculation for cross-domain migration.

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0029] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0030] Unless otherwise expressly stated, throughout the specification and claims, the term "comprising" or its variations such as "including" or "comprises" shall be understood to include the stated elements or components without excluding other elements or other components.

[0031] The flowchart of the end-to-end lake depth inversion method based on hyperspectral imagery and multimodal cues provided in this embodiment of the invention is shown below. Figure 1 The method includes:

[0032] A100: Using the lake boundary vector as the spatial constraint, after retrieving the original hyperspectral image and ICESat altimeter set from the remote sensing data platform, standardization preprocessing is performed to output the corrected hyperspectral image and elevation-normalized water depth value.

[0033] In one implementation, using the lake boundary vector as the spatial constraint, after retrieving the original hyperspectral image and ICESat altimeter set from the remote sensing data platform, standardized preprocessing is performed to output the corrected hyperspectral image and elevation-normalized water depth values. The method step A100 provided by this invention includes:

[0034] A110: Perform radiometric calibration correction on the original hyperspectral image to obtain an apparent reflectance image.

[0035] A120: After performing atmospheric correction on the apparent reflectance image, the apparent reflectance image and the ICESat altimeter set are geometrically registered using a DEM, and a corrected hyperspectral image and ICESat bathymetric set with unified spatial reference are output.

[0036] A130: Perform elevation datum transformation on the ICESat sounding point set and output the elevation-normalized water depth value.

[0037] This embodiment uses the lake boundary vector as a spatial range definition framework, acquires raw hyperspectral images and ICESat altimeter data from a remote sensing platform, and solves the multi-source error problem through a standardized preprocessing chain.

[0038] Specifically, this embodiment converts the raw digital quantization values ​​of the original hyperspectral image into apparent reflectance data with a clear physical definition through sensor calibration parameters, eliminating systematic deviations introduced by changes in illumination conditions, differences in sensor sensitivity, and the radiative transfer process, and constructing a spectral reflectance image with true and reliable radiative characteristics, providing a unified input basis for subsequent atmospheric correction.

[0039] Based on the apparent reflectance image output by radiometric calibration, an atmospheric radiative transfer model or empirical algorithm is used to remove the absorption and scattering effects of the atmosphere on electromagnetic waves, accurately restoring the true reflectance characteristics of the Earth's surface.

[0040] Simultaneously, based on digital elevation model data, the atmospherically corrected reflectivity image and the ICESat altimeter set are jointly geometrically registered. Orthorectification is used to compensate for image point displacement caused by terrain undulations, and the images are unified to the same map projection and spatial coordinate system. This ensures that the spatial position of each pixel in the image is strictly matched with the sounding point, and outputs a corrected hyperspectral image with reliable geometric accuracy and a completely consistent spatial reference, along with a precisely corresponding ICESat sounding point set.

[0041] Based on the instantaneous lake surface elevation data at the measurement point location, the original elevation observation values ​​are converted into normalized water depth values ​​with the horizontal plane as a reference. This eliminates the water depth calculation bias caused by differences in land elevation benchmarks, enabling sparsely distributed depth measurement data to have spatial comparability across water areas and providing standardized true water depth input for constructing physical constraints.

[0042] A200: Encode the elevation-normalized water depth value into a triplet grid structure.

[0043] Specifically, in this embodiment, the elevation-normalized water depth value is encoded as a structured triplet grid, and the triplet grid structure specifically includes a measured value layer, a position mask layer, and a mass weight layer.

[0044] The measured value layer accurately records the water depth observation values ​​corresponding to each grid cell, the location mask layer marks the spatial distribution of effective depth sounding points with binary identifiers, and the quality weight layer assigns different confidence weights based on the reliability of the data source and environmental factors, forming a machine-resolvable physical constraint carrier.

[0045] This triplet structure strictly distinguishes between the missing state of the unobserved region and the physical fact that the actual water depth is zero, avoiding the information ambiguity caused by the confusion of zero values ​​in traditional sparse matrices. It provides a spatially aligned supervision signal basis for the subsequent injection of physical rules, ensuring that the height measurement point prompts can specifically correct local spectral deviations.

[0046] A300: Retrieves the shallow water enhancement band from the expert experience database and obtains enhanced hyperspectral input data by overlaying the shallow water enhancement band onto the corrected hyperspectral image.

[0047] It should be understood that the original hyperspectral image in nearshore waters is easily affected by bottom sediment reflection, suspended matter interference, and mixed pixels in the water-land transition zone, resulting in weak or distorted shallow water signals.

[0048] Based on this, this embodiment calls upon a long-term validated shallow water sensitive band combination from an expert experience database. The shallow water sensitive band combination covers distributed water indices such as NDWI and MNDWI. It is then fused with the corrected hyperspectral image through a band overlay method to construct enhanced hyperspectral input data.

[0049] The resulting enhanced hyperspectral input data retains fine-grained spectral information and incorporates an expert-guided shallow water feature enhancement layer, enabling subsequent convolutional networks to more accurately capture traces of shallow water topographic changes and providing a highly discriminative input base for the spectral extraction branch.

[0050] A400: Perform spatial relation quantization processing on the corrected hyperspectral image and output a signed distance field code.

[0051] In one implementation, the modified hyperspectral image is subjected to spatial relation quantization processing to output a signed distance field code. Step A400 of the method provided by this invention includes:

[0052] A410: Calculate the distributed water index based on the modified hyperspectral image.

[0053] A420: Perform water body segmentation based on the distributed water body index to obtain a water body mask.

[0054] A430: Generate the signed distance field code based on the water mask.

[0055] Specifically, in this embodiment, the green and near-infrared reflectance data of the corrected hyperspectral image are read, and a preset distributed band combination operation is performed, such as reflectance weighted average or texture feature fusion within a local window, to generate a water index value for each pixel. Then, the data is integrated according to the pixel distribution, and the distributed water index is output.

[0056] This embodiment enhances the ability to identify fragmented water bodies, turbid areas, and water-land transition zones by introducing spatial neighborhood statistical characteristics, forming a continuous index matrix that integrates spectral and spatial distribution characteristics, namely the distributed water body index.

[0057] A spatial adaptive thresholding algorithm, such as the OTSU method combined with local window histogram analysis, is adopted to dynamically divide the water body segmentation thresholds of different regions based on the distributed water body index. Specifically, the distributed water body index is traversed, and each pixel is binarized according to its neighborhood distribution characteristics. Pixels with values ​​above the local threshold are marked as water bodies, and pixels with values ​​below the threshold are marked as non-water bodies. This generates a geometrically complete water body mask grid to ensure that the shoreline boundary is consistent with the real landform.

[0058] Using the water-land boundary in the mask as a reference, the Euclidean distance from each pixel in the grid to the nearest shoreline point is calculated. Specifically, all pixels in the mask are traversed. If a pixel belongs to the water area, the positive distance to the nearest land boundary is calculated; if a pixel belongs to the land area, the negative distance to the nearest water boundary is calculated. Finally, a signed floating-point distance matrix is ​​output, which is the signed distance field code. The signed distance field code accurately quantifies the spatial relationship between any position and the shoreline, forming a geometric carrier of physical constraints.

[0059] This embodiment performs spatial relationship quantization processing on the corrected hyperspectral image, identifies the water body coverage by calculating the distributed water body index, extracts an accurate land-water boundary mask based on the segmentation threshold, and generates a signed distance field code based on this mask, providing a basis for calculating the attenuation weight for subsequent shoreline constraints. This enables the end-to-end bi-branch model in the following text to dynamically adjust the nearshore water depth soft constraint strength according to the distance, thereby anchoring the physical boundary of water depth inversion at the geometric level.

[0060] A500: The environmental imaging description information retrieved based on a preset window period is encoded with environmental prompt information to obtain a scene description vector, which includes imaging time, solar altitude angle, solar azimuth angle and lake surface elevation.

[0061] In one implementation, after retrieving free text annotations based on the lake boundary vector, the free text annotations are used as supplementary input for the environmental imaging description.

[0062] A600: Pre-built end-to-end bi-branch model, wherein the end-to-end bi-branch model includes a spectral extraction branch and a cueing branch.

[0063] In this embodiment, the core function of the end-to-end dual-branch model is to realize the coordination and adaptive correction of physical rules for hyperspectral water depth inversion.

[0064] The spectral extraction branch of the model loads enhanced hyperspectral input data, captures spatial texture features layer by layer through convolution operations, constructs a multi-scale spatial pyramid structure and then unfolds the sequence along the spectral dimension, uses a multi-head self-attention mechanism to model the complex inter-band dependencies, decouples the output spatial texture features and spectral response features and fuses them into a spectral feature map, thereby systematically extracting the spectral-spatial response pattern of the image.

[0065] The model's prompt branch is used to receive scene description vectors, triplet grid structures, and signed distance field codes. The environmental description is converted into stable spectrum modulation parameters through the feature coding layer. In the constraint generation layer, the altimeter triplet is parsed to generate position supervision features and the SDF code is converted into shoreline constraint features. Finally, the three types of physical constraints are fused to construct multimodal constraint features, providing real-time correction signals for the spectral branch.

[0066] The dual-branch design in this embodiment avoids interference from the mixing of heterogeneous information by dividing tasks and decoupling.

[0067] A700: During the process of loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output the spectral feature map, the scene description vector, triplet grid structure and signed distance field encoding are simultaneously loaded into the prompting branch to perform physical rule injection and output multimodal constraint features.

[0068] In one implementation, see Figure 2 The enhanced hyperspectral input data is loaded into the spectral extraction branch to perform feature extraction and output a spectral feature map. The method step A700 provided by this invention includes:

[0069] A700-11: Perform convolution operations on the enhanced hyperspectral input data to extract spatial features layer by layer and obtain a basic spatial feature map.

[0070] A700-12: Multi-scale downsampling is performed on the basic spatial feature map using stride convolution to construct spatial pyramid features.

[0071] A700-13: Expand the spatial pyramid features along the spectral dimension into a feature sequence, perform multi-head self-attention operation to model the inter-band dependencies, and output the self-attention reconstructed features.

[0072] A700-14: Perform dual-path decoupling on the self-attention reconstruction features to output spatial texture features and spectral response features.

[0073] A700-15: The spatial texture features and spectral response features are fused to generate the spectral feature map.

[0074] In one implementation, the scene description vector, triplet grid structure, and signed distance field encoding are loaded into the cue branch to perform physical rule injection and output multimodal constraint features. The method step A700 provided by this invention includes:

[0075] A700-21: Pre-constructed hint branch, wherein the hint branch includes a cascaded feature encoding layer and a constraint generation layer.

[0076] A700-22: Input the scene description vector into the feature coding layer to generate stable spectrum modulation parameters, wherein the stable spectrum modulation parameters are used to correct the basic spatial feature map across branches.

[0077] A700-23: The triplet grid structure is parsed in the constraint generation layer to generate the height measurement point position constraint feature.

[0078] A700-24: After the signed distance field encoding is converted into a shoreline constraint weight map in the constraint generation layer, spatial weighted mapping is performed to generate shoreline constraint features.

[0079] A700-25: The multimodal constraint features are generated by fusing the stable spectrum modulation parameters, altimeter position constraint features, and shoreline constraint features.

[0080] Specifically, the dual-branch decoupling design of the model in this embodiment enables the simultaneous loading of the scene description vector, triplet grid structure, and signed distance field encoding into the prompting branch to perform physical rule injection and output multimodal constraint features while loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output spectral feature maps.

[0081] Specifically, the data processing procedure for loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output the spectral feature map is as follows:

[0082] The system loads multi-band image data from the enhanced hyperspectral input data, performs sliding window calculations using a preset convolution kernel, extracts local spatial texture features such as edges and texture patterns layer by layer, and outputs the basic spatial feature map with increased channel count. The spatial resolution of the basic spatial feature map is consistent with the input, but the depth dimension is significantly expanded.

[0083] By applying convolutional kernels with a stride greater than 1 to the basic spatial feature map, and gradually reducing the spatial resolution of the feature map according to preset ratios such as 1 / 2 and 1 / 4, contextual information under different receptive fields is extracted simultaneously, and a spatial pyramid feature containing the original scale and downsampling scale is constructed to form a multi-granular spatial representation.

[0084] Multi-scale feature maps are reorganized into a time series structure according to band order, with each time step corresponding to a single band feature; multi-head self-attention operation is performed on the sequence data to calculate the correlation weights between bands, and the band dependencies are reconstructed through weighted aggregation to output self-attention reconstructed features enhanced with spectral dimension information.

[0085] The self-attention reconstructed features are input into the parallel convolution path. One path uses a directional gradient filter to extract spatial texture features such as shoreline orientation and underwater topographic contours, while the other path uses a spectral response filter to extract band-sensitive features such as the location of water absorption peaks. Finally, the spatial texture features and spectral response features are separated and output. It should be understood that the features obtained here are essentially feature matrices.

[0086] The spatial texture features and spectral response features are subjected to channel splicing operations. Cross-channel information interaction and dimensionality compression are performed through 1×1 convolution kernels to generate a single spectral feature map that integrates spatial and spectral information, which serves as the query signal carrier for subsequent cross-attention.

[0087] Specifically, the data processing procedure for loading the scene description vector, triplet grid structure, and signed distance field encoding into the prompt branch to perform physical rule injection and output multimodal constraint features is as follows:

[0088] The pre-built hint branch cascade structure includes a cascaded initialization feature encoding layer and a constraint generation layer. The initialization feature encoding layer is a fully connected network dedicated to semantic vector transformation, and the constraint generation layer is a lightweight convolutional module dedicated to physical rule parsing. The two layers form a two-stage processing pipeline.

[0089] A vector containing parameters such as imaging time and solar angle is loaded, and a stable spectrum modulation parameter composed of scaling factor γ and offset factor β is generated through nonlinear mapping of a fully connected layer. This parameter is passed to the spectral branch, and a channel-by-channel affine transformation is performed on the basic spatial feature map to achieve cross-branch spectral correction.

[0090] The measured value layer, the location mask layer, and the quality weight layer are read. The effective depth measurement locations are filtered through the location mask. The measured values ​​and weights are bound to generate a spatially aligned supervised feature matrix. The non-measurement point areas are set to zero to form the height measurement point location constraint feature.

[0091] A predefined exponential decay function, such as w=exp(-λ·|SDF|).

[0092] After loading the signed distance field encoding into the constraint generation layer, the shoreline constraint weight map is calculated using an exponential decay function. Then, based on the weight map representing the nearshore water depth benchmark, a spatial weighted mapping is performed on the zero-value template to generate shoreline constraint features whose intensity decays with distance from the shore.

[0093] The stable spectrum modulation parameters, altimeter position constraint features, and shoreline constraint features are tensor-concatenated to generate the multimodal constraint features, which serve as the key-value input source for cross-attention.

[0094] A800: Based on the cross-attention mechanism, the spectral feature map and multimodal constraint features are fused to output pixel-level depth maps and uncertainty maps.

[0095] In this embodiment, spectral feature maps are used as dynamic query signals and multimodal constraint features are used as key-value pair inputs. Spatially aligned cross-attention operations are performed. Specifically, firstly, a position index mapping is established based on pixel spatial coordinates, so that the feature vector of each position in the query matrix is ​​automatically associated with the constraint features of the corresponding position in the key-value matrix. The similarity weight between the query and the key is calculated by matrix multiplication, and after softmax normalization, the value vectors are weighted and aggregated to generate enhanced features that fuse physical constraints.

[0096] The feature is input into the parallel decoding path. One path gradually upsamples through a deconvolutional layer to restore spatial resolution, and combines skip connections to supplement details, outputting a pixel-level water depth raster that covers the entire image, thus obtaining the pixel-level water depth map. The other path calculates the uncertainty quantization value based on the variance of the attention weights, and generates an uncertainty distribution map with the same resolution as the water depth map through 1×1 convolution, realizing pixel-level soft fusion of features and constraints, and avoiding the spatial discontinuity problem caused by hard rule injection.

[0097] This embodiment achieves pixel-level water depth map output in a single inference, avoiding step-by-step error accumulation and correction lag, and improving the technical effect of cross-domain inversion accuracy and timeliness.

[0098] In one implementation, the method provided by the present invention further includes:

[0099] A910: Update the triplet grid structure based on the newly added measurement points to obtain the updated grid structure.

[0100] A920: Update the scene description vector based on the newly added environment prompt to obtain the updated description vector.

[0101] A930: Load the updated raster structure, updated description vector, and signed distance field encoding into the prompt branch to perform physical rule injection and output updated constraint features.

[0102] A940: Based on the cross-attention mechanism, the spectral feature map and the update constraint feature are fused to dynamically update the pixel-level depth map and uncertainty map.

[0103] Specifically, this embodiment reads the spatial coordinates and water depth observation values ​​of newly added ICESat altimetry points, locates the corresponding pixel positions in the original triplet grid, writes the newly added measured values ​​into the corresponding units of the measured value layer, updates the corresponding identifier to the valid state in the position mask layer, calculates the newly added quality weight value according to the measurement point acquisition conditions, and updates the weight layer; simultaneously maintains the consistency of the grid spatial structure and data type, generates an updated grid structure integrating the newly added information, and ensures that the coverage of the altimetry monitoring constraint is dynamically expanded.

[0104] New environmental parameters such as imaging time shift, solar angle change, or lake surface elevation fluctuation are loaded, and the semantics of the parameters are parsed and mapped to the corresponding dimensions of the vector. If it is a text annotation, features are extracted by a pre-trained semantic encoder and embedded into the supplementary dimension. Based on preset rules, the new and old parameters are weighted and fused to reconstruct the numerical distribution of the scene description vector and generate an updated description vector that reflects the dynamic changes of the environment.

[0105] The cached spectral feature map of the spectral branch is reused as a static query signal, and the updated constraint features are used as dynamic key-value inputs. The newly added measurement point area is automatically associated through the spatial location index, which strengthens the query-key-value interaction weight of the location. After performing attention-weighted fusion, the decoder is started locally only for the newly added measurement point influence area defined by the location mask, and the updated pixel-level water depth map is output. The confidence of the corresponding area in the uncertainty map is adjusted simultaneously to achieve incremental refresh of the water depth results of the whole map.

[0106] This embodiment achieves the technical effect of prompt-driven lightweight water depth update, reusing image feature local refresh results, and responding to business needs in minutes.

[0107] In one implementation, the method provided by the present invention further includes:

[0108] A1010: Extract the shoreline neighborhood zone from the water body mask, wherein the shoreline neighborhood zone is identified using a signed distance field.

[0109] A1020: Post-processing optimization of the pixel-level water depth map is performed based on the water body mask, shoreline neighborhood zone, and uncertainty map to generate an underwater terrain mesh model.

[0110] A1030: Generate contour line topology correction products and 3D visualization results based on the underwater terrain mesh model.

[0111] It should be understood that the water mask grid is binary data, where 1 represents water and 0 represents land. In this embodiment, the water mask grid is loaded to identify the water-land boundary line, i.e., the 0-1 abrupt change position, as the initial shoreline.

[0112] Extend a preset buffer distance outward from the shoreline to generate a shoreline neighborhood range raster; mark pixels in the signed distance field whose distance values ​​fall within the positive and negative range of the neighborhood bandwidth threshold as shoreline neighborhoods, and finally output a neighborhood zone identifier raster with SDF distance values.

[0113] The water depth values ​​of non-water pixels are forced to zero based on the water mask to eliminate land misclassification. A zero-depth constraint is applied within the shoreline neighborhood, and the original water depth values ​​are weighted and corrected according to the distance decay function to ensure that the near-shore water depth approaches zero. The uncertainty map output by the model is loaded synchronously, and adaptive Gaussian filtering is performed on high uncertainty areas, combined with neighborhood constraints to smooth local abrupt changes. Finally, outlier water depth points in the connected areas of the water mask are detected, reset to the neighborhood median based on the SDF distance, and an underwater terrain mesh model with consistent topology is output.

[0114] Contour lines are extracted from the underwater topographic mesh model at preset depth intervals. The topological relationship of the contour lines is corrected based on the SDF value of the shoreline neighborhood. Burr segments with a length less than the threshold are deleted and hole polygons are filled in. The contour line vector product conforming to the mapping specifications is output. At the same time, the elevation values ​​of the mesh model are mapped to three-dimensional space. A triangular mesh surface is constructed based on the water mask range. The lighting rendering parameters are set according to the solar azimuth angle corresponding to the imaging time to generate a three-dimensional visualized surface model with lake bottom texture.

[0115] This embodiment achieves the technical effects of forced zeroing of nearshore water depth and outlier cleanup, and outputting topologically consistent isobath products and 3D visualization results.

[0116] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A lake water depth end-to-end inversion method based on hyperspectral image and multi-modal cues, characterized in that, The method comprises the following steps: Taking the lake boundary vector as a spatial range constraint, after calling the original hyperspectral image and the ICESat elevation point set from a remote sensing data platform, standardized preprocessing is performed, and a corrected hyperspectral image and an elevation normalized water depth value are output; Encode the elevation normalized water depth value into a triple grid structure; Call the shallow water enhancement band from the expert experience library, and obtain an enhanced hyperspectral input data by superimposing the shallow water enhancement band on the corrected hyperspectral image; Perform spatial relationship quantization processing on the corrected hyperspectral image, and output signed distance field encoding; Perform environment prompt information coding on the environment imaging description information called based on a preset window period to obtain a scene description vector; Pre-construct an end-to-end double-branch model, wherein the end-to-end double-branch model comprises a spectral extraction branch and a prompt branch; In the process of loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output a spectral feature map, the scene description vector, the triple grid structure and the signed distance field encoding are synchronously loaded into the prompt branch to perform physical rule injection and output multi-modal constraint features; Based on the cross attention mechanism, the spectral feature map and the multi-modal constraint features are fused to output a pixel-level water depth map and an uncertainty map.

2. The hyperspectral image and multi-modal cues based lake water depth end-to-end inversion method of claim 1, wherein, Loading the enhanced hyperspectral input data into the spectral extraction branch to perform feature extraction and output a spectral feature map comprises: Performing convolution operation on the enhanced hyperspectral input data to extract spatial features layer by layer to obtain a basic spatial feature map; Performing multi-scale down-sampling on the basic spatial feature map through stride convolution to construct a spatial pyramid feature; Expanding the spatial pyramid feature into a feature sequence along the spectral dimension, performing multi-head self-attention operation to model the inter-band dependency relationship, and outputting a self-attention reconstruction feature; Performing double-channel decoupling on the self-attention reconstruction feature to output spatial texture features and spectral response features; Fusing the spatial texture features and the spectral response features to generate the spectral feature map.

3. The hyperspectral image and multi-modal cues based lake water depth end-to-end inversion method of claim 2, wherein, Loading the scene description vector, the triple grid structure and the signed distance field encoding into the prompt branch to perform physical rule injection and output multi-modal constraint features comprises: Pre-constructing a prompt branch, wherein the prompt branch comprises a cascaded feature encoding layer and a constraint generation layer; Inputting the scene description vector into the feature encoding layer to generate a stable spectral modulation parameter, wherein the stable spectral modulation parameter is used to correct the basic spatial feature map across branches; Analyzing the triple grid structure in the constraint generation layer to generate an elevation point position constraint feature; After converting the signed distance field encoding into a shoreline constraint weight map in the constraint generation layer, performing spatial weighted mapping to generate a shoreline constraint feature; Fusing the stable spectral modulation parameter, the elevation point position constraint feature and the shoreline constraint feature to generate the multi-modal constraint feature.

4. The hyperspectral imagery and multi-modal cues based lake water depth end-to-end inversion method of claim 1, wherein, Taking the lake boundary vector as a spatial range constraint, after calling the original hyperspectral image and the ICESat elevation point set from a remote sensing data platform, standardized preprocessing is performed, and a corrected hyperspectral image and an elevation normalized water depth value are output, comprising: The original hyperspectral image is subjected to radiation calibration correction processing to obtain an apparent reflectance image; After performing atmospheric correction processing on the apparent reflectance image, the apparent reflectance image and the ICESat height measurement point set are geometrically assisted registered using DEM, and a corrected hyperspectral image and an ICESat depth measurement point set with unified spatial reference are output; The ICESat depth measurement point set is subjected to height reference conversion, and the height normalized water depth value is output.

5. The hyperspectral imagery and multi-modal cues based lake water depth end-to-end inversion method of claim 1, wherein, Further comprising: The three-tuple grid structure is updated according to the newly added measurement points to obtain an updated grid structure; The scene description vector is updated according to the newly added environmental cues to obtain an updated description vector; The updated grid structure, updated description vector and signed distance field encoding are loaded into the prompt branch to perform physical rule injection, and an updated constraint feature is output; Based on the cross-attention mechanism, the spectral feature map and the updated constraint feature are fused to dynamically update the pixel-level water depth map and the uncertainty map.

6. The hyperspectral imagery and multi-modal cues based lake water depth end-to-end inversion method of claim 1, wherein, The corrected hyperspectral image is subjected to spatial relationship quantization processing to output signed distance field encoding, including: The distributed water body index is calculated according to the corrected hyperspectral image; Water body segmentation is performed according to the distributed water body index to obtain a water body mask; The signed distance field encoding is generated based on the water body mask.

7. The hyperspectral image and multi-modal cues based lake water depth end-to-end inversion method of claim 6, wherein, Comprising: The shoreline neighborhood band is extracted from the water body mask, wherein the shoreline neighborhood band is identified using signed distance field; The post-processing optimization of the pixel-level water depth map is performed according to the water body mask, shoreline neighborhood band and uncertainty map to generate an underwater topographic grid model; Based on the underwater topographic grid model, an isobath topological correction product and a three-dimensional visualization result are generated.

8. The hyperspectral imagery and multi-modal cues based lake water depth end-to-end inversion method of claim 2, wherein, The three-tuple grid structure includes a measured value layer, a position mask layer and a quality weight layer, and the scene description vector includes imaging time, solar elevation angle, solar azimuth angle and lake surface elevation.

9. The hyperspectral imagery and multi-modal cues based lake water depth end-to-end inversion method of claim 1, wherein, After retrieving the free text remarks according to the lake boundary vector, the free text remarks are used for supplementary input of the environmental imaging description.

Images