Deep-sea geological mapping method and system based on multi-modal large model

Abstract

The application discloses a deep-sea geological mapping method and system based on a multi-modal large model, which comprises the following steps: obtaining multi-source data such as backscattering and water depth, constructing high-quality visual-masking samples after spatial alignment, topographic parameter calculation and standardization processing; then automatically extracting the category, area and simplified contour coordinates in the mask to generate structured text containing summary and detailed segmentation information, forming image-mask-text three-modal alignment training data; based on a pre-trained visual-language large model, fine-tuning is performed through LoRA technology, a forward mask generation branch and text-mask cycle consistency constraint based on an analytical algorithm are introduced, multi-task joint loss optimization is used, and accurate alignment of the three modal spaces is realized; end-to-end automatic mapping from data to maps is realized through the trained model. The application solves the problems of low efficiency, dependence on experience and uninterpretable results of traditional methods, and significantly improves the intelligent level and reliability of mapping.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and multimodal image processing technology, and in particular to a deep-sea geological mapping method and system based on a multimodal large model. Background Technology

[0002] Seafloor geological mapping is the cornerstone of marine scientific research, resource exploration, environmental assessment, and marine engineering construction. Traditionally, this work has relied heavily on marine scientists manually interpreting acoustic data collected by equipment such as multibeam echo sounders and side-scan sonar. This process not only requires hundreds or even thousands of hours of meticulous interpretation by experts, but the results are also easily influenced by individual experience, subjective judgment, and differences in professional background. This can lead to significant inconsistencies in the analysis results of the same area by different experts or at different times, thus hindering the standardization, efficiency, and repeatability of seafloor geological mapping.

[0003] Although agencies such as the U.S. Geological Survey (USGS) and the National Oceanic and Atmospheric Administration (NOAA) have accumulated public hydrological survey data covering tens of thousands of square kilometers and spanning decades, the application of these valuable data resources in the field of deep learning is still in its relatively early stages. Current research focuses on using traditional machine learning algorithms (such as random forests and Gaussian mixture models) or shallow neural networks to classify and predict seabed sediment types or benthic habitats. These methods often rely heavily on artificially designed features, such as specific textures and statistics extracted from backscattering intensity or water depth data. However, the relationship between seabed acoustic responses (such as backscattering) and underlying geological properties is extremely complex and nonlinear. Designing a feature set that can universally and robustly represent this relationship is itself a major unsolved challenge, which directly limits the upper limit of model performance and generalization ability.

[0004] While convolutional neural networks excel in image recognition, they face the following challenges in seabed geological mapping: Data annotation bottleneck: Fine annotation relies on expensive grab sampling, video observation or expert manual drawing, resulting in an extreme scarcity of large-scale training datasets; Modality fusion bottleneck: Existing methods only utilize visual data such as acoustic images, failing to effectively integrate geological prior knowledge, classification standards, and linguistic descriptions, and thus cannot achieve deep feature learning that conforms to geological logic.

[0005] Although datasets such as SeafloorAI have recently begun to incorporate language descriptions, existing technologies only use text as auxiliary information and have not yet achieved deep integration of visual information and geological semantics. In particular, they lack the ability to reverse-engineer pixel-level geological mapping results from semantic descriptions, which is a key requirement for achieving automated, interpretable, and geologically compliant practical applications.

[0006] To address the aforementioned challenges, the field of deep-sea geological mapping urgently needs a technical solution that can fully utilize multimodal data, deeply integrate visual features and geological semantics, achieve high-performance automated mapping with limited annotation, and possess the ability to generate geological maps from linguistic descriptions. In particular, a method is needed that can overcome the limitations of traditional approaches and achieve intelligent processing throughout the entire process from data understanding to map generation. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this invention aims to provide a deep-sea geological mapping method and system based on a multimodal large model. This method enables end-to-end automated generation from multi-source marine geological data to interpretable geological maps, and possesses the ability to reverse-generate mapping results based on semantic descriptions, thereby significantly improving mapping efficiency, objectivity, and intelligence.

[0008] The technical solution adopted in this invention is as follows: A deep-sea geological mapping method based on a multimodal large model includes the following steps: (1) Acquire multi-source marine geological data, perform spatial reprojection alignment on the multi-source marine geological data; cut the aligned image into image blocks, and construct a sample set for model training and testing; (2) Generate a structured text description containing general and detailed segmentation information for the geological mask of each image block; align the structured text description with the corresponding four-channel image and the original mask to construct an image-mask-text three-modal training sample pair; (3) Based on the image-mask-text three-modal training sample pairs, construct a vision-language-mask three-modal joint model; perform low-rank adaptive fine-tuning based on the pre-trained multimodal large model, guide the optimization of model parameters through the multi-task joint loss function, and enhance the alignment accuracy and consistency of visual features, text semantics and pixel-level masks in the three-modal space; (4) Input the multi-source data of the area to be filled into the trained model to complete the end-to-end automated geological mapping from data to map.

[0009] Further, step (1) includes: acquiring backscattered images, water depth measurement data and their corresponding pixel-level sediment mask labels; calculating topographic derived data, including slope maps and roughness maps, based on the water depth measurement data; performing spatial reprojection on the backscattered images, water depth measurement data, slope maps, roughness maps and pixel-level sediment mask labels to achieve spatial alignment of all data; using the sliding window method to cut the aligned images into multiple image blocks of regular size, and filtering the cut image blocks; constructing a training sample set using the filtered samples; each sample in the sample set contains four visual inputs: backscattered images, water depth, slope, and roughness, as well as corresponding sediment and landform semantic labels.

[0010] Furthermore, step (2) includes the following sub-steps: (2.1) Perform feature extraction and quantification analysis on the sediment mask and landform mask of each image block, automatically identify the activated channels in the mask to determine the geological category and quantity; calculate the pixel area ratio of each geological category in the image block, use the contour extraction algorithm to obtain the polygon contour coordinates, area value and hole structure; use the polygon simplification algorithm to compress the polygon contour coordinates; (2.2) Based on the extracted quantized features and simplified polygon contour coordinates, a templated natural language generation method is used to generate a structured text description for each image patch, including a general description of the category area ratio in the form of a declarative sentence, and a detailed description containing the key coordinate point sequence and geometric attributes. The generated structured text description is aligned with the corresponding backscatter, water depth, slope, roughness and the original mask to construct an image-mask-text three-modal training sample pair.

[0011] Further, step (3) includes: constructing a vision-language-mask trimodal joint model based on a multimodal large model, performing low-rank adaptive fine-tuning based on the pre-trained multimodal large model, its visual encoder receiving backscatter, water depth, slope and roughness four-channel images as input, extracting multi-scale visual features; its text decoder generating structured geological descriptions based on visual feature autoregression, and calculating text generation loss; adding two mask generation branches, the first branch being a forward mask generation branch, which directly decodes the extracted visual features into pixel-level geological masks through an additional decoder, and calculates forward mask loss; the second branch being a text-to-mask cycle consistency branch, which converts the structured geological descriptions generated by the model into pixel-level semantic masks through a geometric topology parsing algorithm, and calculates cycle consistency loss; combining text generation loss, forward mask loss and cycle consistency loss to construct a multi-task joint loss function to guide the optimization of model parameters, and enhance the alignment accuracy and consistency of visual features, text semantics and pixel-level masks in the trimodal space.

[0012] Further, the step of converting the structured geological description generated by the model into a pixel-level semantic mask using a geometric topology parsing algorithm and calculating the cycle consistency loss includes: One-dimensional coordinate sequences are extracted from structured geological description text using predefined regular expressions. A semantic nesting recognition mechanism classifies and maps these coordinate sequences into multi-level geometric topologies based on external boundaries and internal hole identifiers within the text. The shapely library is then used to reconstruct these geometric topologies into polygon objects. To address coordinate drift issues during text generation, buffer-based topology correction and vertex snapping are performed to eliminate self-intersections or unclosed defects in the polygons, ensuring the physical geometric validity of the reconstructed patches. The verified polygon objects are mapped to corresponding sediment or geomorphic feature channels according to geological categories, and then converted into pixel-level semantic masks with the same resolution as the input image using a rasterization engine. Cyclic consistency loss is obtained by calculating the pixel-level difference between the pixel-level semantic mask and the real mask labels.

[0013] Furthermore, the multi-task joint loss function uses the AdamW optimizer for parameter updates, where the exponential decay rate beta_1 of the first moment estimation is 0.9, the exponential decay rate beta_2 of the second moment estimation is 0.999, the initial learning rate is 5e-5, the batch size is 4, and cosine annealing learning rate scheduling is adopted. All parameters are optimized synchronously through backpropagation, forcing the model to learn an accurate, bidirectional aligned representation between visual features, text semantics, and mask space.

[0014] Furthermore, in step (4), the automated geological mapping includes two switchable output modes: a visual-mask mode, which uses the forward mask of the model to generate branches and directly generates pixel-level geological segmentation maps based on the input image; and a visual-language-mask mode, which converts the structured geological description into mapping results to obtain standardized deep-sea geological maps and corresponding interpretable text reports.

[0015] To achieve the above objectives, the present invention also provides a deep-sea geological mapping system based on a multimodal large model, including one or more processors for implementing the above-described deep-sea geological mapping method based on a multimodal large model.

[0016] To achieve the above objectives, the present invention also provides an electronic device, including a memory and a processor, wherein the memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the above-described deep-sea geological mapping method based on a multimodal large model.

[0017] To achieve the above objectives, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described deep-sea geological mapping method based on a multimodal large model.

[0018] The beneficial effects of this invention are as follows: By constructing a unified multimodal model, introducing cyclic consistency constraints between forward mask generation and text-to-mask conversion, and utilizing analytical algorithms to achieve efficient semantic-to-mask conversion, this invention effectively solves the three major challenges of traditional mapping methods: low efficiency, reliance on human experience, and uninterpretable results. At the technical level, multi-task joint optimization improves mapping accuracy and model robustness; at the application level, it provides a dual output mode that is both efficient and direct, and highly interpretable, balancing efficiency and reliability; at the industry level, it promotes the paradigm shift of seabed geological mapping towards standardization and intelligence, providing reliable technical support for marine resource development and environmental assessment. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort: Figure 1 This is a flowchart illustrating a deep-sea geological mapping method based on a multimodal large model, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a deep-sea geological mapping system based on a multimodal large model, provided in an embodiment of the present invention. Figure 3 This is a schematic diagram of an electronic device provided according to an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0022] The core innovation of this invention, a deep-sea geological mapping method and system based on a multimodal large model, lies in constructing a "visual-language-mask" trimodal joint learning and collaborative generation framework. Firstly, it breaks through traditional unidirectional or bimodal learning modes by designing a forward mask generation branch and a cyclic consistency constraint from text to mask. This achieves direct mapping from image to text and from image to mask within a unified model, as well as closed-loop verification from text to mask, forming a collaboratively supervised trimodal alignment learning mechanism. Secondly, it proposes a semantic-to-spatial transformation based on an analytical algorithm. The structured text description (including simplified geometric coordinates) generated by the model is converted into a pixel-level mask using a non-neural network deterministic algorithm, constructing an efficient, accurate, and interpretable "language-driven" mapping path. Thirdly, end-to-end optimization is performed through a multi-task joint loss function (text generation loss, forward mask loss, and cyclic consistency loss), forcing the model to establish a precise and consistent alignment relationship between visual features, geological semantics, and mask space within a shared representation space. This significantly improves the model's generalization ability and the reliability of output results under limited annotation. This invention effectively overcomes the three major challenges of subjective human interpretation, lack of labeled data, and fragmented modal semantics. It promotes the intelligent transformation of seabed geological mapping from experience-dependent to data- and knowledge-driven, providing reliable, efficient, and interpretable technical support for marine resource exploration, environmental assessment, and engineering construction, and has broad prospects for standardized application.

[0023] Figure 1 This is a flowchart illustrating a deep-sea geological mapping method based on a multimodal large model, as provided in an embodiment of the present invention. Figure 1 As shown, the deep-sea geological mapping method based on a multimodal large model in this embodiment of the invention includes the following steps: Step S1: Multi-source data processing and training sample construction. Backscattered images, water depth measurement data, and their corresponding pixel-level sediment mask labels are obtained from public databases (such as the USGS). Based on the water depth measurement data, terrain-derived data, including slope maps and roughness maps, are calculated. Spatial reprojection is performed on the backscattered images, water depth measurement data, slope maps, roughness maps, and sediment mask labels to ensure accurate spatial alignment of all data. The large-scale aligned image is then segmented into multiple regularly sized image patches to construct a sample set for model training and testing.

[0024] Furthermore, step S1 specifically includes the following sub-steps: Step S11: Preprocessing and Spatial Alignment of Multi-Source Marine Geological Data. Multi-source heterogeneous data, including backscatter intensity images, multibeam bathymetry data, and sediment and geomorphological classification vector labels, were acquired by integrating publicly available databases such as the U.S. Geological Survey (USGS). The bathymetry data in Arc / Info Binary Grid format was normalized to GeoTIFF format using the Translate function of the GDAL library with the COMPRESS=LZW compression option configured to preserve all georeferenced information. The coordinate reference system (CRS) of each data source was analyzed using an intelligent coordinate recognition module, and all data were uniformly reprojected to the WGS 84 / UTM zone 19N coordinate system (coordinate system code EPSG:32619). Using the 10-meter resolution of the backscatter images as a baseline, the 30-meter resolution bathymetry data was upsampled to 10-meter resolution using a bilinear resampling method.

[0025] Step S12: Spatial Registration and Topographic Derivative Parameter Calculation. Based on the geometric reference frame of the backscattered image, the spatial intersection of each data source is calculated, and the water depth data and its derived data are accurately registered to the same geographic grid. Bilinear resampling is used to process continuous data (water depth, slope, roughness), and nearest neighbor resampling is used to process categorical data. Based on the registered water depth data, a slope distribution map is generated using GDAL's DEM processing tool (based on the Horn algorithm); simultaneously, a roughness distribution map is generated by calculating the ratio of the actual area to the projected area of ​​the topographic surface, quantifying the complexity of the seabed topography.

[0026] Step S13: Semantic Mask Generation and Training Sample Construction. Using the rasterio and geopandas libraries (two core libraries in Python for geospatial data analysis), sediment classification vector labels are converted into 16-channel one-hot encoded masks, and landform classification vector labels are converted into 21-channel one-hot encoded masks, forming pixel-level semantic labels. A sliding window method is used to segment the large-scale (single image resolution is typically above 30000×30000 pixels) aligned multi-source images into 256×256 pixel image patches. To construct a high-quality training set, the segmented image patches are automatically filtered: the proportion of effective geomorphic categories (non-background categories) in the mask corresponding to each image patch is calculated, and image patches with an effective pixel proportion lower than a preset threshold (e.g., 20%) are removed to exclude regions with excessively low information content or mainly invalid values. Finally, the filtered samples are used to construct the training sample set. Each sample contains four visual inputs (4×3×256×256) of backscatter, water depth, slope, and roughness, as well as corresponding semantic labels of sediment (16×256×256) and landform (21×256×256).

[0027] Step S2: Automatic generation of structured geological semantic descriptions. For the geological masks (sediment and geomorphic areas) of each image patch in Step S1, structured text descriptions are automatically generated. First, features such as mask category, area proportion, and polygon outline are extracted, and a simplified algorithm is used to compress the outline coordinates. Based on this, two types of descriptions are generated using a template method: declarative text summarizing the category proportion, and detailed segmentation text containing key coordinates. Then, this text description is strictly aligned with the corresponding four-channel image and the original mask to construct a "image-mask-text" three-modal training sample pair. Stratified sampling and consistency verification are used to ensure dataset quality.

[0028] Furthermore, step S2 specifically includes the following sub-steps: Step S21: Automatic generation and multimodal alignment of geological semantic descriptions. Feature extraction and quantification analysis are performed on the sediment mask and geomorphic mask for each image patch: activated channels in the mask are automatically identified to determine the geological category and quantity; the pixel area ratio of each category in the image patch is calculated. Simultaneously, a contour extraction algorithm is used to obtain the boundary coordinates, area values, and hole structures of each polygon region. A polygon simplification algorithm (such as the Douglas-Peucker algorithm) is used to simplify the polygon contour coordinates, setting a tolerance threshold of 2.0 pixels. While maintaining key geometric features, the number of contour points is reduced from hundreds to dozens of key points.

[0029] Step S22: Multimodal Training Sample Construction and Quality Control. Based on the extracted quantized features and simplified coordinates, a templated natural language generation method is used to generate structured text descriptions for each image patch. These include a general description of the category area ratio in declarative sentences, and a detailed description containing key coordinate point sequences and geometric attributes, achieving the transformation from pixel-level masks to concise semantic descriptions. The generated text descriptions are precisely aligned with the corresponding four images and the original masks to construct "image-mask-text" three-modal training sample pairs. A stratified sampling strategy is used to divide the training set and the test set (test set accounting for 15%) to ensure consistent distribution across categories.

[0030] Specifically, based on the extracted quantitative features and simplified coordinates, a structured description is generated using a preset text template; a general description, in the format of "X types of sediments exist: A(XX%), B(XX%)..."; and a detailed segmentation description, including the category name (with additional information). <seg>The training dataset includes labels, key coordinate sequences of each polygon, hole information, and pixel area. The generated structured text descriptions are precisely aligned with the corresponding backscattered, water depth, slope, and roughness images, as well as the original pixel-level masks, constructing a three-modal training sample pair of multi-source images, masks, and text. The correspondence between the simplified coordinate sequences in the text descriptions and the original geometric features of the masks is verified to ensure a strict spatial correspondence between image data and mask labels, and that the text descriptions accurately and efficiently reflect the semantic and spatial features of the masks. A stratified sampling strategy is used to divide the training and test sets, maintaining a consistent distribution ratio for each category. Cross-validation of the consistency among text, images, and masks is performed to remove incomplete or misaligned samples, ultimately constructing a high-quality, multimodal aligned training dataset to provide accurate supervision signals for model training.

[0031] Step S3: Trimodal Joint Model Training and Optimization. A "vision-language-mask" trimodal joint training framework based on a multimodal large model is constructed. Low-rank adaptive (LoRA) fine-tuning is performed on a pre-trained multimodal large model (e.g., Qwen2.5-VL-7B). Its visual encoder receives four-channel images (backscattered, water depth, slope, and roughness) as input, extracting multi-scale visual features. Its text decoder generates structured geological descriptions based on visual feature autoregression, achieving cross-modal mapping from image to text and calculating the text generation loss. On this basis, two mask generation branches are added: the first branch is a forward mask generation branch, which directly decodes the extracted visual features into pixel-level geological masks through an additional decoder and calculates the forward mask loss; the second branch is a text-to-mask cycle consistency branch, which parses and converts the model-generated structured geological description (including simplified contour coordinate information) into pixel-level semantic masks and calculates the difference between this reconstructed mask and the real mask label as the cycle consistency loss. By combining text generation loss, forward masking loss, and cycle consistency loss, a multi-task joint loss function is constructed to jointly guide the optimization of model parameters, thereby enhancing the alignment accuracy and consistency of visual features, text semantics, and pixel-level masks in the three-modal space.

[0032] Furthermore, step S3 specifically includes the following sub-steps: Step S31: Multimodal Base Model Construction and Visual Feature Extraction. The Qwen2.5-VL-7B pre-trained model is loaded as the basic architecture. Low-rank adaptive (LoRA) technology is used to efficiently fine-tune the parameters of seven key linear modules in the model's self-attention sublayer (including query matrix parameter q_proj, key matrix parameter k_proj, value matrix parameter v_proj, and output projection parameter o_proj) and feedforward neural network sublayer (including gate projection parameter gate_proj, up-dimensional projection parameter up_proj, and down-dimensional projection parameter down_proj). Specifically, the fine-tuning process involves freezing the original pre-trained weight parameters of the above linear modules and introducing low-rank decomposition matrices A and B with rank r=8 in parallel alongside each linear module. During the training phase, gradient updates are performed only on the parameters of the low-rank decomposition matrices, and the fine-tuned weights are scaled using a scaling factor α=16. The model's visual encoder receives four types of images—backscattered, water depth, slope, and roughness—as input, extracting multi-scale visual features. The text decoder generates structured geological descriptions based on visual feature autoregression and uses cross-entropy loss to calculate the text generation loss L_text.

[0033] Step S32: Forward Mask Generation and Direct Supervision. Based on the features output by the visual encoder, an additional mask decoder based on the U-Net architecture is added. This decoder restores the feature map to 256×256 resolution through four upsampling layers, and finally outputs a 16-channel (sediment) or 21-channel (landform) mask prediction through a 1×1 convolution, directly decoding the visual features into pixel-level geological masks (sediment masks and landform area masks). The forward mask loss L_mask between this generated mask and the real mask labels is calculated. This loss is the cross-entropy loss (this loss can also be Dice loss), achieving direct supervision of the mask segmentation task.

[0034] Step S33: Cyclic Consistency Constraint from Text to Mask. The structured geological description generated by the model (containing simplified contour coordinates, pore structure, etc.) is converted into a pixel-level semantic mask using the improved geometric topology parsing algorithm of this invention. Specifically, the process is as follows: First, a one-dimensional coordinate sequence is extracted from the structured geological description text using predefined regular expressions. Then, through the semantic nesting recognition mechanism unique to the improved algorithm of this invention, the coordinate sequence is classified and mapped into a multi-level geometric topology structure based on the external boundary (Exterior) and internal pore (Interiors) identifiers in the text. Subsequently, the shapely library (a Python library for manipulating and analyzing geometric objects) is called to reorganize the above structure into polygon objects. Finally, to address the potential coordinate drift problem during text generation, a buffer-based... Topological correction and vertex snapping of the region eliminate self-intersection or unclosed defects of polygons to ensure the physical and geometric legitimacy of the reconstructed patches. Next, the verified polygon objects are mapped to the corresponding sediment or landform feature channels according to the geological category. The rasterization engine is used to convert them into a 256×256 pixel-level one-hot encoded mask with the same resolution as the input image, thus completing the reconstruction path of "text → mask". Finally, the cycle consistency loss L_cycle is obtained by calculating the pixel-level difference between the reconstructed mask and the real mask label. This loss is then backpropagated in conjunction with the text generation loss L_text to construct a closed-loop verification mechanism of "image → text → mask", which strengthens the model's understanding and constraint of geological distribution patterns from the dimension of spatial distribution consistency.

[0035] Compared to existing general analytical algorithms, the improved geometric topology analytical algorithm of this invention solves the geometric conflicts and self-intersection problems that easily occur when deep learning-generated text describes complex geological topology (such as nested holes) by introducing semantic nesting recognition and an automated topology correction mechanism. At the same time, the algorithm realizes the direct mapping from unstructured text coordinates to multi-channel semantic masks, which not only improves the physical legitimacy of the "text-mask" conversion, but also significantly enhances the model's understanding accuracy of the large-scale seabed geological distribution patterns by constructing a closed-loop verification mechanism.

[0036] Step S34: Multi-task joint optimization and trimodal alignment. The text generation loss, forward masking loss, and cycle consistency loss are combined to construct a multi-task joint loss function: L_total = L_text + L_mask + L_cycle. The AdamW optimizer is used for parameter updates, where the exponential decay rate beta_1 for first-moment estimation is set to 0.9, and the exponential decay rate beta_2 for second-moment estimation is set to 0.999. The initial learning rate is set to 5e-5, the batch size to 4, and cosine annealing is used for learning rate scheduling. Mixed-precision training is performed for 20 epochs on 8 NVIDIA A100 GPUs. All parameters are simultaneously optimized through backpropagation, forcing the model to learn accurate, bidirectional aligned representations between visual features, text semantics, and the mask space. This joint training mechanism not only optimizes the image-to-text and image-to-mask mappings but also strengthens the consistency of representations within the trimodal shared space through the text-to-mask cycle consistency constraint, thereby improving the model's understanding of geological scenes and the robustness of the output results.

[0037] Step S4: Generation and Application of Geological Mapping Results. First, the data for the area to be surveyed is preprocessed. This preprocessing includes: using spatial projection transformation to uniformly reproject heterogeneous data such as backscattering and water depth to the WGS 84 / UTM coordinate system, and achieving spatial resolution consistency alignment through bilinear resampling; calculating the corresponding slope and roughness maps based on the aligned water depth data; performing numerical normalization on each modal feature to eliminate dimensional differences, and using a sliding window technique with preset overlap to cut continuous data into 256×256 pixel image blocks. Subsequently, the multi-source data preprocessed above is input into the trained model to achieve end-to-end automated geological mapping. The system offers two switchable output modes: one is the efficient and direct "vision → mask" mode, which uses the model's forward mask generation branch to directly and quickly generate pixel-level geological segmentation maps based on the input image; the other is the highly interpretable "vision → language → mask" mode, which reuses the cyclic consistency path from the training phase: first, it generates a structured geological description, and then uses the same rule-based geometric topology parsing algorithm as during training to transform the semantic description into a mapping result, ultimately obtaining standardized deep-sea geological maps and corresponding interpretable text reports.

[0038] Corresponding to the aforementioned embodiments of the deep-sea geological mapping method based on a multimodal large model, the present invention also provides embodiments of a deep-sea geological mapping system based on a multimodal large model.

[0039] See Figure 2 The deep-sea geological mapping system based on a multimodal large model provided in this embodiment includes one or more processors for implementing the deep-sea geological mapping method based on a multimodal large model in the above embodiment.

[0040] The embodiments of the deep-sea geological mapping system based on a multimodal large model of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar devices or systems. The system embodiments can be implemented through software, hardware, or a combination of both. Taking software implementation as an example, as a logical system, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, such as... Figure 2 The diagram shown is a hardware structure diagram of any device with data processing capabilities within the deep-sea geological mapping system based on a multimodal large model according to the present invention. (Except for...) Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown, any data processing device in which the system is located in the embodiment may also include other hardware depending on the actual function of the data processing device, which will not be described in detail here.

[0041] The implementation process of the functions and roles of each unit in the above system is detailed in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0042] For the system embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0043] Corresponding to the aforementioned embodiments of the deep-sea geological mapping method based on a multimodal large model, this application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the deep-sea geological mapping method based on a multimodal large model as described above. Figure 3 The diagram shown is a hardware structure diagram of any device with data processing capabilities for the deep-sea geological mapping method based on a multimodal large model provided in this application embodiment.

[0044] Corresponding to the aforementioned embodiments of the deep-sea geological mapping method based on a multimodal large model, this embodiment of the invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the deep-sea geological mapping method based on a multimodal large model as described in the above embodiments.

[0045] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be any data processing device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0046] The above embodiments are only used to illustrate the design concept and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made based on the principles and design ideas disclosed in the present invention are within the protection scope of the present invention.< / seg>

Claims

1. A deep-sea geological mapping method based on a multimodal large model, characterized in that, Includes the following steps: (1) Acquire multi-source marine geological data and perform spatial reprojection alignment on the multi-source marine geological data; The aligned image is cut into image patches to construct a sample set for model training and testing; (2) Generate a structured text description containing general and detailed segmentation information for the geological mask of each image block; align the structured text description with the corresponding four-channel image and the original mask to construct an image-mask-text three-modal training sample pair; (3) Based on the image-mask-text three-modal training sample pairs, construct a vision-language-mask three-modal joint model; perform low-rank adaptive fine-tuning based on the pre-trained multimodal large model, guide the optimization of model parameters through the multi-task joint loss function, and enhance the alignment accuracy and consistency of visual features, text semantics and pixel-level masks in the three-modal space; (4) Input the multi-source data of the area to be filled into the trained model to complete the end-to-end automated geological mapping from data to map.

2. The deep-sea geological mapping method based on a multimodal large model according to claim 1, characterized in that, Step (1) includes: acquiring backscattered images, water depth measurement data, and their corresponding pixel-level sediment mask labels; calculating topographic derived data, including slope maps and roughness maps, based on the water depth measurement data; performing spatial reprojection on the backscattered images, water depth measurement data, slope maps, roughness maps, and pixel-level sediment mask labels to achieve spatial alignment of all data; using the sliding window method to cut the aligned images into multiple image blocks of regular size, and filtering the cut image blocks; constructing a training sample set using the filtered samples; each sample in the sample set contains four visual inputs: backscattered images, water depth, slope, and roughness, as well as corresponding sediment and geomorphological semantic labels.

3. The deep-sea geological mapping method based on a multimodal large model according to claim 2, characterized in that, Step (2) includes the following sub-steps: (2.1) Perform feature extraction and quantitative analysis on the sediment mask and landform mask of each image block, and automatically identify the activated channels in the mask to determine the geological category and quantity; Calculate the pixel area ratio of each geological category in the image patch, and use a contour extraction algorithm to obtain the polygon contour coordinates, area values ​​and hole structures; then use a polygon simplification algorithm to compress the polygon contour coordinates. (2.2) Based on the extracted quantized features and simplified polygon contour coordinates, a templated natural language generation method is used to generate a structured text description for each image patch, including a general description of the category area ratio in the form of a declarative sentence, and a detailed description containing the key coordinate point sequence and geometric attributes. The generated structured text description is aligned with the corresponding backscatter, water depth, slope, roughness and the original mask to construct an image-mask-text three-modal training sample pair.

4. The deep-sea geological mapping method based on a multimodal large model according to claim 2, characterized in that, Step (3) includes: constructing a vision-language-mask trimodal joint model based on a multimodal large model, performing low-rank adaptive fine-tuning based on the pre-trained multimodal large model, its visual encoder receiving backscatter, water depth, slope and roughness four-channel images as input, extracting multi-scale visual features; its text decoder generating structured geological descriptions based on visual feature autoregression, and calculating text generation loss; adding two mask generation branches, the first branch being the forward mask generation branch, which directly decodes the extracted visual features into pixel-level geological masks through an additional decoder, and calculates the forward mask loss; the second branch being the text-to-mask cycle consistency branch, which converts the structured geological descriptions generated by the model into pixel-level semantic masks through a geometric topology parsing algorithm, and calculates the cycle consistency loss; combining the text generation loss, forward mask loss and cycle consistency loss to construct a multi-task joint loss function to guide the optimization of model parameters, and enhance the alignment accuracy and consistency of visual features, text semantics and pixel-level masks in the trimodal space.

5. The deep-sea geological mapping method based on a multimodal large model according to claim 4, characterized in that, The structured geological description generated by the model is converted into a pixel-level semantic mask through a geometric topology parsing algorithm, and the cycle consistency loss is calculated. include: One-dimensional coordinate sequences are extracted from structured geological description text using predefined regular expressions. A semantic nesting recognition mechanism classifies and maps these coordinate sequences into multi-level geometric topologies based on external boundaries and internal hole identifiers within the text. The shapely library is then used to reconstruct these geometric topologies into polygon objects. To address coordinate drift issues during text generation, buffer-based topology correction and vertex snapping are performed to eliminate self-intersections or unclosed defects in the polygons, ensuring the physical geometric validity of the reconstructed patches. The verified polygon objects are mapped to corresponding sediment or geomorphic feature channels according to geological categories, and then converted into pixel-level semantic masks with the same resolution as the input image using a rasterization engine. Cyclic consistency loss is obtained by calculating the pixel-level difference between the pixel-level semantic mask and the real mask labels.

6. The deep-sea geological mapping method based on a multimodal large model according to claim 4, characterized in that, The multi-task joint loss function uses the AdamW optimizer for parameter updates. The exponential decay rate beta_1 of the first moment estimation is 0.9, and the exponential decay rate beta_2 of the second moment estimation is 0.

999. The initial learning rate is 5e-5, the batch size is 4, and cosine annealing learning rate scheduling is adopted. All parameters are optimized synchronously through backpropagation, forcing the model to learn an accurate, bidirectional aligned representation between visual features, text semantics, and mask space.

7. The deep-sea geological mapping method based on a multimodal large model according to claim 1, characterized in that, In step (4), the automated geological mapping includes two switchable output modes: visual-mask mode, which uses the forward mask of the model to generate branches and directly generates pixel-level geological segmentation maps based on the input image; and visual-language-mask mode, which transforms the structured geological description into mapping results to obtain standardized deep-sea geological maps and corresponding interpretable text reports.

8. A deep-sea geological mapping system based on a multimodal large model, characterized in that, It includes one or more processors for implementing the deep-sea geological mapping method based on a multimodal large model as described in any one of claims 1-7.

9. An electronic device comprising a memory and a processor, characterized in that, The memory is coupled to the processor; wherein the memory is used to store program data, and the processor is used to execute the program data to implement the deep-sea geological mapping method based on a multimodal large model as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the deep-sea geological mapping method based on a multimodal large model as described in any one of claims 1-7.

Images