Adaptive coastline edge detection method based on multi-modal large model

Abstract

The invention discloses a self-adaptive coastline edge detection method based on a multi-modal large model, particularly relates to the technical field of remote sensing image processing and computer vision, and provides a multi-modal edge detection large model DirectSAM-RS capable of accepting text prompt in the remote sensing field based on DirectSAM model architecture by introducing a text encoder and a multi-modal fusion module; the method comprises the following steps: constructing a large-scale pre-training data set RemoteContour-34k formed by a remote sensing image-text-edge triple; the DirectSAM-RS model is subjected to multi-modal pre-training by using a RemoteContour-34k data set, so that field adaptation from a natural image to a remote sensing image is realized, and a basis is provided for fine tuning of a coastline edge detection task in the remote sensing field; and carrying out fine adjustment on the multi-modal edge detection large model on the coastline edge detection data set, and optimizing the coastline edge detection capability of the model.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing and computer vision, and in particular to an adaptive coastline edge detection method based on a multi-modal large model. Background Art

[0002] At present, coastline edge detection mainly relies on traditional image processing algorithms or small-scale dedicated models based on deep learning. Traditional methods rely on artificially designed features and have poor adaptability to complex scenes; deep learning models usually require a large amount of labeled data for training specific scenes, which is difficult to be used in multiple scenes and tasks, and is prone to overfitting when training data is insufficient. In addition, existing methods usually find it difficult to fully utilize multimodal data in remote sensing images (such as optical images, text prompts, image semantics, etc.), resulting in poor performance of the model when processing complex coastline scenes (such as coastal distribution of buildings, vegetation-covered areas, artificial ports, etc.).

[0003] In recent years, the development of multimodal large models has brought new possibilities for solving the above problems. By integrating multimodal information such as images and text, multimodal large models have demonstrated strong generalization and task adaptability. However, at present, such models are mainly used in the field of natural images, and research on remote sensing images (especially coastline edge detection tasks) is still in the initial exploration stage and cannot directly meet actual needs. Summary of the invention

[0004] In view of this, in view of the problems of insufficient accuracy, poor adaptability, low robustness, etc. in the existing coastline edge detection methods, the present invention proposes an adaptive coastline edge detection method based on a multimodal large model. This method introduces a large-scale pre-trained multimodal basic model and combines it with the specific needs of the remote sensing field to solve the shortcomings of traditional methods in terms of strong data dependence, insufficient scene adaptability, and weak multimodal information fusion capabilities, so as to solve the problems raised in the background technology.

[0005] In order to achieve the above object, the present invention provides the following technical solution: an adaptive coastline edge detection method based on a multimodal large model, comprising the following steps: S1: Based on the DirectSAM model architecture, the text encoder and multimodal fusion module are introduced, and DirectSAM-RS, a large multimodal edge detection model that can accept text prompts in the remote sensing field, is proposed as the basic architecture for the coastline edge detection task; S2: Construct a large-scale pre-training dataset RemoteContour-34k consisting of remote sensing image-text-edge triplets, focusing on annotating edge features under complex terrain and multiple interference factors, to provide a basis for pre-training of the multimodal edge detection large model DirectSAM-RS; S3: Use the RemoteContour-34k dataset to perform multimodal pre-training on the multimodal edge detection large model DirectSAM-RS to achieve domain adaptation from natural images to remote sensing scenes, so as to improve the model's perception of coastline features, including the accurate capture of the coherence and details of the coastline edge, laying the foundation for fine-tuning the coastline edge detection task in the remote sensing field; S4: Fine-tune the multimodal edge detection model DirectSAM-RS on the coastline edge detection dataset to optimize the model's ability to detect coastline edges, enhance the model's robustness and accuracy to complex edge changes, and achieve efficient detection and accurate extraction of coastline edges; S5: Verify the performance of the DirectSAM-RS model in the coastline edge detection task in both Zero-Shot and Fine-Tuned modes, prove its superiority, and visualize the inference results.

[0006] Preferably, the specific process of constructing the multimodal edge detection large model DirectSAM-RS in step S1 is: S11: Visual encoder structure and its initialization: Use the multi-level visual encoder of the DirectSAM model as the visual encoder of the multimodal edge detection large model DirectSAM-RS and use its parameters for initialization; S12: Encode textual prompts using a global text encoder with fixed parameters: The BERT text encoder is used as the global text encoder to encode the textual cues containing the coastline into a compact vector in the semantic space: ; in, For text prompts, is the embedding dimension, Refers to the length The sequence contains 1 Classification vector and word embedding vectors, represents the real number domain, which means that all elements of the matrix are real numbers. represents the encoding process of the pre-trained text encoder BERT, is a compact vector in the semantic space; S13: Insert cross-attention feature fusion module: In each stage of the DirectSAM model visual encoder, there are 4 stages in total, and a cross-attention layer is inserted to achieve dynamic interaction between image features and text semantics; Set up The stage image features are , Represents the number of color channels of the image feature matrix, Represents the height of the feature matrix image, Indicates the width of the feature matrix image; is a compact vector in semantic space, and the cross attention is calculated as: ; in, is a learnable parameter, , , is the process vector in the attention mechanism calculation, and the image features are mapped to , the text features are mapped to and , Representation vector The transposed vector of ; is a normalization function for multi-classification problems, defined as ; Indicates The score of each dimension, the denominator is the exponential sum of all dimension scores, the numerator yes The exponential form reflects Relative importance among all scores; The output features are added channel by channel to the original image features: ; in, For the The multimodal fusion features of the stage, Indicates that the output sequence of the cross attention module is adjusted to Dimensions match; S14: Design text prompt template: To enhance semantic guidance, the text Normalize to structured prompts: ; in is the prompt category, which includes buildings, coastlines or roads; is a template for structured text prompts, represented as The edge of the text is used to provide semantic guidance. Through template constraints, the multimodal edge detection model DirectSAM-RS learns to align text semantics with visual edge features and suppress irrelevant regional responses. S15: Lightweight Contour Decoder: The multi-layer perceptron MLP decoder is used to integrate multi-stage fusion features Upsample and fuse to output edge probability map : ; in Expressing the Layer fusion features Perform upsampling operations to unify the features of different layers to the same spatial resolution to facilitate splicing operations; It means that the upsampled multi-level features are combined along the channel dimension as ; It is the concatenation of multi-level multi-modal features, integrating feature information of all levels to form a global feature representation containing multi-scale multi-modal information, providing input for subsequent edge decoding; is the Sigmoid activation function, and the MLP contains two fully connected layers and a ReLU activation layer.

[0007] Preferably, the specific process of constructing the large-scale pre-training data set RemoteContour-34k in step S2 is: S21: Collecting images from existing remote sensing semantic segmentation datasets and its segmentation mask , the segmentation mask is transformed into The process of converting to single-pixel edge labels is as follows: For each segmentation mask Extract all target boundary point sets ; Initialize edge labels is an all-zero matrix, and the boundary point set exist Set the corresponding element in to 1 to get the edge label ; Optimize edge continuity and remove isolated noise points, Indicates the height of the image. Indicates the width of the image; S22: For each image Generate text prompt , generate structured text based on the category labels of the original segmentation mask (such as "coastline" and "road") ,in For the prompt category; S23: Integrating images ,text , edge labels , stored as Triplet, as the basic unit constituting the large-scale pre-training dataset RemoteContour-34k.

[0008] Preferably, the specific process of using the RemoteContour-34k data set to perform multimodal pre-training on the multimodal edge detection large model DirectSAM-RS in step S3 is: S31: Input data: Remote sensing images Input DirectSAM-RS visual encoder, text prompt Input text encoder BERT; where Indicates the height of the image. Represents the width of the image, and 3 represents the number of channels; S32: Decode and calculate loss: The edge probability map output by the multimodal edge detection large model DirectSAM-RS is , and the true edge label Calculate the loss pixel by pixel: ; The optimization goal is to minimize the edge detection loss , the parameters of the multimodal edge detection model DirectSAM-RS Optimize to the optimal model parameters :

[0009] in, Indicates that the function The variable that reaches the minimum value A collection of .

[0010] Preferably, in step S4, a coastline edge detection dataset SLSD is used as fine-tuning data for the multimodal edge detection large model DirectSAM-RS, and the dataset SLSD contains high-resolution remote sensing images and their corresponding coastline edge labels.

[0011] Preferably, in step S5, the performance of the remote sensing multimodal edge detection large model DirectSAM-RS on the coastline edge detection task is verified under two experimental settings, and the specific process is as follows: S51: Verify model performance in Zero-Shot mode: In zero-shot mode, the model is not fine-tuned for the coastline edge detection task, and only relies on the multimodal knowledge learned in the pre-training stage for reasoning. The test images and corresponding text prompts in the SLSD coastline edge detection dataset are input into the model, and the multimodal edge detection large model DirectSAM-RS processes the input through its multimodal fusion module, combining image features with text prompts to generate edge detection results. Using the real annotations of the SLSD dataset, ODS-F (the highest F1 score in the entire dataset), OIS (the highest F1 score in a single image), and LineIoU@3 (with The zero-shot performance of the model is evaluated by using the intersection-over-union ratio of the convolution kernels of different sizes and bold sizes; S52: Performance verification in Fine-Tuned mode: In the fine-tuning mode, the multimodal edge detection large model DirectSAM-RS is fine-tuned on the training set of the SLSD dataset to further optimize its performance in the coastline edge detection task; the fine-tuned multimodal edge detection large model DirectSAM-RS achieves the optimal method on the SLSD dataset.

[0012] The present invention has the following advantages: 1. The multimodal edge detection large model DirectSAM-RS independently constructed by the present invention has strong versatility and adaptability. Compared with the traditional dedicated models that rely on small-scale data sets for training, this model has stronger generalization ability and can accurately detect the edge of the coastline in complex scenes. It has strong adaptability to complex scenes and does not require a large amount of annotation or retraining for specific scenes. It is suitable for multiple scenes and multiple tasks.

[0013] 2. The multimodal edge detection large model DirectSAM-RS designed by the present invention introduces a text prompt mechanism to enhance the flexibility of edge detection tasks by dynamically adjusting the prompt content.

[0014] 3. The present invention converts segmentation masks into edge labels, and can efficiently convert existing remote sensing image segmentation data into edge detection data. Compared with the prior art, it solves the problem of insufficient data in edge detection tasks and reduces the problem of overfitting that is prone to occur when training data is insufficient.

[0015] 4. The present invention ensures the capture of coastline detail information and understanding of global contours in remote sensing images through multi-scale feature fusion, and can better adapt to the diversity and complexity of coastlines under different resolutions and scene conditions. Compared with the existing technology, it meets the actual needs encountered in the processing of remote sensing images (especially coastline edge detection tasks). BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of DirectSAM-RS architecture of edge detection multimodal large model in the method of the present invention; Figure 2 Schematic diagram of a method for converting a segmentation mask into an edge label in the method of the present invention. DETAILED DESCRIPTION

[0017] The following is a description of the implementation of the present invention by specific embodiments. People familiar with the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] like Figure 1 As shown, the present invention provides an adaptive coastline edge detection method based on a multimodal large model, comprising the following steps: S1: Based on the DirectSAM model architecture, we introduced a text encoder (BERT pre-trained text encoder) and a multimodal fusion module to build a large multimodal edge detection model DirectSAM-RS that can accept text prompts in the remote sensing field as the basic architecture for the coastline edge detection task; S2: Construct a large-scale pre-training dataset RemoteContour-34k consisting of remote sensing image-text-edge triplets; S3: Use the RemoteContour-34k dataset to perform multimodal pre-training on the DirectSAM-RS model to achieve domain adaptation from natural images to remote sensing images, providing a basis for fine-tuning the coastline edge detection task in the remote sensing domain; S4: Fine-tune the multimodal edge detection model on the coastline edge detection dataset to optimize the model's ability to detect coastline edges; S5: Verify the performance of the model on the coastline edge detection task in both zero-shot and fine-tuned modes, prove its superiority, and visualize the inference results.

[0019] The specific process of constructing the multimodal large model DirectSAM-RS in step S1 is as follows: S11: Visual encoder structure and its initialization: like Figure 1 As shown, DirectSAM's multi-level visual encoder is used as the visual encoder of DirectSAM-RS to obtain multi-level visual features. :

[0020] ;in is the number of feature channels, is the original image height, is the original image width; S12: Encode textual prompts using a global text encoder with fixed parameters: The BERT text encoder is used as a global text encoder to encode text cues (such as "coastline") into compact vectors in semantic space: ; in, For text prompts, is the embedding dimension, Refers to the length The sequence contains 1 Classification vector and word embedding vectors, represents the real number domain, which means that all elements of the matrix are real numbers. represents the encoding process of the pre-trained text encoder BERT, is a compact vector in the semantic space; S13: Insert cross-attention feature fusion module: Insert cross-attention layers in each stage of DirectSAM visual encoder (4 stages in total) to realize the dynamic interaction between image features and text semantics; The stage image features are , Represents the number of color channels of the image feature matrix, Represents the height of the feature matrix image, Indicates the width of the feature matrix image; is a compact vector in semantic space, and the cross attention is calculated as: ; in, is a learnable parameter, , , is the process vector in the attention mechanism calculation, and the image features are mapped to , the text features are mapped to and , Representation vector The transposed vector of ; is a normalization function for multi-classification problems, defined as ; Indicates The score of each dimension, the denominator is the exponential sum of all dimension scores, the numerator yes The exponential form reflects Relative importance among all scores; The output features are added channel by channel to the original image features: ; in, For the The multimodal fusion features of the stage, Indicates that the output sequence of the cross attention module is adjusted to Dimensions match; S14: Design text prompt template: To enhance semantic guidance, the text Normalize to structured prompts: ; in is the prompt category, which includes buildings, coastlines or roads; is a template for structured text prompts, represented as The edge of the text is used to provide semantic guidance. Through template constraints, the multimodal edge detection model DirectSAM-RS learns to align text semantics with visual edge features and suppress irrelevant regional responses. S15: Lightweight Contour Decoder: A multi-layer perceptron (MLP) decoder is used to fuse multi-stage features. Upsample and fuse to output edge probability map : ; in Indicates Layer fusion features Perform upsampling operations to unify the features of different layers to the same spatial resolution to facilitate splicing operations; It means that the upsampled multi-level features are combined along the channel dimension as ; It is the concatenation of multi-level multi-modal features, integrating feature information of all levels to form a global feature representation containing multi-scale multi-modal information, providing input for subsequent edge decoding; is the Sigmoid activation function, and the MLP contains two fully connected layers and a ReLU activation layer.

[0021] The specific process of constructing the large-scale pre-training dataset RemoteContour-34k in step S2 is as follows: S21: Collect images and their segmentation masks from existing remote sensing semantic segmentation datasets , the segmentation mask is transformed into The process of converting to single-pixel edge labels (such as Figure 2 shown): For each segmentation mask Extract all target boundary point sets ; Initialize edge labels is an all-zero matrix, and the boundary point set exist Set the corresponding element in to 1 to get the edge label ; Optimize edge continuity and remove isolated noise points, Indicates the height of the image. Indicates the width of the image; S22: Generate textual hints for each image , generate structured text based on the category labels of the original segmentation mask (such as "coastline" and "road") ,in is the prompt category; S23: Integrating images ,text , edge labels , stored as Triplet, as the basic unit constituting the large-scale pre-training dataset RemoteContour-34k, contains 34291 sets of triplets.

[0022] The specific process of multi-modal pre-training of DirectSAM-RS using RemoteContour-34k in step S3 is as follows: S31: Input data: Remote sensing images Input DirectSAM-RS visual encoder, text prompt Input text encoder BERT; S32: Decode and calculate loss: The marginal probability map output by the model is , and the true edge label Calculate the loss pixel by pixel: ; The optimization goal is to minimize the edge detection loss , the model parameters Optimize to the optimal model parameters : ; in, Indicates that the function The variable that reaches the minimum value A collection of .

[0023] In step S4, the Shoreline Edge Detection Dataset (SLSD) is used as fine-tuning data. The dataset contains high-resolution remote sensing images and their corresponding shoreline edge labels, covering a variety of geographical environments (such as beaches, rocky coasts, estuaries, etc.). The fine-tuning process aims to make the model better adapt to the unique characteristics of shoreline scenes (such as complex water-land boundaries, multi-scale texture changes, etc.) while retaining its general edge detection capabilities learned in multimodal pre-training.

[0024] In step S5, the performance of the remote sensing multimodal edge detection large model DirectSAM-RS on the coastline edge detection task is verified under two experimental settings. The specific process is as follows: S51: Verify model performance in Zero-Shot mode: In zero-shot mode, the model is not fine-tuned for the coastline edge detection task, and only relies on the multimodal knowledge learned in the pre-training stage for reasoning. The test images and corresponding text prompts in the SLSD coastline edge detection dataset are input into the model. DirectSAM-RS processes the input through its multimodal fusion module, and generates edge detection results by combining image features with text prompts. Using the real annotations of the SLSD dataset, ODS-F (the highest F1 score in the entire dataset), OIS (the highest F1 score in a single image), and LineIoU@3 (with The zero-shot performance of the model is evaluated by using the intersection-over-union ratio of the convolution kernels of different sizes and bold sizes; S52: Performance verification in Fine-Tuned mode: In the fine-tuning mode, this embodiment fine-tunes DirectSAM-RS on the training set of the SLSD dataset to further optimize its performance in the coastline edge detection task; the fine-tuned DirectSAM-RS achieves the optimal method on the SLSD dataset; S53: Experimental results: As shown in Table 1, the performance of the DirectSAM-RS model in both zero-shot (ZS) and fine-tuning (FT) modes has been comprehensively evaluated. The comparison results show that fine-tuning significantly improves the detection accuracy of the model. In zero-shot mode, the ODS, OIS and LineIoU@3 of DirectSAM-RS are 0.639, 0.803 and 0.504 respectively. Although DirectSAM-RS (ZS) has not been fine-tuned, the model still shows a certain ability to detect coastline edges. After fine-tuning, the performance of DirectSAM-RS is significantly improved, with ODS reaching 0.958, OIS reaching 1.000, and LineIoU@3 reaching 0.895, all of which surpass the control groups HED and SegFormer-b5, becoming the best performing method. Table 1 Performance comparison of DirectSAM-RS model on SLSD dataset

[0025] This embodiment compares the detection method of the present invention with the existing detection method. The detection accuracy of the method of the present invention in complex coastline scenes (such as tidal changes, vegetation coverage, artificial embankments, etc.) is significantly improved. The ODS (Optimal Dataset Scale) index after DirectSAM-RS fine-tuning reaches 0.958, which is 7% higher than the existing best method HED, demonstrating excellent detection capabilities for complex coastline contours. Compared with traditional methods based on pixels or small-scale task-specific models, the present invention has improved flexibility and accuracy, and can better adapt to the actual needs of multiple tasks and multiple scenarios. The present invention achieves high reusability and scalability of the method, and only needs to modify a small part of the content to be applied to other remote sensing tasks (such as building contour extraction, road detection, etc.); the present invention is not only suitable for coastline monitoring, but can also be widely used in disaster emergency response, ecological protection and resource management, etc., providing efficient and intelligent technical support for related fields.

[0026] Although the present invention has been described in detail above by general description and specific embodiments, it is obvious to those skilled in the art that some modifications or improvements can be made to the present invention. Therefore, these modifications or improvements made without departing from the spirit of the present invention all belong to the scope of protection claimed by the present invention.

Claims

1. An adaptive coastline edge detection method based on a multimodal large model, characterized by: The steps include: S1: Based on the DirectSAM model architecture, the text encoder and multimodal fusion module are introduced, and then DirectSAM-RS, a large multimodal edge detection model that can accept text prompts in the remote sensing field, is proposed and used as the basic architecture for the coastline edge detection task; S2: Construct a large-scale pre-training dataset RemoteContour-34k consisting of remote sensing image-text-edge triplets, focusing on annotating edge features under complex terrain and multiple interference factors, to provide a basis for pre-training of the multimodal edge detection large model DirectSAM-RS; S3: Use the RemoteContour-34k dataset to perform multimodal pre-training on the multimodal edge detection large model DirectSAM-RS to improve the model's ability to perceive coastline features, including the accurate capture of the continuity and details of the coastline edge, laying the foundation for fine-tuning the coastline edge detection task in the field of remote sensing; S4: Fine-tune the multimodal edge detection model DirectSAM-RS on the coastline edge detection dataset to achieve efficient detection and accurate extraction of coastline edges.

2. The method for adaptive coastline edge detection based on a multimodal large model according to claim 1, characterized in that: The specific process of constructing the multimodal edge detection large model DirectSAM-RS in step S1 is as follows: S11: Visual encoder structure and its initialization: Use the multi-level visual encoder of the DirectSAM model as the visual encoder of the multimodal edge detection large model DirectSAM-RS and use its parameters for initialization; S12: Encode textual prompts using a global text encoder with fixed parameters: The BERT text encoder is used as the global text encoder to encode the textual cues containing the coastline into a compact vector in the semantic space: ； in, For text prompts, is the embedding dimension, Refers to the length The sequence contains 1 Classification vector and word embedding vectors, represents the real number domain, which means that all elements of the matrix are real numbers. represents the encoding process of the pre-trained text encoder BERT, is a compact vector in the semantic space; S13: Insert cross-attention feature fusion module: In each stage of the DirectSAM model visual encoder, there are 4 stages in total, and a cross-attention layer is inserted to achieve dynamic interaction between image features and text semantics; Set up The stage image features are , Represents the number of channels of the image feature matrix, represents the height of the feature matrix, Indicates the width of the feature matrix; is a compact vector in semantic space, and the cross attention is calculated as: ； in, is a learnable parameter, , , is the process vector in the attention mechanism calculation, and the image features are mapped to , the text features are mapped to and , Representation vector The transposed vector of ; is a normalization function for multi-classification problems, defined as ; Indicates The score of each dimension, the denominator is the exponential sum of all dimension scores, the numerator yes The exponential form of ; the output features are added channel by channel to the original image features: ； in, For the The multimodal fusion features of the stage, Indicates that the output sequence of the cross attention module is adjusted to Dimensions match; S14: Design text prompt template: To enhance semantic guidance, the text Normalize to structured prompts: ； in is the prompt category, which includes buildings, coastlines or roads; is a template for structured text prompts, represented as Through template constraints, the multimodal edge detection model DirectSAM-RS learns to align text semantics with visual edge features and suppress irrelevant regional responses; S15: Lightweight Contour Decoder: A multi-layer perceptron MLP decoder is used to integrate multi-stage fusion features Upsample and fuse to output edge probability map , ensuring the capture of coastline details and understanding of the global outline in remote sensing images: ； in Indicates Layer fusion features Perform upsampling operations to unify the features of different layers to the same spatial resolution to facilitate splicing operations; It means that the upsampled multi-level features are combined along the channel dimension as ; It is the concatenation of multi-level multi-modal features, integrating feature information of all levels to form a global feature representation containing multi-scale multi-modal information, providing input for subsequent edge decoding; is the Sigmoid activation function, and the MLP contains two fully connected layers and a ReLU activation layer.

3. The method for adaptive coastline edge detection based on a multimodal large model according to claim 1, characterized in that: The specific process of constructing the large-scale pre-training dataset RemoteContour-34k in step S2 is as follows: S21: Collecting images from existing remote sensing semantic segmentation datasets and its segmentation mask , the segmentation mask is transformed into The process of converting to single pixel edge labels is as follows: For each segmentation mask Extract all target boundary point sets ; Initialize edge labels is an all-zero matrix, and the boundary point set exist Set the corresponding element in to 1 to get the edge label ; Optimize edge continuity and remove isolated noise points. Indicates the height of the image. Indicates the width of the image; S22: For each image Generate text prompt , according to the original segmentation mask Category labels to generate structured text ,in For the prompt category; S23: Integrating images ,text , edge labels , stored as Triplet, as the basic unit constituting the large-scale pre-training dataset RemoteContour-34k.

4. The method for adaptive coastline edge detection based on a multimodal large model according to claim 1, characterized in that: In step S3, the specific process of using the RemoteContour-34k data set to perform multimodal pre-training on the multimodal edge detection large model DirectSAM-RS is as follows: S31: Input data: Remote sensing images Input multimodal edge detection model DirectSAM-RS visual encoder, text prompt Input text encoder BERT; where Indicates the height of the image. Represents the width of the image, and 3 represents the number of channels; S32: Decode and calculate loss: The edge probability map output by the multimodal edge detection large model DirectSAM-RS is , and the true edge label Calculate the loss pixel by pixel: ； The optimization goal is to minimize the edge detection loss , the parameters of the multimodal edge detection model DirectSAM-RS Optimize to the optimal model parameters : ； in, Indicates that the function The variable that reaches the minimum value A collection of .

5. The method for adaptive coastline edge detection based on a multimodal large model according to claim 1, characterized in that: In step S4, a coastline edge detection dataset SLSD is used as fine-tuning data for the multimodal edge detection large model DirectSAM-RS. The dataset SLSD includes high-resolution remote sensing images and their corresponding coastline edge labels.

Images