A method and device for extracting a pit pond water surface based on a SAM remote sensing image

Abstract

The application discloses a kind of SAM-based remote sensing image pit pond water surface extraction method and device, wherein the method comprises: obtaining suitable SAM model version and configuring operating environment, then based on the YOLOv10 model trained, pit pond target detection is carried out to remote sensing image pit pond, and the target detection frame generated;And the target detection frame generated is as boundary box prompt (box prompt) with remote sensing image is input into SAM model, obtains remote sensing image pit pond segmentation result.The method of the application can not only improve the precision and efficiency of pit pond extraction, but also adapt to pit pond extraction requirements of different scales and complexities, provide strong support for water resource management, environmental monitoring and disaster warning and other fields;The pit pond detection obtained has high accuracy, the whole device is robust and fast, lightweight and easy to deploy, and an innovative application is proposed for pit pond water surface extraction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of extracting pit pond water surface in remote sensing images, and in particular to a SAM-based remote sensing image pit pond water surface extraction method and device. BACKGROUND

[0002] The spatiotemporal distribution characteristics of water bodies are of great significance for water resource monitoring and application, and the classification and change analysis of land cover types including water body distribution are the most common applications of remote sensing technology. Pit ponds, as a part of water bodies, refer to artificially excavated or naturally formed water surfaces with a storage capacity of less than 100,000 cubic meters, and are an important ecological infrastructure of cities and one of the foundations for urban residents to continuously obtain natural services. Obtaining the spatiotemporal distribution information of pit pond water surface and monitoring the dynamic characteristics of pit ponds are of great significance for water resource protection, ecological environment protection, etc.

[0003] Remote sensing images can comprehensively reflect the physical and chemical characteristics of the earth's surface by recording the reflection and radiation information of ground objects in different wave bands. Segmentation and extraction of remote sensing images is a research hotspot in the academic and industrial communities, and has a very wide application in urban planning, change detection, and geographic information system construction. Using remote sensing data to extract water body parameters such as location, area, and shape has become an effective method and means for quickly obtaining water body parameters.

[0004] In recent years, SAM (Segment Anything Mode) is a new type of image segmentation model that has demonstrated outstanding performance in multiple fields due to its strong generalization ability and flexible prompting mechanism. However, SAM is a class-agnostic method that relies heavily on manually provided prior prompts such as points, boxes, and rough masks. Due to the complexity of the background of remote sensing images, the diversity of scenes, and the lack of clearly defined object edges, this limits the automation and applicability of SAM in remote sensing image segmentation tasks. Therefore, there are still some problems in the task of extracting water bodies from remote sensing images using SAM according to current research.

[0005] Firstly, the SAM model is for segmentation of everything and does not contain class information. Secondly, a specific prompt needs to be added to the SAM model for segmentation of specific models. How to provide an accurate prompt. Finally, how to improve the robustness of the model for segmentation of water bodies in different regions.

[0006] The patent document with the application number 202410463849.3 discloses a method and device for extracting pit pond water surface in remote sensing image. The method for extracting pit pond water surface in remote sensing image comprises the following steps: dividing the remote sensing image into a plurality of image blocks with overlapping regions; inputting all the image blocks into a deep learning model for reasoning and processing to obtain a vector graph corresponding to each image block; setting an edge threshold; marking the edge water area on each vector graph according to the edge threshold, and removing the edge water area to obtain a to-be-fused graph corresponding to each vector graph; and fusing all the to-be-fused graphs to obtain a final graph to extract the pit pond.

[0007] Through the scheme, the accuracy of pit pond detection can be improved, but the scheme lacks effective use of the SAM model and the YOLO model, which is not conducive to accurately and quickly obtaining the pit pond detection result. SUMMARY

[0008] In view of the above problems, the purpose of the present application is to provide a method and device for extracting pit pond water surface in remote sensing image based on SAM, which uses the YOLOv10 model to provide accurate box prompt information for the SAM, and then uses the SAM model to segment the pit pond water surface according to the obtained box coordinates, combines the semantic segmentation and target detection computer vision tasks, and proposes a new method for extracting pit pond water surface in remote sensing image.

[0009] The present application provides a method and device for extracting pit pond water surface in remote sensing image based on SAM.

[0010] The first aspect is a method for extracting pit pond water surface in remote sensing image based on SAM, which comprises the following steps:

[0011] S1, obtaining a suitable SAM model version and configuring the running environment of the SAM model;

[0012] S2, performing pit pond target detection on the pit pond in the remote sensing image based on the trained YOLOv10 model to generate a target detection frame;

[0013] S3, inputting the generated target detection frame as a boundary box prompt box prompt and the remote sensing image into the SAM model to obtain a pit pond segmentation result of the remote sensing image.

[0014] Optionally, the remote sensing image and the SAR image are input into the YOLOv10 model in a multi-modal fusion manner.

[0015] Optionally, the SAM model comprises:

[0016] An image encoder Image encoder outputs image embedding coding information according to the input remote sensing image.

[0017] A prompt encoder receives a target detection box of the YOLOv10 model to obtain box prompt encoding information.

[0018] A mask decoder converts the image embedding encoding information and the box prompt encoding information into valid output.

[0019] Optionally, the image embedding encoding information is further convolved with a mask.

[0020] Optionally, the YOLOv10 model comprises:

[0021] A backbone extracts multi-scale features from an input remote sensing image.

[0022] A neck converges the multi-scale features to realize multi-scale feature fusion.

[0023] A head generates multiple predictions or a best prediction for each object based on the fused multi-scale features to obtain a target detection box.

[0024] Optionally, the training process of the YOLOv10 model comprises the steps of:

[0025] Prepare a remote sensing image dataset, perform dataset labeling and preprocessing, and divide it into a training set and a validation set.

[0026] Configure a loss function, an optimization algorithm, and a learning rate, and configure training resources.

[0027] Record the loss function and each round of evaluation indicators, and recursively call the training process until the loss function and the evaluation indicators meet the expectations, and complete the model training.

[0028] After the model training is completed, the validation set is used for verification, and the weight file is saved.

[0029] Second aspect: a SAM-based remote sensing image pit pond water surface extraction device, comprising:

[0030] A YOLOv10 model module performs pit pond target detection on a remote sensing image pit pond to generate a target detection box.

[0031] A SAM model module inputs the generated target detection box as a boundary box prompt box prompt and a remote sensing image into a SAM model to obtain a remote sensing image pit pond segmentation result.

[0032] Third aspect: an electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method provided in the first aspect when executing the program.

[0033] Fourth aspect: a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the steps of the method provided in the first aspect.

[0034] Advantages of the present application:

[0035] 1、The present application utilizes the advantages of the SAM model in the field of image segmentation and applies it to the extraction of pit pond water in high-resolution remote sensing images. The YOLOv10 model is used to analyze the texture and color features of the pit pond in high-resolution remote sensing images, which serves as the box prompt information for the SAM model. Combined with the powerful segmentation capability of the SAM model, the automatic segmentation of pit ponds and non-pit pond areas is realized. This method not only improves the accuracy and efficiency of pit pond extraction, but also adapts to different scales and complexities of pit pond extraction requirements, providing strong support for water resource management, environmental monitoring, and disaster warning fields. The obtained pit pond detection has high accuracy, the entire device has good robustness and fast running speed, is lightweight and easy to deploy, and provides an innovative application for pit pond water extraction.

[0036] 2、The present application combines the semantic segmentation of the SAM model and the target detection of the YOLOv10 model, and provides the SAM with box prompt information. The powerful segmentation capability of the SAM is fully utilized to extract the pit pond water surface in the remote sensing image, so that the segmentation result of the SAM has semantic nature. The method has good robustness and good segmentation effect for pit pond water surface in different environments. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 The flowchart of the SAM-based remote sensing image pit pond water extraction method of the present application;

[0038] Figure 2 The structure diagram of the SAM model of the present application;

[0039] Figure 3 The structure diagram of the YOLOv10 model of the present application;

[0040] Figure 4 The structure diagram of the SAM remote sensing image pit pond water extraction device of the present application;

[0041] Figure 5 The training flowchart of the YOLOv10 model of the present application;

[0042] Figure 6 A principle flow chart of the SAM remote sensing image pit pond water surface extraction device of the present application;

[0043] Figure 7 A structural principle diagram of the SAM remote sensing image pit pond water surface extraction device of the present application;

[0044] Figure 8 A structural schematic diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0045] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar symbols represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0046] Currently, there are still some problems in extracting pit pond water in remote sensing images based on SAM: for example, the SAM model is to segment everything, and does not contain class information itself, and needs to add a specific prompt to the SAM model segmentation specific model, but how to accurately obtain the specific prompt of the SAM model is a current difficult problem, and the specific prompt is not easy to obtain, which affects the task accuracy of SAM in extracting pit pond water in remote sensing images.

[0047] In view of the above problems, the present application provides a SAM-based remote sensing image pit pond water surface extraction method, Figure 1 A flowchart of the SAM-based remote sensing image pit pond water surface extraction method provided by the embodiment of the present application, the method comprising:

[0048] S1, obtain a suitable SAM model version, and configure the running environment of the SAM model.

[0049] The SAM model structure is shown in Figure 2 The model is roughly composed of an image encoder (image encoder) composed of a standard vit, a prompt encoder (prompt encoder) and a mask decoder (Mask decoder) three modules.

[0050] Among them, the image encoder (Image encoder) outputs image embedding (Image embedding) coding information according to the input remote sensing image;

[0051] Prompt encoder, used for receiving the encoding information of point, box, and txt, for example, receiving the target detection box of YOLOv10 model to obtain the encoding information of box prompt; and combining the image embedding to send into the mask decoder.

[0052] Mask decoder, used for converting the image embedding and the encoding information of box prompt into valid mask.

[0053] Further, as shown in the image embedding, the encoding information is also convoluted with the mask, the mask is an input of the encoder, and the convolution embedding dense prompt is beneficial to the extraction of pit details. Figure 2

[0054] S2, based on the trained YOLOv10 model, the pit target in the remote sensing image is detected, and the generated target detection box is generated.

[0055] The structure diagram of YOLOv10 is shown in the figure. Figure 3 As shown in the figure, the model is composed of backbone network, neck network, and head network module, wherein:

[0056] Backbone network: mainly responsible for feature extraction, which can use enhanced CSPNet (cross-stage local network) to improve gradient flow and reduce computational redundancy, and extract multi-scale features from the input remote sensing image;

[0057] Neck network: converges multi-scale features and transmits them to head network, mainly including PAN layer (path aggregation network), which can realize effective multi-scale feature fusion.

[0058] Head network: including one-to-many head module and one-to-one head module.

[0059] Among them, one-to-many head module can generate multiple predictions for each object to provide rich supervision signals and improve learning accuracy.

[0060] ​One-to-one head, generates one best prediction for each object without NMS (non-maximum suppression), thereby reducing latency and improving efficiency.

[0061] Further, when the pit pond target is detected by using the YOLOv10, a remote sensing image + SAR image multi-modal fusion mode can be adopted to improve the precision of the model, and after the precision is improved, the result of target detection prediction is taken as a boundary box prompt (box prompt) and is input into the SAM model to realize accurate segmentation of the pit pond water surface.

[0062] Further, as shown in the training of the YOLOv10 model, Figure 5 includes the following steps:

[0063] Before the model training starts, the remote sensing image dataset is prepared, the data is labeled and preprocessed, and the training set and the validation set are reasonably divided.

[0064] A suitable loss function is selected, an optimization algorithm and a learning rate are configured, and training resources such as cpu training, single-gpu training and multi-card multi-machine training are configured.

[0065] During the training process, the loss function and the evaluation index are recorded, if overfitting, gradient disappearance and other problems occur, then the data preprocessing is restarted to find the reason and modify the model weight, the training process is called repeatedly until the loss function and the evaluation index reach the expectation, and the model training is completed.

[0066] After the model training is completed, the validation set is used for verification, and the weight file is saved to provide the boundary box prompt (box prompt) information for the SAM model when the pit pond water surface is extracted.

[0067] S3, the generated target detection frame is taken as a boundary box prompt (box prompt) and is input into the SAM model together with the remote sensing image to obtain the pit pond segmentation result of the remote sensing image.

[0068] As shown in Figure 4 and Figure 6 The device of the present application is composed of a SAM model and a YOLOv10 model, the YOLOv10 model first provides accurate boundary box prompt (box prompt) information for the SAM model, and then the SAM model segments the pit pond water surface according to the obtained boundary box coordinates; effectively solve the problem that the SAM model only has segmentation without semantics, provide class information, and how to obtain good prompt (prompt) information;

[0069] The application combines semantic segmentation and target detection two computer vision tasks, and proposes a new type of pit pond water surface extraction method of remote sensing image; YOLOv10 provides box prompt information for SAM, fully utilizes the powerful segmentation capability of SAM, extracts the pit pond water surface in the remote sensing image; at the same time, the target detection provides the category information for the segmentation of SAM, so that the segmentation result of SAM has semantic nature; the method has good robustness, and has good segmentation effect on the pit pond water surface in different environments.

[0070] The application also discloses a SAM-based remote sensing image pit pond water surface extraction device, as shown in the figure, the device comprises a YOLOv10 model module and a SAM model module, etc. Figure 7

[0071] The input remote sensing image is input to the YOLOv10 model module, the YOLOv10 model is used for pit pond target detection of the remote sensing image pit pond, and the generated target detection frame is generated.

[0072] Then the generated target detection frame is input as a boundary box prompt (box prompt) together with the remote sensing image into the SAM model module, and the SAM model is used to obtain the pit pond segmentation result of the remote sensing image.

[0073] By using the device structure, the module combination of SAM+YOLOv10 is adopted, the remote sensing image pit pond water surface extraction is realized, the obtained pit pond detection has high accuracy, the whole device has good robustness and fast running speed, is light in weight and easy to deploy, and is an innovative application in the field of pit pond water surface extraction.

[0074] The application also provides an electronic device, Figure 8 The structure diagram of the electronic device provided by the embodiment of the application is shown in the figure, the electronic device can comprise a processor, a communications interface, a memory and a communications bus, wherein the processor, the communications interface and the memory complete mutual communication through the communications bus. Figure 8

[0075] S1, obtaining a suitable SAM model version, configuring the running environment of the SAM model;

[0076] S2, based on the trained YOLOv10 model, performing pit pond target detection on the remote sensing image pit pond, and generating a target detection frame;

[0077] ​​S3, input the generated target detection frame as a boundary frame prompt box prompt together with the remote sensing image into the SAM model to obtain a remote sensing image pit pond segmentation result.

[0078] In addition, the logic instructions in the above-mentioned memory can be realized in the form of a software function unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0079] The embodiments of the present application also provide a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the method provided by the above-mentioned embodiments, for example, comprising: S1, obtaining a suitable SAM model version, and configuring a running environment of the SAM model;

[0080] S2, performing pit pond target detection on the remote sensing image pit pond based on the trained YOLOv10 model to generate a target detection frame;

[0081] S3, input the generated target detection frame as a boundary frame prompt box prompt together with the remote sensing image into the SAM model to obtain a remote sensing image pit pond segmentation result.

[0082] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment scheme. Those skilled in the art can understand and implement it without creative labor.

[0083] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0084] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A SAM-based method for extracting water surface of pits and ponds from remote sensing images, characterized in that, The method comprises the following steps: S1, obtaining a SAM model version, and configuring a running environment of the SAM model; S2, performing pit pond target detection on a remote sensing image pit pond based on a trained YOLOv10 model to generate a target detection frame; S3, inputting the generated target detection frame as a boundary box prompt box prompt and the remote sensing image into the SAM model to obtain a remote sensing image pit pond segmentation result; The SAM model comprises: an image encoder Image encoder configured to output image embedding Image embedding encoding information based on input remote sensing images; a prompt encoder Prompt encoder configured to receive the target detection frame of the YOLOv10 model to obtain boundary box prompt box prompt encoding information; a mask decoder Mask decoder configured to convert the image embedding Image embedding encoding information and the boundary box prompt box prompt encoding information into valid output Valid Mask; The YOLOv10 model comprises: a backbone network Backbone configured to extract multi-scale features from input remote sensing images; a neck network Neck configured to converge the multi-scale features to realize multi-scale feature fusion; a head network Head configured to generate multiple predictions or a best prediction for each object based on the fused multi-scale features to obtain a target detection frame; The training process of the YOLOv10 model comprises the following steps: preparing a remote sensing image dataset, performing dataset labeling and preprocessing, and dividing the dataset into a training set and a validation set; configuring a loss function, an optimization algorithm, and a learning rate, and configuring training resources; recording the loss function and evaluation indicators for each round, and calling the training process in a loop until the loss function and evaluation indicators meet the expectations, and the model training is completed; after the model training is completed, the validation set is used for verification, and the weight file is saved.

2. The SAM-based method for extracting water bodies from remote sensing images according to claim 1, wherein, The remote sensing image and the SAR image are multi-modal fusion input into the YOLOv10 model.

3. The SAM-based method for extracting water bodies from remote sensing images according to claim 1, wherein, The image embedding Image embedding encoding information is further convolved with a mask Mask.

4. A device for extracting water surface of a pit pond from a remote sensing image based on SAM using the method according to any one of claims 1 to 3, characterized in that, The device comprises: a YOLOv10 model module configured to perform pit pond target detection on a remote sensing image pit pond to generate a target detection frame; a SAM model module configured to input the generated target detection frame as a boundary box prompt box prompt and the remote sensing image into the SAM model to obtain a remote sensing image pit pond segmentation result.

5. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps of the SAM-based remote sensing image pit pond water surface extraction method according to any one of claims 1 to 3.

6. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the SAM-based remote sensing image pit pond water surface extraction method according to any one of claims 1 to 3.

Images