Farmland road automatic segmentation method, system, equipment and medium

Abstract

The invention provides a farmland road automatic segmentation method, system and device and a medium, and belongs to the technical field of image segmentation, the scheme uses an open source data set to carry out model training and prompt project input, reduces the demand for a special marking sample which is high in cost and difficult to obtain, and especially in a changeful farmland environment, greatly improves the segmentation efficiency of the farmland road. The dependence on the marked data is effectively reduced; meanwhile, by introducing an advanced road segmentation model based on an SAM network architecture and combining with OSM road vector data based on an open source, a prompt project is adaptively generated, so that SAM is activated to extract the farmland road, and the farmland road extraction precision is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of road segmentation, and in particular relates to a method, system, equipment and medium for automatic segmentation of farmland roads. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] At present, the research on road extraction methods is relatively mature, especially the more popular road extraction technology based on deep learning, which has achieved high accuracy and has been widely used in high-resolution remote sensing images, and a large number of standard road segmentation datasets have been publicly released.

[0004] However, current research and datasets mainly focus on urban scenes, and there is a relative technical gap in road extraction for farmland scenes. In addition, most existing road extraction technologies rely on a large number of high-quality and diverse labeled samples. In particular, remote sensing images in farmland scenes are easily affected by seasons, which makes it difficult to migrate models to different farmland scenes and remote sensing images of different resolutions under limited data conditions. Summary of the invention

[0005] The solution described in this embodiment provides a method, system, device and medium for automatic segmentation of farmland roads to solve the problems of poor repeatability and migration application capability of the automatic road segmentation method in the prior art for road extraction tasks in farmland scenarios, because the seasonal changes and scene changes in the farmland environment require continuous updating and expansion of the data set, and the problem of insufficient road extraction accuracy due to the similarity between the spectral characteristics of the farmland roads and the surrounding environment.

[0006] According to a first aspect of an embodiment of the present invention, a method for automatically segmenting a farmland road is provided, comprising: Obtain the remote sensing image of the area to be extracted and the road centerline vector data of the current area; Based on the road centerline vector data of the current area, generating point prompts, box prompts and mask prompts respectively; Based on the remote sensing image, the point prompt, the box prompt and the mask prompt are respectively combined, and the pre-trained deep learning-based road segmentation model is used to obtain the first road extraction result, the second road extraction result and the third road extraction result; wherein the deep learning-based road segmentation model adopts the pre-trained deep learning model, and the pre-trained deep learning model is fine-tuned through a preset farmland road data set; Based on the first road extraction result, the second road extraction result and the third road extraction result, the final farmland road extraction result is obtained by fusion of the results.

[0007] Furthermore, the deep learning-based road segmentation model includes an image encoder, a hint encoder and a mask decoder; wherein the image encoder is used to map the input remote sensing image to a high-dimensional image embedding space, the hint encoder is used to convert points and bounding boxes into feature representations of embedded tags through position encoding, and downsample the mask input into a higher-level feature representation through a convolutional neural network, and the mask decoder is used to fuse the feature representations between image features and hint embeddings based on a cross-attention mechanism to obtain a mask output.

[0008] Furthermore, the pre-trained deep learning model is fine-tuned by using a preset farmland road dataset, specifically, only the mask decoder is fine-tuned, and a loss function based on a combination of a binary cross entropy loss function and a Dice system is used during the fine-tuning process.

[0009] Furthermore, the generation of the point prompt, the frame prompt and the mask prompt is specifically as follows: By converting the road centerline vector data into raster labels and adopting a regular grid sampling method, the set of pixel points labeled as roads and falling on the grid is used as point prompts; By obtaining the bounding box of the road centerline vector data, a box hint is obtained; The road centerline vector data is converted into raster labels and the expansion algorithm is used to generate binary labels that meet the preset width as mask prompts.

[0010] Furthermore, the result fusion is specifically as follows: the three results based on point prompts, based on box prompts and based on mask prompts are fused through a logical OR operation; the fused results are smoothed through morphological operations to smooth discrete points and missing roads to obtain the final road segmentation result.

[0011] Optionally, the road centerline vector data is obtained from an OSM road vector dataset.

[0012] Optionally, the deep learning-based road segmentation model adopts a SAM network architecture, specifically a ViT-base network model.

[0013] According to a second aspect of an embodiment of the present invention, there is provided a system for automatically segmenting farmland roads, comprising: A data acquisition unit, which is used to acquire remote sensing images of the area to be road extracted and road centerline vector data of the current area; A prompt automatic acquisition unit, which is used to generate a point prompt, a box prompt and a mask prompt respectively based on the road centerline vector data of the current area; A road extraction unit, which is used to obtain a first road extraction result, a second road extraction result, and a third road extraction result based on the remote sensing image, respectively in combination with point prompts, box prompts, and mask prompts, using a pre-trained deep learning-based road segmentation model; wherein the deep learning-based road segmentation model adopts a pre-trained deep learning model, and fine-tunes the pre-trained deep learning model through a preset farmland road data set; A fusion unit is used to obtain a final farmland road extraction result by fusing the first road extraction result, the second road extraction result and the third road extraction result.

[0014] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, comprising a memory, a processor and a computer program stored and running on the memory, wherein when the processor executes the program, the method for automatically segmenting farmland roads is implemented.

[0015] According to a fourth aspect of an embodiment of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method for automatically segmenting farmland roads is implemented.

[0016] One or more of the above technical solutions have the following beneficial effects: The present invention provides a method and system for automatic segmentation of farmland roads. The scheme reduces the need for special labeled samples that are expensive and difficult to obtain by using open source data sets for model training and prompt engineering input, especially in a changing farmland environment, effectively reducing the dependence on labeled data; at the same time, by introducing an advanced road segmentation model based on the SAM network architecture, combined with the adaptive generation of prompt engineering based on open source OSM road vector data, to activate SAM to extract farmland roads, effectively improving the accuracy of farmland road extraction. The scheme of the present invention adopts a pre-trained ViT-Base model and fine-tunes it in combination with a small amount of farmland road data, so that the method described in this embodiment is not only applicable to high-resolution remote sensing images of different resolutions, but also can adapt to remote sensing images of various geographical regions and different seasonal changes, and has a wide range of application prospects; at the same time, selectively using a miniaturized ViT-Base model for initialization and using a small amount of samples for fine-tuning can significantly reduce the demand for computing resources without sacrificing accuracy, making the technology more suitable for use in resource-constrained environments. In the result fusion of the present invention, the noise of the point prompt-based extraction results is reduced by adjusting the confidence of the three results, while the confidence of the discrete points based on the box prompt and the mask prompt is improved to extract more targets. On the basis of this fusion post-processing, a morphological algorithm is used to smooth discrete points and missing roads, so that even if there is very little known OSM road data, the farmland road extraction method described in this embodiment does not require additional manual marking, and farmland roads can be automatically extracted. The advantages of additional aspects of the present invention will be partially given in the following description, and part will become obvious from the following description, or will be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0018] Figure 1 A road schematic diagram of farmland scene 1 from the DeepGlobe road dataset described in an embodiment of the present invention; Figure 2 This is a road schematic diagram of farmland scene 2 from the DeepGlobe road dataset described in an embodiment of the present invention; Figure 3 A road schematic diagram of a city scene from the DeepGlobe road dataset described in an embodiment of the present invention; Figure 4 A road schematic diagram of a coastal city scene from the DeepGlobe road dataset described in an embodiment of the present invention; Figure 5The high-resolution remote sensing image with a resolution of 0.1 m described in the embodiment of the present invention has a typical farmland scene; Figure 6 In the embodiment of the present invention Figure 5 SAM automatic segmentation results of remote sensing images; Figure 7 The objects not extracted in the SAM automatic segmentation result in the embodiment of the present invention; Figure 8 Schematic diagram of the structure and fine-tuning process of the road segmentation model described in an embodiment of the present invention; Fig. 9 This is a schematic diagram of the original SAM automatic segmentation result described in an embodiment of the present invention; Fig.10 It is a schematic diagram of the SAM automatic segmentation result after fine-tuning based on part of the DeepGlobe road dataset described in an embodiment of the present invention; Fig.11 This is a schematic diagram of the result of generating a frame prompt based on OSM in an embodiment of the present invention; Fig.12 Schematic diagram of the segmentation result based on the original SAM and the frame prompt described in the embodiment of the present invention; Fig.13 Schematic diagram of the segmentation result based on the fine-tuned SAM and box prompts described in an embodiment of the present invention; Fig.14 This is a schematic diagram of a point prompt visualization result generated based on OSM road vector data in an embodiment of the present invention; Fig.15 This is a schematic diagram of a visualization result of a box prompt generated based on OSM road vector data in an embodiment of the present invention; Fig.16 This is a schematic diagram of a mask prompt visualization result generated based on OSM road vector data in an embodiment of the present invention; Fig.17 This is a schematic diagram of the OSM road visualization result described in an embodiment of the present invention; Fig.18 Schematic diagram of the segmentation result based on point prompts described in an embodiment of the present invention; Fig.19 Schematic diagram of the segmentation result based on frame prompts described in an embodiment of the present invention; Fig. 20 Schematic diagram of the segmentation result based on mask prompts described in an embodiment of the present invention; Fig.21 This is a schematic diagram of the segmentation result after fusion processing described in an embodiment of the present invention; Fig. 22 This is a flow chart of a method for automatic segmentation of farmland roads described in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0020] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0021] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0022] like Fig. 22 As shown, an embodiment of the present invention provides an automatic farmland road segmentation method. The method uses the zero-sample learning ability of the SAM visual large model to introduce the visual large model into the farmland road extraction task. The method not only overcomes the limitations of SAM in processing slender and sparsely distributed farmland roads, but also does not require any additional manual marking or manual interactive input prompts. At the same time, the method is applicable to remote sensing images of different spatial resolutions, and can adapt to the differences in farmland scenes caused by different geographical regions and seasonal changes, thereby providing more accurate and robust road extraction performance; Specifically, the method specifically includes the following processing procedures: Step 1: Obtain the remote sensing image of the area to be extracted and the road centerline vector data of the current area; The acquisition of the data specifically includes: First, remote sensing images are collected and preprocessed according to the task requirements. The image preprocessing process usually includes multiple links such as radiation calibration, atmospheric correction, orthorectification, image registration, image fusion, cropping and splicing. Since the data publishing platform preprocesses the original remote sensing data to a certain extent and publishes different levels of data, when downloading or receiving image data, users need to take corresponding preprocessing steps according to the actual data situation, and consider the actual area scope, and need to crop or splice the image.

[0023] Secondly, download the latest road centerline vector data (hereinafter referred to as OSM road vector data) from the OSM website (https: / / www.openstreetmap.org) and further preprocess the road vector data. The preprocessing process includes coordinate system conversion and regional clipping to make the geographic coordinate system and regional range of the road vector data consistent with the geographic coordinate system and boundary range of the remote sensing image. All data preprocessing processes can use ENVI or ArcGIS professional software, or use the arcpy library of the Python computer language for batch processing.

[0024] Subsequently, the pre-processed remote sensing images (tif format) and road vector data (shp format) need to be cropped in batches. Since the later model fine-tuning uses the open source dataset DeepGlobe Road ExtractionDataset (hereinafter referred to as DeepGlobe road dataset), the size of this dataset is In order to avoid data loss in practical applications, the preprocessed remote sensing images are cropped into Image blocks of the same size do not overlap each other, and the part that is less than the size is filled with 0. This process can be processed in batches using the GDAL library of the Python computer language, and the geospatial information of each image block (tif format) is retained. The corresponding image boundary vector data is generated using the cropped image blocks, and then the road vector data (shp format) corresponding to the boundary of each image block is cropped from the preprocessed road vector data.

[0025] Step 2: Based on the road centerline vector data of the current area, point prompts, box prompts and mask prompts are generated respectively; Step 3: Based on the remote sensing image, the point prompt, the box prompt and the mask prompt are respectively combined, and the pre-trained deep learning-based road segmentation model is used to obtain the first road extraction result, the second road extraction result and the third road extraction result; wherein the deep learning-based road segmentation model adopts a pre-trained deep learning model, and the pre-trained deep learning model is fine-tuned through a preset farmland road data set; In a specific implementation, the road segmentation model based on deep learning adopts a SAM network architecture. Specifically, the road segmentation model based on deep learning includes an image encoder, a prompt encoder and a mask decoder; wherein the image encoder is used to map the input remote sensing image to a high-dimensional image embedding space, the prompt encoder is used to convert points and bounding boxes into feature representations of embedded tags through position encoding, and the mask input is downsampled and converted into a higher-level feature representation through a convolutional neural network. The mask decoder is used to fuse the feature representations between image features and prompt embeddings based on a cross-attention mechanism to obtain a mask output.

[0026] The SAM visual model has been trained on a large-scale dataset of 1 billion pixels, and is fine-tuned using only a small amount of publicly available road datasets in farmland scenarios. The dataset used for fine-tuning is a road dataset with farmland scenarios selected from the DeepGlobe road dataset.

[0027] Step 4: Based on the first road extraction result, the second road extraction result and the third road extraction result, the final farmland road extraction result is obtained by fusing the results.

[0028] In this way, the above scheme of this embodiment reduces the need for expensive and difficult-to-obtain specially labeled samples by utilizing open source data sets for model training and prompt engineering input, especially in the ever-changing farmland environment, effectively reducing the dependence on labeled data; at the same time, by introducing an advanced road segmentation model based on the SAM network architecture, combined with the open source OSM road vector data-based adaptive generation prompt engineering to activate SAM to extract farmland roads, the accuracy of farmland road extraction is effectively improved.

[0029] In one or more embodiments, the road segmentation model based on deep learning adopts a pre-trained deep learning model, and fine-tuning is performed based on the pre-trained deep learning model through a preset farmland road dataset; specifically, the following steps are included: The selection of the pre-trained model has been described in the previous embodiment. The road segmentation model adopts the network architecture in the SAM model, including an image encoder, a prompt encoder, and a mask decoder. The image encoder uses a pre-trained mask autoencoder based on Vision Transformer (ViT) to map the input image to a high-dimensional image embedding space. The prompt encoder converts points and bounding boxes into feature representations of embedded tags through position encoding, and downsamples the mask input into higher-level feature representations through a convolutional neural network. Finally, the mask decoder fuses the interaction between image features and prompt embeddings through a cross-attention mechanism, and finally generates a mask output. The SAM model launched by META was trained on the large-scale SA-1B dataset, and three different models (ViT-base, ViT-large, and ViT-huge) were released. Through experimental evaluation, larger ViT models, such as ViT-large and ViT-huge, have slightly improved accuracy, but the computational requirements have increased significantly. Therefore, in order to balance segmentation performance and computational efficiency, this embodiment selects the smallest ViT-Base as a pre-trained model to initialize the SAM model parameters.

[0030] Model parameter fine-tuning The SAM model has demonstrated zero-shot learning capabilities in various downstream tasks, allowing direct inference of unfamiliar images with minimal input cues. However, the model performance in the road extraction task of remote sensing images is poor, as shown in the following Figure 2 As shown in the figure, the model can directly infer objects such as fields and buildings that are distributed in blocks, but cannot extract objects such as roads and rivers that are continuously distributed and elongated. Therefore, in order to improve the ability of the SAM model to identify objects that are continuously distributed and elongated in remote sensing images, a road segmentation dataset of a certain scale that meets the requirements of road extraction is used to fine-tune the model parameters. The open source DeepGlobe road dataset (such as Figures 1 to 4The figure shows the roads in different scenes in the dataset, which are farmland scene 1, farmland scene 2, urban scene and coastal city scene. The dataset contains high-resolution satellite images with a resolution of 0.5m and pixel-based road labels (image size is ), covering road conditions in various scenarios such as cities, farmlands, and mountainous areas. In order to meet the road extraction capabilities in farmland scenarios, more than 130 images with farmland scenes and corresponding label datasets were selected from the DeepGlobe road dataset as the training set for the fine-tuning model.

[0031] Among them, Figures 5 to 7 As shown in the figure, the SAM automatic segmentation results of the remote sensing image with a resolution of 0.1m are shown, where Figure 5 It is a high-resolution remote sensing image with a resolution of 0.1m and a typical farmland scene. Figure 6 This is the SAM automatic segmentation result (different colors represent a mask extraction result). Figure 7 These are the objects that are not extracted in the SAM automatic segmentation results (the extracted objects are hidden in black).

[0032] During the training process, the scheme described in this embodiment freezes the image encoder and prompt encoder of the SAM model, and only fine-tunes the mask decoder (see the attached Figure 8 The segmentation prediction result output by the mask decoder With ground truth labels For comparison, a method combining the binary cross entropy loss function ( , Formula 1) and Dice coefficient ( , Formula 2) is the loss function ( , Formula 3) and use the AdamW optimizer to update the parameters in the model to gradually reduce the loss function. Finally, when the loss function converges, the model parameters are saved as the fine-tuned model. At this time, the fine-tuned model has a certain ability to recognize continuously distributed and slender road objects, as shown in the attached figure. Figures 9 to 13 As shown in the figure, the model before fine-tuning and the model after fine-tuning based on part of the DeepGlobe road dataset are compared in the automatic segmentation task and the task of extracting farmland roads based on the OSM road vector data. The segmentation results are marked in red on the remote sensing image, and the box prompt project generated based on the OSM road vector data is visualized with yellow marks. Specifically, Fig. 9 This is a schematic diagram of the original SAM automatic segmentation result described in an embodiment of the present invention; Fig.10 It is a schematic diagram of the SAM automatic segmentation result after fine-tuning based on part of the DeepGlobe road dataset described in an embodiment of the present invention; Fig.11This is a schematic diagram of the result of generating a frame prompt based on OSM in an embodiment of the present invention; Fig.12 Schematic diagram of the segmentation result based on the original SAM and the frame prompt described in the embodiment of the present invention; Fig.13 Schematic diagram of the segmentation result based on the fine-tuned SAM and box prompts described in an embodiment of the present invention;

[0033] in, and Represents image pixels The true label and model prediction label, is the total number of pixels in the input image.

[0034] In this way, by adopting the pre-trained ViT-Base model and fine-tuning it with a small amount of farmland road data, the method described in this embodiment is not only applicable to high-resolution remote sensing images of different resolutions, but also can adapt to remote sensing images of various geographical regions and different seasonal changes, and has broad application prospects; at the same time, the selective use of a miniaturized ViT-Base model for initialization and a small amount of samples for fine-tuning can significantly reduce the demand for computing resources without sacrificing accuracy, making this technology more suitable for use in resource-constrained environments.

[0035] In one or more implementations, based on the road centerline vector data of the current area, point prompts, box prompts, and mask prompts are generated respectively, specifically: By converting the road centerline vector data into raster labels and adopting a regular grid sampling method, the set of pixel points labeled as roads and falling on the grid is used as point prompts; By obtaining the bounding box of the road centerline vector data, a box hint is obtained; The road centerline vector data is converted into raster labels and the expansion algorithm is used to generate binary labels that meet the preset width as mask prompts.

[0036] In the specific implementation, the solution uses the open source OSM road vector data to automatically generate prompt projects, without the need for additional manual interactive input of prompt projects, and makes full use of rich historical data to achieve actual batch road extraction tasks. In this regard, this embodiment automatically generates three types of prompt projects based on OSM road vector data: point, box, and mask, where: For the point prompt project, the OSM road vector data is first converted into raster labels (0 / 1 binary labels, and the label image size is ), and then a regular grid sampling method is used to take the set of pixel points labeled as roads and falling on the grid as the point prompt project, that is, ,in, .

[0037] For the box prompt project, first convert the OSM road vector data .shp format into .geojson format to encode the geographic data structure of the road centerline. The converted GeoJSON data contains information such as the number, type, and geographic coordinates of the starting point, some intersections, and end points of each road. For this, you can directly obtain each linear road from the GeoJSON data and the corresponding starting point geographic coordinates ) and the geographic coordinates of the destination Then, through coordinate transformation, the geographic coordinates of longitude and latitude are transformed into image coordinates, that is, ), . Since roads are crisscrossed on the image and have long and narrow shapes and different lengths, if only the bounding boxes generated by the starting and ending points of the roads (named as initial bounding boxes) are used as the box prompt project, there will be a lot of redundant information. Even an initial bounding box generated by one road may cover the entire image range. At the same time, there may be overlaps between the initial bounding boxes generated by multiple roads, which is not conducive to the accurate generation of box prompt projects. Therefore, in order to improve the boundary accuracy of the box prompt, the initial bounding box is split into several small boxes according to the length-to-width ratio, and then the generation of the prompt box is optimized, which can be expressed as ,in and are the image coordinates of the upper left corner and the lower right corner respectively. , and n+1 represents the number of bounding boxes generated for a road.

[0038] For the mask hinting project, the OSM road vector data is first converted into raster labels (0 / 1 binary labels, and the label image size is ). Through the expansion algorithm, the rasterized linear binary labels are converted into binary labels with a certain width as mask hints.

[0039] Furthermore, based on the point prompts, box prompts and mask prompts obtained above, combined with the prompt projects with three input forms of points, boxes and masks in the SAM model, the prompt projects automatically generated based on OSM road vector data are distributed and input into the fine-tuned SAM model in the corresponding form and format. Finally, one remote sensing image can obtain three road extraction results.

[0040] In this way, through the above design, the scheme is based on the introduced SAM model, combined with the prompt engineering adaptively generated by using the open source OSM road vector data, which can effectively activate the SAM model to extract farmland roads and improve the extraction accuracy and efficiency of farmland roads; At the same time, in the process of generating frame prompts, considering that the roads are crisscrossed on the image and have slender shapes and different lengths, if only the bounding boxes generated by the starting and ending points of the roads (named as initial bounding boxes) are used as frame prompts, there will be a lot of redundant information, and even an initial bounding box generated by one road may cover the entire image range. At the same time, there may be overlaps between the initial bounding boxes generated by multiple roads, which is not conducive to the accurate generation of frame prompts. Therefore, in order to improve the boundary accuracy of the frame prompt, the initial bounding box is split into several small boxes according to the length and width ratio, thereby optimizing the generation of the prompt box.

[0041] In one or more embodiments, the first road extraction result, the second road extraction result and the third road extraction result are combined to obtain a final farmland road extraction result; wherein the result fusion specifically adopts the following processing process: Three types of farmland road extraction results can be obtained through point prompts, box prompts, and mask prompts. Due to the complexity of farmland scenes and the quality differences of OSM road vector data itself, the segmentation results under different prompting engineering forms have different degrees of noise and road disconnection. Therefore, in the process of post-processing results, the scheme described in this embodiment first adopts a result-level fusion strategy to merge the three results based on point prompts, box prompts, and mask prompts through logical or operations, so as to improve the confidence of the extraction results based on box prompts and mask prompts, so as to extract more targets. Based on the above, a series of morphological operations are performed to smooth the discrete points and missing roads, including: first, a corrosion operation is performed to reduce the discrete noise; the road extraction result can be approximately regarded as a rectangle. Since the roads are distributed in both horizontal and vertical directions, two opening operations will be performed, that is, the sizes of the rectangular convolution kernels in both horizontal and vertical directions are set respectively, such as and The convolution kernel is used to enhance the connectivity of adjacent road areas; then, it is further smoothed by a filter (such as a bilateral filter or a Gaussian filter) to reduce the sharp edges and noise that may be introduced by erosion and opening operations. Figures 14 to 21 As shown, even if there is very little known OSM road data, the farmland road extraction method described in this embodiment can automatically extract farmland roads without additional manual marking, and the extraction result is better than the OSM road data.

[0042] in, Figures 14 to 21The three forms of engineering schematic diagrams generated based on OSM road vector data, the corresponding farmland road extraction results and the final post-processed farmland road results are shown. Fig.14 , Fig.15 , Fig.16 The yellow mark in the middle is a visual reminder project. Fig.17 This is the visualization result after rasterizing the OSM road centerline vector data and expanding it to a certain width. Fig.18 , Fig.19 , Fig. 20 There are three forms of prompting the farmland road extraction results under the project. Fig.21 This is the result after fusion processing. Black represents the background and white marks represent the road.

[0043] Furthermore, based on the road extraction results of the acquired remote sensing images, in actual use, it is necessary to further stitch the extracted results, restore them to the size range of the original remote sensing images according to the cropping order during image preprocessing, batch process them through the GDAL library of the Python language, and retain the corresponding geographic spatial information, and finally obtain the road extraction results of the target farmland scene area.

[0044] In this way, through the above-mentioned result-level fusion strategy, the noise of the point-based prompt extraction results is reduced by adjusting the confidence of the three results, while the confidence of the discrete points based on the box prompt and the mask prompt is improved to extract more targets. On the basis of this fusion post-processing, the morphological algorithm is used to smooth the discrete points and missing roads, so that even if there is very little known OSM road data, the farmland road extraction method described in this embodiment does not require additional manual marking, and farmland roads can be automatically extracted.

[0045] It is understandable that in one or more embodiments, other road data can be used to replace OSM-based road vector data, and even rasterized road segmentation data can be used, but the subsequent processing details for the generation of box prompts will be different. If it is vector data, it can be converted into geojson format according to the method in the above embodiment, and then the key point information can be extracted, and then the segmented and more accurate box prompts can be extracted; if it is rasterized road segmentation data, it can be obtained by identifying the pixel category information and only one box, and then the road segmentation cannot be performed.

[0046] In another embodiment, based on the above-mentioned method for automatic segmentation of farmland roads, a corresponding system for automatic segmentation of farmland roads is provided, including: A data acquisition unit, which is used to acquire remote sensing images of the area to be road extracted and road centerline vector data of the current area; A prompt automatic acquisition unit, which is used to generate a point prompt, a box prompt and a mask prompt respectively based on the road centerline vector data of the current area; A road extraction unit, which is used to obtain a first road extraction result, a second road extraction result, and a third road extraction result based on the remote sensing image, respectively in combination with point prompts, box prompts, and mask prompts, using a pre-trained deep learning-based road segmentation model; wherein the deep learning-based road segmentation model adopts a pre-trained deep learning model, and fine-tunes the pre-trained deep learning model through a preset farmland road data set; A fusion unit is used to obtain a final farmland road extraction result by fusing the first road extraction result, the second road extraction result and the third road extraction result.

[0047] It can be understood that the method described in the above embodiment corresponds to the system described in this embodiment, and its technical details have been described in detail in the method embodiment, so they will not be repeated here.

[0048] In further embodiments, there is also provided: An electronic device includes a memory and a processor, and computer instructions stored in the memory and executed on the processor, wherein when the computer instructions are executed by the processor, the method described in Embodiment 1 is performed. For the sake of brevity, no further description is given here.

[0049] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0050] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0051] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the method described in embodiment 1 is completed.

[0052] The method in the first embodiment can be directly embodied as a hardware processor, or a combination of hardware and software modules in the processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware. To avoid repetition, it will not be described in detail here.

[0053] Those skilled in the art will appreciate that the units, i.e., algorithm steps, of the various examples described in the present embodiment can be implemented in electronic hardware or in a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0054] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for automatic segmentation of farmland roads, characterized in that: include: Obtain the remote sensing image of the area to be extracted and the road centerline vector data of the current area; Based on the road centerline vector data of the current area, generating point prompts, box prompts and mask prompts respectively; Based on the remote sensing image, the point prompt, the box prompt and the mask prompt are respectively combined, and the pre-trained deep learning-based road segmentation model is used to obtain the first road extraction result, the second road extraction result and the third road extraction result; wherein the deep learning-based road segmentation model adopts the pre-trained deep learning model, and the pre-trained deep learning model is fine-tuned through a preset farmland road data set; Based on the first road extraction result, the second road extraction result and the third road extraction result, the final farmland road extraction result is obtained by fusion of the results.

2. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The road segmentation model based on deep learning includes an image encoder, a hint encoder and a mask decoder; wherein the image encoder is used to map the input remote sensing image to a high-dimensional image embedding space, the hint encoder is used to convert points and bounding boxes into feature representations of embedded tags through position encoding, and the mask input is downsampled and converted into a higher-level feature representation through a convolutional neural network, and the mask decoder is used to fuse the feature representations between image features and hint embeddings based on a cross-attention mechanism to obtain a mask output.

3. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The pre-trained deep learning model is fine-tuned using a preset farmland road dataset, specifically: only the mask decoder is fine-tuned, and a loss function based on a combination of a binary cross entropy loss function and a Dice system is used during the fine-tuning process.

4. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The generation of the point prompt, frame prompt and mask prompt is specifically as follows: By converting the road centerline vector data into raster labels and adopting a regular grid sampling method, the set of pixel points labeled as roads and falling on the grid is used as point prompts; By obtaining the bounding box of the road centerline vector data, a box hint is obtained; The road centerline vector data is converted into raster labels and the expansion algorithm is used to generate binary labels that meet the preset width as mask prompts.

5. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The result fusion is specifically as follows: fusing the three results based on point prompts, based on box prompts and based on mask prompts through a logical OR operation; and performing morphological operations on the fused results to smooth discrete points and missing roads to obtain the final road segmentation result.

6. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The road centerline vector data is obtained from the OSM road vector dataset.

7. The method for automatic segmentation of farmland roads according to claim 1, characterized in that: The deep learning-based road segmentation model adopts a SAM network architecture, specifically a ViT-base network model.

8. A farmland road automatic segmentation system, characterized in that: include: A data acquisition unit, which is used to acquire remote sensing images of the area to be road extracted and road centerline vector data of the current area; A prompt automatic acquisition unit, which is used to generate a point prompt, a box prompt and a mask prompt respectively based on the road centerline vector data of the current area; A road extraction unit, which is used to obtain a first road extraction result, a second road extraction result, and a third road extraction result based on the remote sensing image, respectively in combination with point prompts, box prompts, and mask prompts, using a pre-trained deep learning-based road segmentation model; wherein the deep learning-based road segmentation model adopts a pre-trained deep learning model, and fine-tunes the pre-trained deep learning model through a preset farmland road data set; A fusion unit is used to obtain a final farmland road extraction result by fusing the first road extraction result, the second road extraction result and the third road extraction result.

9. An electronic device comprising a memory, a processor and a computer program stored and running on the memory, characterized in that: When the processor executes the program, an automatic farmland road segmentation method as described in any one of claims 1-7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, an automatic farmland road segmentation method as described in any one of claims 1 to 7 is implemented.

Images