A film-covered farmland remote sensing unsupervised segmentation method based on a SAM model and automatic prompt generation

Abstract

The application discloses a kind of based on SAM model and automatic prompt generation's film mulching farmland remote sensing unsupervised segmentation method, comprising the following steps: remote sensing image acquisition and pretreatment, optimal three-band combination construction and color gamut normalization, automatic prompt point generation based on brightness and gradient fusion, independent region unsupervised segmentation based on SAM model, post-processing and result output;The application realizes the full-automatic, zero manual intervention of film mulching farmland segmentation by the automatic prompt point generation strategy of brightness and gradient fusion, greatly reduces the labor cost, and improves the cross-region generalization ability of the method.Compared with the traditional pixel-level classification method, the film mulching plot generated by the application is free of holes, the boundary is clear and smooth, and there is no salt and pepper noise, and the region-independent segmentation mechanism avoids plot sticking, and spatial consistency is significantly improved.

Description

Technical Field

[0001] This invention relates to the fields of remote sensing image processing and agricultural information technology, specifically to an unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts. Background Technology

[0002] While agricultural film mulching can effectively retain heat and moisture and increase yields, its residues also cause serious "white pollution," threatening soil health and agricultural sustainability. Utilizing remote sensing technology to quickly and accurately obtain the spatial distribution of large-scale film-covered farmland is crucial for environmental monitoring and agricultural management.

[0003] Existing remote sensing methods for identifying mulched farmland can be mainly divided into two categories. The first category consists of traditional pixel-based machine learning methods (such as random forests and support vector machines). These methods rely on a large number of manually labeled samples and do not fully utilize spatial context information, resulting in segmentation results that are prone to salt-and-pepper noise and jagged, discontinuous boundaries. The second category consists of semantic segmentation networks based on deep learning. Although the accuracy has been improved, pixel-level labeled data is still required, and the model's generalization ability is limited by the distribution of training data, making it difficult to apply across regions.

[0004] In recent years, existing companies have proposed SAM models that have demonstrated strong zero-shot segmentation capabilities. However, directly applying them to the identification of mulched farmland in remote sensing images faces three problems: First, SAM is designed for three-channel natural images, while remote sensing images usually contain multispectral information, requiring adaptation; second, SAM relies on manually provided cue points or cue boxes, making it difficult to achieve large-scale automated application; and third, buildings, roads, etc. in remote sensing images have similar brightness to agricultural film, which can easily lead to misjudgment.

[0005] To address the above problems, this invention proposes a fully automated remote sensing segmentation method for mulched farmland that requires no manual annotation. Summary of the Invention

[0006] The purpose of this invention is to provide a remote sensing unsupervised segmentation method for mulched farmland based on the SAM model and automatically generated prompts, so as to solve the problems existing in the prior art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a remote sensing unsupervised segmentation method for mulched farmland based on a SAM model and automatically generated prompts, comprising the following steps:

[0008] S1: Acquire multispectral remote sensing images and perform radiometric calibration, atmospheric correction and geometric fine correction preprocessing;

[0009] S2: Based on the multispectral remote sensing image, select red light, green light, and near-infrared bands and map them to the R, G, and B channels of the RGB image respectively. Then, use quantile normalization to stretch the values ​​of each band to the [0, 255] range to obtain an RGB image adapted to SAM input.

[0010] S3: Based on the RGB image, calculate the brightness map and extract the high brightness region through dynamic threshold segmentation. At the same time, calculate the multi-scale fusion gradient map and extract the high gradient region through threshold segmentation. Perform a logical "AND" operation on the two to obtain the candidate region, and then generate an automatic prompt point set through DBSCAN clustering.

[0011] S4: Input the set of automatically prompted points into the SAM model to generate an initial segmentation mask for the mulched farmland, and segment each candidate region independently, while eliminating building areas based on the number of connected components and compactness.

[0012] S5: Take the union of all segmentation masks, perform morphological closing and opening operations in sequence, and output the final spatial distribution map of the covered farmland.

[0013] Furthermore, in step S2, the low quantile is set as [missing information] during quantile normalization. High quantiles are ,in and These are the mean and standard deviation for this band, respectively. The value range is 2.0 to 3.0.

[0014] Furthermore, in step S3, the dynamic threshold... The calculation formula is: ,in , and , The mean and standard deviation of the brightness of the agricultural film and background areas are respectively obtained by bimodal fitting of the image brightness histogram; before extracting the high brightness areas, the connected components of the brightness map are screened to remove noise areas with an area of ​​less than 50 pixels.

[0015] Furthermore, in step S3, the multi-scale gradient fusion uses a Gaussian pyramid to construct three scales, with the weighting weights of the gradient magnitudes at each scale being 0.5, 0.3, and 0.2 respectively; the high gradient region screening threshold is 1.5 times the overall gradient mean.

[0016] Furthermore, in step S3, the neighborhood radius of DBSCAN clustering is 5 pixels, and the minimum number of points is set to 1; for areas with an area exceeding 5000 pixels, sampling points are increased at equal intervals until the total number of points is no less than 5.

[0017] Furthermore, in step S4, the criteria for building removal are: the number of connected components in the candidate region is greater than 10 and the average area is less than 50 pixels, while the overall compactness is less than 0.3. The overall compactness is the ratio of the region area to the convex hull area.

[0018] Furthermore, in step S4, the SAM model adopts the ViT-H version, and during segmentation, the attention range is limited by the region clipping operation, retaining only the segmentation results within the current candidate region boundary.

[0019] Furthermore, in step S5, both the morphological closing and opening operations use a 5×5 pixel circular structural element.

[0020] Furthermore, the spatial resolution of the multispectral remote sensing image is better than or equal to 3 meters.

[0021] Furthermore, The preferred value is 2.5.

[0022] Compared with the prior art, the beneficial effects of the present invention are:

[0023] Completely unsupervised and requiring no labeled samples, this method achieves fully automated, zero-human-intervention-based segmentation of mulched farmland through an automatic cue point generation strategy that fuses brightness and gradient data. This significantly reduces labor costs and improves the method's cross-regional generalization ability.

[0024] The segmentation accuracy is high and the spatial continuity is good. Compared with traditional pixel-level classification methods, the agricultural film plots generated by this invention have no holes inside, clear and smooth boundaries, and no salt-and-pepper noise. The region-independent segmentation mechanism avoids plot adhesion and significantly improves spatial consistency.

[0025] It exhibits strong adaptability to high-resolution remote sensing imagery. By designing combinations of red, green, and near-infrared bands and quantile normalization, it fully leverages multispectral information, maintaining high resolution even when the agricultural film and background spectra are similar, and demonstrating good robustness to images from different time phases and regions.

[0026] With a high degree of automation, it is suitable for wide-ranging applications. No manual provision of any prompts is required; this invention automatically generates a uniformly distributed set of prompts, which, combined with the SAM model, enables batch and operational monitoring, supporting national or provincial-level dynamic monitoring of mulched farmland.

[0027] The post-processing is excellent, resulting in high-quality output images. The combined use of morphological closing and opening operations effectively eliminates micro-holes and isolated noise, and the output results can be directly used for subsequent analyses such as area statistics and change detection. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating the overall technical process of the method of the present invention.

[0029] Figure 2 This is a schematic diagram of the automatic prompt point generation step in this invention;

[0030] Figure 3 This is a comparison diagram of the effectiveness of this invention in dividing mulched farmland with other traditional methods (RF, SVM);

[0031] Figure 4 This is a comparison chart of the ablation experiment results in this invention. Detailed Implementation

[0032] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0033] See Figure 1-4 The present invention adopts the following technical solution: a remote sensing unsupervised segmentation method for mulched farmland based on the SAM model and automatically generated prompts, comprising the following steps:

[0034] S1: Remote Sensing Image Acquisition and Preprocessing

[0035] Multispectral remote sensing images covering the study area, including blue, green, red, and near-infrared bands, were acquired, with a spatial resolution of at least 3 meters. The raw remote sensing DN value images were then subjected to radiometric calibration, atmospheric correction, and geometric fine correction to remove reflectance anomalies caused by atmospheric scattering, sensor system errors, and geographic coordinate offsets. The final output was a standardized surface reflectance multispectral image. Radiometric calibration utilized official calibration coefficients from the satellite sensor to convert DN values ​​to reflectance at the top of the atmosphere. Atmospheric correction employed the 6S model or the FLAASH atmospheric correction model to eliminate atmospheric aerosol and water vapor interference. Geometric fine correction used high-precision field control point vectors as a reference for pixel georegistration.

[0036] S2: Optimal Three-Band Combination Construction and Color Gamut Normalization

[0037] To adapt to the native RGB three-channel input format of the SAM model and fully utilize the differentiated spectral reflectance characteristics of the mulch film in the green, red, and near-infrared bands, a customized band mapping and quantile normalization scheme was developed, which was broken down into two sub-steps: band combination design and quantile normalization.

[0038] S2.1 Band Combination Design: Based on the actual spectral response characteristics of mulched farmland, the mulch film has a high reflectivity in the green band and a significant difference in reflectivity between the near-infrared band and bare soil and vegetation. Three characteristic bands in the multispectral image—red, green, and near-infrared—were selected and mapped one by one to the R, G, and B channels of the SAM input RGB image. The mapping rules are: R channel = original multispectral red band reflectivity, G channel = original multispectral green band reflectivity, and B channel = original multispectral near-infrared band reflectivity. Invalid information in the blue band is discarded. Under the premise of adapting to the three-channel constraints of SAM, the characteristic spectral information of the mulch film is preserved to the maximum extent.

[0039] S2.2 Quantile Normalization: Quantile stretching and normalization are performed on the mapped R, G, and B single-band data one by one. First, the mean value of the reflectance of all pixels in each single band is calculated. with standard deviation ; Set the low quantile point for band pixel truncation = High quantile = Where k is the band stretching adjustment coefficient, The value range is limited to 2.0 to 3.0, which is preferred in this invention. =2.5; pixels with pixel values ​​below the low quantile are uniformly assigned a value of 0, and pixels with pixel values ​​above the high quantile are uniformly assigned a value of 255. Pixel values ​​within the high and low quantile range are uniformly stretched to the range of 0 to 255 pixel values ​​according to the linear mapping rule, and finally an 8-bit standard RGB image conforming to the SAM input format is generated.

[0040] S3: Automatic cue point generation based on brightness and gradient fusion

[0041] This step is the core of achieving automated SAM operation without human prompts. It filters candidate areas for plastic film in parallel from two dimensions: brightness features and edge gradient features. After the two types of candidate masks are fused, uniformly distributed SAM cue points are automatically generated through DBSCAN spatial clustering. This step is broken down into brightness-guided branch, gradient-guided branch, and mask fusion clustering sampling sub-steps:

[0042] S3.1 Brightness Guidance Point Generation: Calculates the full-image brightness map based on the standard RGB image output by S2. Brightness calculation formula: , , , Coordinates The pixel's three-channel grayscale values ​​are displayed; a dynamic adaptive threshold is used. High-brightness mulch film candidate regions are extracted from the segmented brightness map; dynamic threshold calculation formula:

[0043]

[0044] In the formula: , These represent the mean and standard deviation of brightness in the area covered by the plastic film, respectively. , These represent the mean and standard deviation of brightness in the non-mulch film background area, respectively. , , , The four statistical parameters were obtained by solving the bimodal fitting algorithm based on the full-image brightness histogram.

[0045] After threshold segmentation, connected component analysis is performed on the high-brightness binary image to remove small noise patches with a single connected component area of ​​less than 50 pixels. The remaining binary image is used as a brightness candidate mask.

[0046] S3.2 Gradient-Guided Cue Point Generation: A multi-scale Sobel edge operator is used to solve the gradient magnitude map of the standard RGB image scale by scale. Three gradient calculation scales are constructed based on the Gaussian pyramid, and the gradient magnitudes at the three scales are denoted as follows: Multi-scale gradient fusion is achieved using a weighted summation method. The formula for calculating the fused gradient is as follows:

[0047]

[0048] The gradient weights for each scale are fixed. The gradient segmentation threshold is set at 1.5 times the overall mean of the gradient values ​​of the whole image fusion. Pixels with values ​​higher than this threshold are identified as strong edge candidate pixels of the mulch film, and edge candidate masks are generated.

[0049] S3.3 Fusion and Redundancy Removal, and Cue Point Sampling: The brightness candidate mask output from S3.1 and the edge candidate mask output from S3.2 are subjected to a pixel-level logical AND operation. Only pixels that simultaneously satisfy the constraints of high brightness and strong edges are retained, generating the final mulch film candidate area mask. For each independent candidate connected region within the mask, the DBSCAN density clustering algorithm is used to perform spatial clustering processing on all effective pixels within the region. DBSCAN has fixed parameters: neighborhood radius = 5 pixels, minimum number of cluster points MinPts = 1, and the spatial geometric center point of each cluster is used as a SAM cue point. For large-area contiguous mulch film candidate regions with a single area exceeding 5000 pixels, an adaptive supplementary sampling rule with equal-spacing grids is adopted to ensure that the total number of cue points generated in the final region is not less than 5. Finally, all points are summarized to obtain a spatially uniform automatic cue point set.

[0050] S4: Unsupervised segmentation of independent regions based on the SAM model

[0051] Based on a pre-trained SAM large model, combined with automatically generated cue points, independent segmentation of different regions is achieved. A morphological prior-based building removal strategy is used to eliminate false candidate regions of white buildings. The process consists of two sub-steps: independent segmentation of blocks and automatic building removal.

[0052] S4.1 Independent Region Segmentation: The boundary range of a single candidate region output from S3.3 and all automatically prompted points within the region are input into the pre-trained SAM model one by one. This invention limits the SAM to use the ViT-H weight version. During the segmentation process, candidate region boundary clipping constraints are introduced to restrict the feature attention of the SAM model to fall only inside the current candidate region's bounding box, shielding the interference of adjacent plot pixels. An independent initial segmentation mask for each candidate region is generated to avoid the problem of adjacent covered plot segmentation results sticking together.

[0053] S4.2 Semantic Prior-Based Building Removal: Building removal can be performed after the candidate mask is generated in S3 or after the mask is generated in S4.1. The building removal quantification judgment rule is as follows: if the total number of connected components inside a single candidate region is greater than 10, the average area of ​​a single connected component in the region is less than 50 pixels, and the overall compactness of the region (actual area of ​​the region ÷ area of ​​the convex hull of the region) is less than 0.3, then the candidate region is determined to be a white building pseudo-target and is directly removed from the candidate mask and is no longer sent to the SAM model for segmentation.

[0054] S5: Post-processing and Result Output

[0055] The independent segmentation masks generated from all candidate regions in S4.1 are subjected to pixel-level logical "union" operations and spliced ​​together to form an integrated binary initial mask for the entire study area. Two post-processing steps are then performed: morphological closing operation and morphological opening operation. Both operations use a 5×5 circular structural element. The closing operation (dilation followed by erosion) is used to fill the small voids inside the covered plots caused by local spectral abrupt changes. The opening operation (erosion followed by dilation) is used to eliminate sporadic isolated noise patches at the edges of the patches. After post-processing, a binary raster map of the spatial distribution of covered farmland with the same spatial resolution as the original remote sensing image is output. This raster can be directly imported into GIS software for secondary applications such as area statistics and temporal change detection.

[0056] Example: Remote sensing segmentation experiment of mulched farmland in Fuyu City, Jilin Province

[0057] This embodiment selects the entire dryland farmland covered with plastic film in Fuyu City, Jilin Province as the research experimental area. It uses 3-meter spatial resolution multispectral remote sensing images acquired by the PlanetScope satellite's onboard sensor. The original images include four standard observation bands: blue, green, red, and near-infrared. Specific step-by-step implementation details are as follows:

[0058] S1 remote sensing image preprocessing: PlanetScope official sensor calibration parameters were retrieved, and the original DN values ​​of the image were converted pixel-by-pixel into surface reflectance data for the top of the atmosphere; the FLAASH model was used for full-frame atmospheric correction to eliminate interference from water vapor and aerosol scattering; the geographic registration of the image was completed using the vector boundary of the county-level administrative divisions of Fuyu City as the geometric correction benchmark; and the preprocessed multispectral image was cropped based on the administrative boundary vector of Fuyu City to remove invalid pixels outside the study area, resulting in a standardized multispectral reflectance image of the entire Fuyu City area.

[0059] S2 band combination and quantile normalization process: Strictly adhere to the channel mapping rule of R=red band, G=green band, B=near-infrared band; calculate the mean pixel reflectance for each band. Standard deviation elongation coefficient The preferred value of this invention is 2.5, and the cutoff interval is set accordingly. Pixels below the lower limit are set to 0, and pixels above the upper limit are set to 255. Pixels within the range are linearly stretched to the grayscale range of 0 to 255 to generate an 8-bit standard RGB image adapted to ViT-H-SAM.

[0060] S3 Automatic Prompt Point Generation Process: ① Brightness Branch: Using... The full-image brightness map is obtained by fitting a bimodal brightness histogram. Substitute the four statistical values ​​into... ① Solve for the dynamic segmentation threshold. After threshold segmentation, remove small connected regions with an area <50 pixels to generate brightness candidate masks; ② Gradient branch: Construct a 3-layer Gaussian pyramid, use the Sobel operator to calculate the gradient on different scales, and fix the gradient fusion weights at 0.5, 0.3, and 0.2. The gradient filtering threshold is taken as 1.5 times the average gradient of the fused gradient of the whole image to generate edge candidate masks; ③ After mask logic and fusion, DBSCAN clustering parameters: neighborhood radius 5 pixels, minimum number of points 1; Supplement sampling at equal intervals for contiguous overlay areas with a single connected region area >5000 pixels to ensure that the total number of prompt points in a single region is ≥5, and summarize to generate an automatic prompt point dataset for the entire study area.

[0061] In the S4SAM segmentation and building removal process: the official open-source pre-trained ViT-H version SAM weight file from Meta is used. Each candidate region is bound to a region border and internal cue points as input to SAM. During segmentation, the attention range is limited to the candidate region boundary. The building removal criteria are strictly followed: number of connected components > 10, average area of ​​a single block < 50 pixels, and compactness < 0.3. White building patches that meet all three conditions are directly removed and do not participate in SAM segmentation.

[0062] S5 morphological post-processing output stage: All independent candidate regions are segmented by SAM and stitched together in a union. A 5×5 pixel circular structural element is used uniformly. Closing operation is first used to fill the plot holes, and then opening operation is used to remove isolated noise points. Finally, a binary raster of the spatial distribution of mulched farmland in Fuyu City is output. The spatial resolution of the output raster remains unchanged at 3 meters compared with the original PlanetScope image.

[0063] Quantitative verification data of implementation effect: Under the premise of zero manual annotation and zero manual prompting for point marking throughout the process, the overall classification accuracy of the segmentation results of this invention is OA=94.2% and Kappa consistency coefficient=0.91. Compared with traditional machine learning algorithms: Random Forest (RF) has an overall accuracy of 82.5% and Support Vector Machine (SVM) has an overall accuracy of 79.3%. The accuracy of this invention is significantly better than the two traditional algorithms. Visual interpretation results show that there are no voids inside the output overlay patches, the boundaries of the plots are continuous and smooth, white roof buildings and hardened roads are all automatically removed, and there is no large-area false target misclassification phenomenon. This fully verifies the engineering feasibility and high-precision advantages of the technical solution of this invention.

[0064] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A remote sensing unsupervised segmentation method for mulched farmland based on the SAM model and automatically generated prompts, characterized in that: Includes the following steps: S1: Acquire multispectral remote sensing images and perform radiometric calibration, atmospheric correction and geometric fine correction preprocessing; S2: Based on the multispectral remote sensing image, select red light, green light, and near-infrared bands and map them to the R, G, and B channels of the RGB image respectively. Then, use quantile normalization to stretch the values ​​of each band to the [0, 255] range to obtain an RGB image adapted to SAM input. S3: Based on the RGB image, calculate the brightness map and extract the high brightness region through dynamic threshold segmentation. At the same time, calculate the multi-scale fusion gradient map and extract the high gradient region through threshold segmentation. Perform a logical "AND" operation on the two to obtain the candidate region, and then generate an automatic prompt point set through DBSCAN clustering. S4: Input the set of automatically prompted points into the SAM model to generate an initial segmentation mask for the mulched farmland, and segment each candidate region independently, while eliminating building areas based on the number of connected components and compactness. S5: Take the union of all segmentation masks, perform morphological closing and opening operations in sequence, and output the final spatial distribution map of the covered farmland.

2. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S2, the low quantile is set as the quantile normalization point. High quantiles are ,in and These are the mean and standard deviation for this band, respectively. The value range is 2.0 to 3.

0.

3. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S3, the dynamic threshold The calculation formula is: ,in , and , The mean and standard deviation of the brightness of the agricultural film and background areas are respectively obtained by bimodal fitting of the image brightness histogram; before extracting the high brightness areas, the connected components of the brightness map are screened to remove noise areas with an area of ​​less than 50 pixels.

4. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S3, the multi-scale gradient fusion uses a Gaussian pyramid to construct three scales, with the weighting weights of the gradient magnitudes at each scale being 0.5, 0.3, and 0.2 respectively; the high gradient region screening threshold is 1.5 times the overall gradient mean.

5. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S3, the neighborhood radius of DBSCAN clustering is 5 pixels, and the minimum number of points is set to 1; for areas with an area exceeding 5000 pixels, sampling points are added at equal intervals until the total number of points is no less than 5.

6. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S4, the criteria for building removal are: the number of connected components in the candidate region is greater than 10 and the average area is less than 50 pixels, while the overall compactness is less than 0.

3. The overall compactness is the ratio of the region area to the convex hull area.

7. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S4, the SAM model adopts the ViT-H version. During segmentation, the attention range is limited by the region clipping operation, and only the segmentation results within the current candidate region boundary are retained.

8. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: In step S5, both the morphological closing and opening operations use a 5×5 pixel circular structural element.

9. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 1, is characterized in that: The spatial resolution of the multispectral remote sensing image is better than or equal to 3 meters.

10. The unsupervised remote sensing segmentation method for mulched farmland based on the SAM model and automatically generated prompts, as described in claim 2, is characterized in that: The preferred value is 2.5.

Images