Straw coverage detection method and device based on cross-scale multi-source data fusion

Abstract

The present application provides a kind of straw coverage detection method and device of cross-scale multi-source data fusion, applied to remote sensing image processing technical field, above-mentioned method includes: the ground photograph is input to visual big model and is carried out semantic segmentation, obtains the straw coverage of corresponding ground scale range;Based on the straw coverage of ground scale range, the scale range adaptation is carried out to unmanned aerial vehicle image, and unmanned aerial vehicle sample image is obtained;Based on the image similarity calculation of unmanned aerial vehicle image block corresponding ground photograph and unmanned aerial vehicle sample image, the overall similarity of each unmanned aerial vehicle image block is obtained;Select a fixed proportion of target unmanned aerial vehicle image block to generate new sample;Based on unmanned aerial vehicle image straw coverage sample set, construct straw coverage regression model;Satellite remote sensing image is input to straw coverage regression model, and the straw coverage extraction result of target area is obtained;Through the present application, the straw coverage of large area range can be realized accurate extraction.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image processing technology, and in particular to a method and apparatus for detecting straw coverage through cross-scale multi-source data fusion. Background Technology

[0002] Traditional methods for detecting crop straw cover in farmland include visual inspection or measurement using a tape measure in the field. However, these direct measurement or photographic methods are laborious, time-consuming, and, due to their inherent discontinuity, unsuitable for large-scale implementation.

[0003] Satellite remote sensing technology has become a popular method for estimating crop straw cover due to its high spatial coverage and fast revisit time. The most widely used approach to estimating crop straw cover using satellite remote sensing is to establish a correlation model between field measurement data and straw cover spectral indices using parametric and nonparametric methods. Commonly used spectral indices include the Normalized Difference Senescent Vegetation Index (NDSVI) and the Normalized Difference Residue Index (NDRI). The accuracy of this method, which establishes a correlation model between field measurement data and satellite spectral indices, largely depends on the quantity of field observation data, which is costly.

[0004] It is evident that the straw coverage detection methods in related technologies have significant limitations and technical problems. Summary of the Invention

[0005] This invention provides a method and apparatus for detecting straw coverage by cross-scale multi-source data fusion, which solves the shortcomings of existing straw coverage detection methods and enables accurate extraction of straw coverage over a large area.

[0006] This invention provides a method for detecting straw coverage by cross-scale multi-source data fusion, comprising the following steps. The process involves acquiring satellite remote sensing imagery, UAV imagery, and ground-based photographs of the target area. The ground-based photographs are input into a large-scale visual model for semantic segmentation to obtain the straw coverage at the ground scale corresponding to the ground-based photographs. Based on the straw coverage at the ground scale, the UAV imagery is scale-adapted to obtain UAV sample images. The UAV sample images are divided into multiple UAV image blocks according to a preset grid. Image similarity is calculated between the ground-based photographs and the UAV sample images corresponding to each UAV image block to obtain the overall similarity of each UAV image block. Each UAV image block is sorted according to the overall similarity, and a fixed proportion of target UAV image blocks are selected to generate new samples, resulting in a UAV image straw coverage sample set. Based on the UAV image straw coverage sample set, a straw coverage regression model based on the Transformer model is constructed. The satellite remote sensing imagery is input into the straw coverage regression model to obtain the straw coverage extraction result of the target area output by the straw coverage regression model.

[0007] According to the present invention, a method for detecting straw coverage by cross-scale multi-source data fusion is provided. The method involves adapting the scale range of a UAV image to obtain a UAV sample image based on the straw coverage within the ground scale range. The method includes: determining the side length of the ground scale range corresponding to the ground-captured photograph; using the ratio of the side length of the ground scale range to the ground resolution of the UAV image as the scale size of the UAV image; and cropping the UAV image according to the scale size to obtain the UAV sample image. The straw coverage of the UAV sample image is correlated with the straw coverage within the ground scale range.

[0008] According to the present invention, a method for detecting straw coverage through cross-scale multi-source data fusion, wherein the method calculates the overall similarity of each drone image block by performing image similarity calculation based on ground-captured photos and drone sample images corresponding to each of the plurality of drone image blocks, including: for each drone image block, performing the following processing: determining a first structural similarity index between the drone image block and the ground-captured photos corresponding to the drone image block; determining a second structural similarity index between the drone image block and the drone sample images corresponding to the drone image block; and performing a weighted summation based on the first structural similarity index and the second structural similarity index to obtain the overall similarity of the drone image block.

[0009] According to the present invention, a method for detecting straw coverage through cross-scale multi-source data fusion includes the following steps: selecting a fixed proportion of target UAV image blocks to generate new samples and obtaining a set of UAV image straw coverage samples; determining the straw coverage of the UAV sample image with the highest overall similarity to the target UAV image blocks; and using the product of the straw coverage and the overall similarity of the target UAV image blocks as the straw coverage of the target UAV image blocks to obtain the set of UAV image straw coverage samples.

[0010] According to the present invention, a method for detecting straw coverage using cross-scale multi-source data fusion, wherein determining the straw coverage of the drone sample image with the highest overall similarity to the target drone image block includes: determining a first structural similarity index between the similarity of the target drone image block and each ground-captured photograph; determining a second structural similarity index between the similarity of the target drone image block and the drone sample image corresponding to each ground-captured photograph; performing a weighted summation based on the first structural similarity index and the second structural similarity index to obtain the overall similarity of each drone sample image corresponding to the target drone image block; and taking the straw coverage of the drone sample image with the highest overall similarity as the straw coverage of the target drone image block.

[0011] According to the present invention, a method for detecting straw coverage by cross-scale multi-source data fusion is provided. The method involves constructing a straw coverage regression model based on a Transformer model, using the straw coverage sample set from UAV imagery. This includes: aggregating the straw coverage sample set from UAV imagery based on the linear relationship between the spatial resolution of the satellite remote sensing imagery and the spatial resolution of the UAV imagery, to obtain a straw coverage sample set at the satellite imagery scale; and constructing a straw coverage regression model based on a Transformer model using the straw coverage sample set at the satellite imagery scale, with satellite image reflectance and spatial features as inputs and straw coverage as the regression object.

[0012] This invention also provides a straw coverage detection device based on cross-scale multi-source data fusion, comprising the following modules: an acquisition module for acquiring satellite remote sensing images, UAV images, and ground-based photographs of a target area; a segmentation module for inputting the ground-based photographs into a large-scale visual model for semantic segmentation to obtain the straw coverage at the ground scale corresponding to the ground-based photographs output by the large-scale visual model; an adaptation module for performing scale range adaptation on the UAV images based on the straw coverage at the ground scale to obtain UAV sample images; a partitioning module for partitioning the UAV sample images according to a preset rule grid to obtain multiple UAV image blocks; and a similarity module for performing similarity analysis on the multiple UAV image blocks. The image similarity calculation is performed between the ground-captured photo corresponding to each UAV image block in the image block and the UAV sample image to obtain the overall similarity of each UAV image block; the generation module is used to sort each UAV image block according to the overall similarity and select a fixed proportion of target UAV image blocks to generate new samples, resulting in a UAV image straw coverage sample set; the construction module is used to construct a straw coverage regression model based on the UAV image straw coverage sample set; the extraction module is used to input the satellite remote sensing image into the straw coverage regression model to obtain the straw coverage extraction result of the target area output by the straw coverage regression model.

[0013] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the straw coverage detection method of cross-scale multi-source data fusion as described above.

[0014] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the straw coverage detection method by cross-scale multi-source data fusion as described above.

[0015] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the straw coverage detection method by cross-scale multi-source data fusion as described above.

[0016] The present invention provides a method and apparatus for detecting straw coverage by cross-scale multi-source data fusion. First, it uses a large visual model to accurately segment ground photos to obtain highly reliable ground-scale coverage ground values. Based on this, it performs scale adaptation and grid division on UAV images. Then, it selects representative target image blocks by calculating the similarity between image blocks and ground photos to automatically generate a UAV coverage sample set. Finally, it uses these high-quality samples to train a Transformer regression model, effectively transferring the accurate features learned at small scales (ground and UAV) to large-scale satellite images, ultimately achieving high-precision straw coverage extraction over large areas based on satellite images. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the straw coverage detection method based on cross-scale multi-source data fusion provided by the present invention.

[0019] Figure 2 This is a schematic diagram of sample generation based on ground photographs provided by the present invention.

[0020] Figure 3 This is a schematic diagram of sample generation based on UAV image amplification provided by the present invention.

[0021] Figure 4 This is a flowchart of the straw coverage detection method based on cross-scale multi-source data fusion provided by the present invention.

[0022] Figure 5 This is a schematic diagram of the module of the straw coverage detection device for cross-scale multi-source data fusion provided by the present invention.

[0023] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The terms "first," "second," etc., used in the specification and claims of this invention are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0026] This invention relates to remote sensing image processing technology, specifically to a method for detecting straw coverage by cross-scale multi-source data fusion. This method calculates and analyzes straw coverage in farmland over a large area by combining ground-captured photographs, UAV aerial images, and satellite images, thereby serving the application needs of monitoring farmland conditions in agricultural remote sensing monitoring.

[0027] Remote sensing data has a very wide range of applications and has been effectively applied in many fields such as environmental monitoring and agricultural remote sensing monitoring. In agriculture, remote sensing technology is widely used for crop growth monitoring and yield prediction. It analyzes crop growth status through vegetation indices (such as NDVI and EVI) to identify areas of slow or vigorous growth; combines historical data with real-time remote sensing imagery to establish yield models and predict grain yields in advance; and it can track crop growth stages (sowing, emergence, flowering, maturity, etc.) to optimize agricultural scheduling. Meanwhile, remote sensing technology has also been widely applied in precision agriculture and variable management. It can combine remote sensing data on soil nutrients and crop requirements to achieve precise fertilization / irrigation; and it can use drone or satellite data to provide operational guidance for agricultural machinery, generating prescription maps for variable fertilization.

[0028] Furthermore, remote sensing technology has significant potential in land use and farmland management. It can identify farmland boundaries, abandoned land, and reclaimed land, supporting arable land protection; distinguish crop types (such as wheat / corn) through multi-temporal imagery, monitoring crop rotation systems; and assess the distribution and status of irrigation canals, roads, and other facilities, monitoring farmland infrastructure. In the field of agricultural remote sensing monitoring, one issue requiring attention is the calculation of straw cover in farmland. The proportion of farmland covered by straw is called straw cover degree.

[0029] Applying crop straw mulch after harvest can mitigate wind and water erosion, increase soil organic carbon content and microbial populations, improve soil water retention capacity, and enhance soil physicochemical properties; this is known as conservation tillage. Experience has shown that conservation tillage is the most environmentally friendly farming practice, significantly reducing greenhouse gas emissions, increasing crop yields in some cases, and enhancing soil microbial diversity and soil organic carbon. Returning crop straw to the field, as an alternative to traditional straw disposal methods, provides an effective way to mitigate air pollution and reduce harmful emissions. Therefore, it is crucial to rapidly and on a large scale estimate the straw coverage of farmland.

[0030] Currently, drones are becoming an important data acquisition platform in the field of agricultural remote sensing. Because drones can acquire data at relatively low flight altitudes, the images they obtain have a high degree of similarity to photos taken from the ground during field observations. Therefore, drone imagery has become an important tool for establishing correlations between ground-based photographs and satellite imagery.

[0031] In this embodiment of the invention, the correlation between observation data at multiple scales (ground photos), drone images, and satellite remote sensing images) is established through data fusion. Initial training samples are obtained through a small number of ground photos, and the samples are further amplified by drone images. Finally, a straw coverage analysis model is established using satellite images to obtain straw coverage information for farmland over a large area.

[0032] Optionally, the straw coverage detection method based on cross-scale multi-source data fusion in this embodiment of the invention can be executed by a server, by a terminal device, or by both a server and a terminal device. For example, the straw coverage detection method based on cross-scale multi-source data fusion in this embodiment can be executed by a server.

[0033] Figure 1 This is a flowchart illustrating the straw coverage detection method based on cross-scale multi-source data fusion provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps.

[0034] Step 101: Acquire satellite remote sensing images, drone images, and ground-based photographs of the target area.

[0035] Acquire satellite remote sensing images of the target area, including: acquiring satellite remote sensing images covering the entire target agricultural area. The spatial extent of the images should completely include the farmland plots to be analyzed. The satellite remote sensing images should include visible and near-infrared bands to support subsequent spectral analysis.

[0036] Acquiring drone imagery of the target area includes: planning drone flight paths within the target area to ensure that the drone imagery covers all farmland plots where ground photo sampling points are located. The drone imagery must have RGB or visible light bands to support similarity comparison with ground photos.

[0037] Acquire ground-based photographs of the target area, including: evenly distributing sampling points within the target farmland area, with each sampling point representing an independent observation unit; photographing a 1m x 1m ground area at each sampling point; and taking photos vertically downwards to ensure the images are free from distortion, shadows, or reflections. Clear coverage of straw and bare ground is essential for visual model segmentation and recognition (such as SAM).

[0038] It should be noted that satellite remote sensing imagery, drone imagery, and ground-based photographs must be acquired within the same crop straw mulch cycle to avoid errors introduced by environmental changes. All imagery must undergo geometric correction and geographic coordinate matching to ensure spatial alignment of multi-source data at the same location.

[0039] Step 102: Input the ground-captured photos into the visual big data model for semantic segmentation to obtain the straw coverage of the ground scale range corresponding to the ground-captured photos output by the visual big data model.

[0040] In this embodiment of the invention, a large visual model is used to automatically analyze ground-based photographs, identify the straw coverage in the photographs, and calculate the straw coverage within the ground-scale range represented by the photographs.

[0041] In some embodiments, the visual big model refers to a general model capable of semantic segmentation of natural scene images, namely the Segment Anything Model (SAM model). The SAM model can automatically perform semantic segmentation on input ground photographs, distinguishing between bare ground and straw.

[0042] The straw coverage represented by the ground-based photograph is calculated by taking the photograph vertically downwards. If the ground area of ​​the photograph is 1 meter × 1 meter, the straw coverage of the sample point is obtained by dividing the number of pixels representing straw coverage in the ground-based photograph by the total number of pixels in the photograph (expressed as a percentage). The ground-based photograph and the obtained straw coverage data constitute a sample based on the ground-based photograph.

[0043] Step 103: Based on the straw coverage at the ground scale, the drone imagery is scale-adapted to obtain drone sample images.

[0044] In this embodiment of the invention, the straw coverage at the ground scale is adapted to the straw coverage at the drone scale based on the spatial resolution of the drone image, forming a straw coverage sample based on the drone image (i.e., drone sample image).

[0045] According to the present invention, a method for detecting straw coverage through cross-scale multi-source data fusion is provided. Based on straw coverage within a ground-scale range, the method adapts UAV imagery to a scale range to obtain UAV sample images, including:

[0046] Determine the side length of the ground scale range corresponding to the ground-based photographs;

[0047] The ratio of the side length of the ground scale range to the ground resolution of the UAV image is used as the scale of the UAV image.

[0048] The drone images are cropped according to their scale to obtain drone sample images, where the straw coverage of the drone sample images is correlated with the straw coverage at the ground scale.

[0049] In this embodiment of the invention, based on the ground resolution of the UAV image, the size of the UAV image sample is obtained by dividing the side length of the ground area covered by the ground-captured photo by the ground resolution of the UAV image; the UAV image is cropped with the obtained UAV image sample size as a constraint, and combined with the straw coverage data calculated above to form a UAV sample image.

[0050] In some embodiments, the ground side length is fixed at 1 meter (determined by the shooting specifications). The ground resolution of the UAV image is calculated using the flight altitude and the camera focal length (e.g., 0.02 meters / pixel). If the UAV resolution is 0.02 meters / pixel, then the sample size = 1 / 0.02 = 50 pixels × 50 pixels.

[0051] Centered on each ground photo sampling point, the corresponding area is cropped from the UAV imagery according to the calculated size (e.g., 50×50 pixels) to obtain UAV sample image patches. The straw coverage of the UAV sample image patches directly inherits (or corresponds to) the calculated result (percentage value) of the corresponding ground photo, without the need for recalculation.

[0052] Through this invention, the size is calculated using resolution ratios, ensuring a strict correspondence between the UAV sample and the actual ground area, thus eliminating scale errors. The true coverage value of the ground photograph (verified through large-scale visual model segmentation) is directly inherited, avoiding deviations introduced by recalculating the UAV scale.

[0053] Step 104: Divide the UAV sample image into multiple UAV image blocks according to the preset rule grid.

[0054] In this embodiment of the invention, the regular grid is a uniform two-dimensional segmentation frame that covers the entire UAV sample image. The grid size (i.e., the width and height of each cell) is predefined and can be adjusted according to actual application requirements, such as dividing it into fixed-size squares, for example, 50×50 pixels or 100×100 pixels.

[0055] The purpose of grid partitioning is to decompose UAV sample images into multiple independent image blocks, facilitating block-by-block processing. This avoids the subjectivity of manual random partitioning and ensures the systematic and repeatable nature of coverage sample generation.

[0056] In some embodiments, a computational geometry algorithm (such as the sliding window method) is used to overlay a regular grid onto the UAV sample image: the window size is determined by the grid size, and the window is moved in row-major order, starting from the image coordinate (0,0). The area covered by each window is cropped into an independent UAV image patch. For example, if the sample image size is 500×500 pixels and the grid cell size is 50×50 pixels, then 100 image patches (10 rows × 10 columns) are obtained after partitioning. If the image size is not divisible by the grid size, the remaining boundary pixels are discarded.

[0057] Each output UAV image block is stored as an independent data unit, along with its grid coordinate index (such as row and column numbers) to facilitate subsequent similarity matching.

[0058] Step 105: Calculate the image similarity between the ground-captured photos corresponding to each UAV image block and the UAV sample images in multiple UAV image blocks to obtain the overall similarity of each UAV image block.

[0059] In this embodiment of the invention, image similarity calculations are performed on each UAV image block with the corresponding ground-captured photos and UAV sample images to obtain the overall similarity.

[0060] According to the present invention, a method for detecting straw coverage based on cross-scale multi-source data fusion is provided. This method calculates image similarity between ground-captured photos and drone sample images corresponding to each drone image block from multiple drone image blocks, obtaining the overall similarity of each drone image block, including:

[0061] For each drone image block in a set of multiple drone image blocks, perform the following processing:

[0062] Determine the first structural similarity index between the UAV image patch and the corresponding ground-captured photo;

[0063] Determine the second structural similarity index between the UAV image patch and the corresponding UAV sample image;

[0064] The overall similarity of UAV image blocks is obtained by weighted summation of the first structural similarity index and the second structural similarity index.

[0065] In this embodiment of the invention, the overall similarity refers to the similarity calculated between the UAV image block and the UAV sample image and the ground-taken photo, respectively. The similarity calculation adopts the SSIM (Structural Similarity) index, which measures the structural similarity between two images. Then, the two calculated similarity values ​​are multiplied by 0.5 and summed to obtain the overall similarity SSIM_total.

[0066] Here, the first structural similarity index is used to measure the structural similarity between a UAV image patch and its corresponding ground-based photograph. The second structural similarity index is used to measure the structural similarity between a UAV image patch and its corresponding UAV sample image.

[0067] In this embodiment of the invention, the structural similarity index (SSIM) formula is used to calculate the similarity of brightness, contrast, and structure between two images.

[0068] Through the embodiments of the present invention, the reliability of the truth value is ensured by the structural similarity between the UAV image block and the corresponding ground-captured photo, and the scale consistency is verified by the structural similarity between the UAV image block and the corresponding UAV sample image.

[0069] Step 106: Sort each UAV image block according to the overall similarity, and select a fixed proportion of target UAV image blocks to generate new samples, thus obtaining a UAV image straw coverage sample set.

[0070] In this embodiment of the invention, selecting a fixed proportion means that the similarity calculated for all UAV image blocks is sorted in descending order from largest to smallest, and then the top 80% of UAV image blocks in descending order are selected to generate samples.

[0071] According to the present invention, a method for detecting straw coverage through cross-scale multi-source data fusion is provided, which selects a fixed proportion of target UAV image blocks to generate new samples, thereby obtaining a set of UAV image straw coverage samples, including:

[0072] Determine the straw coverage of the drone sample image that has the highest overall similarity to the target drone image patch;

[0073] The straw coverage is calculated by multiplying the straw coverage by the overall similarity of the target drone image patch, thus obtaining the straw coverage sample set of drone image straw coverage.

[0074] In this embodiment of the invention, a fixed proportion of image blocks are selected from all drone image blocks to generate new samples. The straw coverage of these samples is calculated by normalization scaling based on the closest sample, resulting in a large number of drone image straw coverage samples.

[0075] Here, the normalized scaling method calculation refers to the calculation of the straw coverage of the drone image block to be processed, which is obtained by multiplying the straw coverage of the sample closest to the drone image block by the overall similarity. That is, if the drone image block to be processed is completely consistent with the sample image (SSIM_total=1), then the straw coverage of the sample is directly assigned to the drone image block to be processed (×SSIM_total=×1).

[0076] In some embodiments, for each target UAV image patch, the overall similarity (SSIM_total) with all known samples (including ground-captured photos and UAV image samples) is calculated.

[0077] Select the sample with the highest overall similarity as the most similar sample, and read the straw coverage of the most similar sample;

[0078] If the overall similarity between the target drone image patch and the closest sample is 1 (completely identical), then its straw coverage is directly inherited; if there are differences between the target drone image patch and the closest sample, then the straw coverage is reduced according to the similarity ratio.

[0079] refer to Figure 2 , Figure 2 This is a schematic diagram of sample generation based on ground photos provided by the present invention, which includes: manually photographed points / samples based on ground photos, drone shooting areas, and agricultural areas.

[0080] refer to Figure 3 , Figure 3 This is a schematic diagram of sample generation based on UAV image augmentation provided by the present invention, which includes: a sample based on UAV image augmentation, a UAV shooting area, and an agricultural area.

[0081] Through the embodiments of the present invention, a similarity weighting mechanism is adopted, so that the closer the image structure is to the reference sample, the more reliable the straw coverage assignment is.

[0082] According to the present invention, a method for detecting straw coverage through cross-scale multi-source data fusion determines the straw coverage of a drone sample image that has the highest overall similarity to a target drone image patch, including:

[0083] Determine the first structural similarity index between the target UAV image patch similarity and each ground-captured photograph;

[0084] Determine the second structural similarity index between the similarity of the target UAV image patch and the UAV sample image corresponding to each ground-captured photo;

[0085] The overall similarity of each UAV sample image corresponding to the target UAV image block is obtained by weighted summation of the first structural similarity index and the second structural similarity index.

[0086] The straw coverage of the drone sample image with the highest overall similarity is used as the straw coverage of the target drone image block.

[0087] In this embodiment of the invention, the closest sample refers to the sample with the highest overall similarity after calculating the overall similarity between the UAV image block and all ground-taken photos and corresponding UAV sample images. The straw coverage of the closest sample is then used as the straw coverage of the target UAV image block.

[0088] Through the embodiments of the present invention, ground-captured photos ensure the accuracy of the true values, UAV sample images ensure consistency at the same scale, and a weighting mechanism balances the contributions of the two, avoiding bias from a single data source.

[0089] Step 107: Based on the sample set of straw coverage from UAV imagery, construct a straw coverage regression model based on the Transformer model.

[0090] In this embodiment of the invention, the obtained UAV image straw coverage samples are aggregated to form a straw coverage sample set at the satellite image scale, and a straw coverage regression model is constructed based on the Transformer model.

[0091] Here, aggregating straw coverage samples from UAV images into a pixel-by-pixel straw coverage sample set from satellite images means aggregating straw coverage samples at the UAV scale into straw coverage samples at the corresponding satellite image scale based on the spatial resolution correspondence between UAV images and satellite images.

[0092] According to the present invention, a method for detecting straw coverage based on cross-scale multi-source data fusion is provided. This method constructs a straw coverage regression model based on a Transformer model, using a set of straw coverage samples from UAV imagery. The method includes:

[0093] Based on the linear relationship between the spatial resolution of satellite remote sensing images and the spatial resolution of UAV images, the straw coverage sample set of UAV images is aggregated to obtain a straw coverage sample set at the satellite image scale.

[0094] Based on a sample set of straw coverage at the satellite image scale, a straw coverage regression model based on the Transformer model is constructed, using satellite image reflectance and spatial features as inputs and straw coverage as the regression object.

[0095] In this embodiment of the invention, constructing a straw coverage regression model based on the Transformer model means taking straw coverage sample data at the satellite image scale, using satellite image reflectivity and spatial characteristics as inputs, straw coverage as the regression object, selecting the Transformer model, setting model parameters, and then optimizing the model. After the model optimization is stable, a straw coverage regression model that can be used for satellite imagery is obtained.

[0096] Through the embodiments of the present invention, this cross-scale multi-source data fusion straw coverage detection method makes full use of the correlation between observation data at different scales (ground photos, UAV images, satellite images). Through hierarchical decoupling processing, a large-area straw coverage detection method based on satellite images can be effectively established.

[0097] Step 108: Input the satellite remote sensing image into the straw coverage regression model to obtain the straw coverage extraction result of the target area output by the straw coverage regression model.

[0098] In this embodiment of the invention, satellite remote sensing images, drone images, and ground-based photos obtained through various observation methods such as ground (field ground-based photos), drone (drone-based images), and satellite (satellite remote sensing images) are comprehensively utilized. First, semantic segmentation is performed on the ground-based photos based on a large visual model to obtain the straw coverage in the photos. Then, the straw coverage degree of the area is calculated. Next, straw coverage sample data based on drone images is obtained by comparing and learning between the ground-based photos and drone images. Finally, a straw coverage regression model based on satellite remote sensing images is established based on the Transformer model to achieve accurate extraction of straw coverage over a large area.

[0099] refer to Figure 4 , Figure 4 This is a flowchart of the straw coverage detection method based on cross-scale multi-source data fusion provided by the present invention.

[0100] like Figure 4As shown, ground-based photographs are first input into an automatic analysis module based on a large visual model to generate ground-scale straw coverage information. Then, based on this, straw coverage samples adapted to the UAV scale are generated. This step simultaneously receives UAV image input and guides a sample amplification branch based on contrast enhancement between UAV imagery and ground photographs. Another branch performs straw coverage regression on satellite imagery based on a Transformer model. Finally, the two branches merge to output straw coverage extraction results over a large area.

[0101] Through the embodiments of this invention, firstly, a visual large model is used to accurately segment ground photos to obtain highly reliable ground-scale coverage ground values; based on this, the scale of UAV images is adapted and gridded; then, by calculating the similarity between image patches and ground photos, representative target image patches are selected to automatically generate a UAV coverage sample set; finally, these high-quality samples are used to train a Transformer regression model, effectively transferring the accurate features learned at small scales (ground and UAV) to large-scale satellite images, ultimately achieving high-precision straw coverage extraction over large areas based on satellite images.

[0102] The straw coverage detection device based on cross-scale multi-source data fusion provided by the present invention will be described below. The straw coverage detection device based on cross-scale multi-source data fusion described below can be referred to in correspondence with the straw coverage detection method based on cross-scale multi-source data fusion described above.

[0103] refer to Figure 5 , Figure 5 This is a schematic diagram of the module of the straw coverage detection device for cross-scale multi-source data fusion provided by the present invention.

[0104] The acquisition module 501 is used to acquire satellite remote sensing images, UAV images, and ground-captured photos of the target area.

[0105] The segmentation module 502 is used to input the ground-shot photos into the visual big model for semantic segmentation, and obtain the straw coverage of the ground scale range corresponding to the ground-shot photos output by the visual big model.

[0106] The adaptation module 503 is used to adapt the scale range of the UAV imagery based on the straw coverage within the ground scale range to obtain UAV sample images.

[0107] The segmentation module 504 is used to segment the UAV sample image according to a preset rule grid to obtain multiple UAV image blocks;

[0108] The similarity module 505 is used to calculate the image similarity between the ground-captured photos corresponding to each UAV image block and the UAV sample images in multiple UAV image blocks, so as to obtain the overall similarity of each UAV image block.

[0109] The generation module 506 is used to sort each UAV image block according to the overall similarity and select a fixed proportion of target UAV image blocks to generate new samples, thereby obtaining a UAV image straw coverage sample set.

[0110] Module 507 is used to construct a straw coverage regression model based on the Transformer model based on the straw coverage sample set of UAV imagery.

[0111] The extraction module 508 is used to input satellite remote sensing images into the straw coverage regression model to obtain the straw coverage extraction results of the target area output by the straw coverage regression model.

[0112] Specifically, the straw coverage detection device based on cross-scale multi-source data fusion provided by the present invention can realize all the method steps implemented in the above-mentioned cross-scale multi-source data fusion straw coverage detection method embodiment, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0113] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 6As shown, the electronic device may include: a processor 610, a communication interface 620, a memory 630, and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute a straw coverage detection method based on cross-scale multi-source data fusion. This method includes: acquiring satellite remote sensing images, UAV images, and ground-based photographs of the target area; inputting the ground-based photographs into a visual large-scale model for semantic segmentation to obtain the straw coverage at the ground scale range corresponding to the ground-based photographs output by the visual large-scale model; performing scale range adaptation on the UAV images based on the straw coverage at the ground scale range to obtain UAV sample images; dividing the UAV sample images according to a preset rule grid to obtain multiple UAV image blocks; and based on the straw coverage at the ground scale range, performing scale range adaptation on the UAV images to obtain UAV sample images. Image similarity calculations are performed between the ground-captured photos corresponding to each UAV image block and the UAV sample images to obtain the overall similarity of each UAV image block. Each UAV image block is then sorted according to its overall similarity, and a fixed proportion of target UAV image blocks are selected to generate new samples, resulting in a UAV image straw coverage sample set. Based on this sample set, a straw coverage regression model based on the Transformer model is constructed. Satellite remote sensing images are input into the straw coverage regression model to obtain the straw coverage extraction results for the target area output by the model.

[0114] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0115] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cross-scale multi-source data fusion straw coverage detection method provided by the above methods. This method includes: acquiring satellite remote sensing images, UAV images, and ground photographs of a target area; inputting the ground photographs into a visual large model for semantic segmentation to obtain the straw coverage of the ground photographs corresponding to the ground scale range output by the visual large model; performing scale range adaptation on the UAV images based on the straw coverage of the ground scale range to obtain UAV sample images; and following preset rules... The UAV sample images are divided into grids to obtain multiple UAV image blocks. Image similarity is calculated between the ground-captured photos corresponding to each UAV image block and the UAV sample images to obtain the overall similarity of each UAV image block. Each UAV image block is sorted according to its overall similarity, and a fixed proportion of target UAV image blocks are selected to generate new samples, resulting in a UAV image straw coverage sample set. Based on this sample set, a straw coverage regression model based on the Transformer model is constructed. Satellite remote sensing imagery is input into the straw coverage regression model to obtain the straw coverage extraction results for the target area output by the model.

[0116] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, this computer program implements a method for detecting straw coverage by cross-scale multi-source data fusion provided by the methods described above. This method includes: acquiring satellite remote sensing images, UAV images, and ground-based photographs of a target area; inputting the ground-based photographs into a large-scale visual model for semantic segmentation to obtain the straw coverage at the ground scale corresponding to the ground-based photographs output by the large-scale visual model; performing scale range adaptation on the UAV images based on the straw coverage at the ground scale to obtain UAV sample images; and dividing the UAV sample images according to a preset rule grid. Multiple UAV image patches are obtained. Image similarity is calculated between the ground-captured photos corresponding to each UAV image patch and the UAV sample images to obtain the overall similarity of each UAV image patch. Each UAV image patch is sorted according to its overall similarity, and a fixed proportion of target UAV image patches are selected to generate new samples, resulting in a UAV image straw coverage sample set. Based on this sample set, a straw coverage regression model based on the Transformer model is constructed. Satellite remote sensing images are input into the straw coverage regression model to obtain the straw coverage extraction results for the target area output by the model.

[0117] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0118] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0119] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting straw cover through cross-scale multi-source data fusion, characterized in that, include: Acquire satellite remote sensing images, drone images, and ground-based photographs of the target area; The ground-photographed images are input into a large visual model for semantic segmentation, and the straw coverage of the ground-scale range corresponding to the ground-photographed images is obtained from the output of the large visual model. Based on the straw coverage within the aforementioned ground scale range, the drone imagery is scale-adapted to obtain drone sample images; The UAV sample image is divided into multiple UAV image blocks according to a preset grid pattern. Image similarity calculation is performed on the ground-captured photos corresponding to each of the multiple UAV image blocks and the UAV sample images to obtain the overall similarity of each UAV image block; Each UAV image block is sorted according to the overall similarity, and a fixed proportion of target UAV image blocks are selected to generate new samples, resulting in a UAV image straw coverage sample set. Based on the aforementioned set of straw coverage samples from UAV images, a straw coverage regression model based on the Transformer model is constructed. The satellite remote sensing image is input into the straw coverage regression model to obtain the straw coverage extraction result of the target area output by the straw coverage regression model; The scale range adaptation of the UAV imagery based on the straw coverage within the ground scale range yields UAV sample images, including: Determine the side length of the ground scale range corresponding to the ground-captured photograph; The ratio of the side length of the ground scale range to the ground resolution of the UAV image is used as the scale of the UAV image. The drone image is cropped according to the stated scale to obtain a drone sample image, wherein the straw coverage of the drone sample image is related to the straw coverage within the stated ground scale range.

2. The method for detecting straw coverage by cross-scale multi-source data fusion according to claim 1, characterized in that, The step involves calculating the image similarity between the ground-captured photographs corresponding to each of the multiple UAV image blocks and the UAV sample images to obtain the overall similarity of each UAV image block, including: For each of the plurality of drone image blocks, the following processing is performed: Determine a first structural similarity index between the UAV image block and the corresponding ground photograph; Determine a second structural similarity index between the UAV image block and the corresponding UAV sample image; The overall similarity of the UAV image blocks is obtained by weighted summation of the first structural similarity index and the second structural similarity index.

3. The method for detecting straw coverage by cross-scale multi-source data fusion according to claim 1, characterized in that, The process involves selecting a fixed proportion of target UAV image blocks to generate new samples, resulting in a UAV image straw coverage sample set, including: Determine the straw coverage of the drone sample image that has the highest overall similarity to the target drone image block; The straw coverage is calculated by multiplying the straw coverage by the overall similarity of the target UAV image block, thus obtaining the straw coverage sample set of the UAV image straw coverage.

4. The method for detecting straw coverage by cross-scale multi-source data fusion according to claim 3, characterized in that, The determination of the straw coverage of the drone sample image with the highest overall similarity to the target drone image block includes: Determine a first structural similarity index between the similarity of the target UAV image blocks and each ground-captured photograph; Determine a second structural similarity index between the similarity of the target UAV image block and the UAV sample image corresponding to each ground-captured photo; The overall similarity of each UAV sample image corresponding to the target UAV image block is obtained by weighted summation of the first structural similarity index and the second structural similarity index. The straw coverage of the drone sample image with the highest overall similarity is taken as the straw coverage of the target drone image block.

5. The method for detecting straw coverage by cross-scale multi-source data fusion according to claim 1, characterized in that, The construction of a straw coverage regression model based on the Transformer model, based on the set of straw coverage samples from UAV imagery, includes: Based on the linear relationship between the spatial resolution of the satellite remote sensing image and the spatial resolution of the UAV image, the straw coverage sample set of the UAV image is aggregated to obtain a straw coverage sample set at the satellite image scale. Based on the straw coverage sample set at the scale of the satellite image, a straw coverage regression model based on the Transformer model is constructed, using satellite image reflectance and spatial features as inputs and straw coverage as the regression object.

6. A straw cover detection device based on cross-scale multi-source data fusion, employing the straw cover detection method based on cross-scale multi-source data fusion as described in claim 1, characterized in that... include: The acquisition module is used to acquire satellite remote sensing images, UAV images, and ground-based photographs of the target area. The segmentation module is used to input the ground-captured photos into a large visual model for semantic segmentation, and to obtain the straw coverage of the ground-scale range corresponding to the ground-captured photos output by the large visual model. An adaptation module is used to adapt the UAV imagery to the scale range based on the straw coverage within the ground scale range to obtain UAV sample images. The segmentation module is used to segment the UAV sample image according to a preset rule grid to obtain multiple UAV image blocks; The similarity module is used to calculate the image similarity between the ground-captured photos corresponding to each of the multiple UAV image blocks and the UAV sample images, so as to obtain the overall similarity of each UAV image block. The generation module is used to sort each UAV image block according to the overall similarity and select a fixed proportion of target UAV image blocks to generate new samples, thereby obtaining a UAV image straw coverage sample set. The construction module is used to construct a straw coverage regression model based on the Transformer model based on the set of straw coverage samples from UAV images; The extraction module is used to input the satellite remote sensing image into the straw coverage regression model to obtain the straw coverage extraction result of the target area output by the straw coverage regression model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the straw coverage detection method based on cross-scale multi-source data fusion as described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the straw coverage detection method based on cross-scale multi-source data fusion as described in any one of claims 1 to 5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the straw coverage detection method based on cross-scale multi-source data fusion as described in any one of claims 1 to 5.

Images