Method and system for extracting winter wheat planting areas from remote sensing images based on collaborative large and small models

Abstract

The present invention discloses a method and system for extracting winter wheat planting areas from remote sensing images with large and small models in coordination, and belongs to the field of crop recognition from remote sensing images. The method of the present invention comprises: extracting spectral reflectance information of remote sensing image pixels and calculating vegetation index; training a small winter wheat pixel-level recognition model based on depth vector data description; after crop distribution estimation, screening the crop distribution estimation results according to the prediction score; using the nearest neighbor index as a spatial distribution aggregation discrimination index, and adopting a corresponding large and small model coordination strategy according to the judgment result of crop spatial distribution aggregation, so as to extract the boundary information of the winter wheat planting area. Based on the idea of ​​large and small model coordination, the present invention integrates the advantages of the small winter wheat pixel-level recognition model and the large image segmentation model in the extraction task, and can improve the spatial continuity of the recognition results while improving the crop recognition accuracy, and can provide support for large-scale agricultural remote sensing monitoring tasks.

Description

Technical Field

[0001] The present invention belongs to the field of crop recognition using remote sensing images, and in particular relates to a method and system for extracting winter wheat planting areas from remote sensing images using large and small models in coordination. Background Art

[0002] For a long time, global agricultural development has been threatened by the reduction of arable land resources, water shortages, and frequent extreme climates. Timely and accurate acquisition of large-scale crop planting information is an important basis for agricultural management decisions and a necessary means for global food security and sustainable development. Remote sensing is a non-contact, long-distance detection technology with the advantages of wide coverage, short revisit cycle, and strong timeliness. It is widely used in many types of earth observation tasks, especially for the rapid and accurate acquisition of large-scale crop information. It is a key technical means for accurate monitoring of planted crops.

[0003] Deep learning technology has become a research hotspot in the field of remote sensing crop identification due to its significant advantages in mining and utilizing image features. However, the existing remote sensing crop identification models based on deep learning rely on massive multi-label sample data. Even if the identification target is single, it still needs the assistance of other ground object label data. Limited by the differentiated ground object coverage types and spatial distribution in the study area, a large amount of label data needs to be collected to train and optimize the model, and the training cost is high. There are large differences in the spatial morphology and image texture of crop planting areas in different study areas, and the generalization ability of the model trained with data from a single region is limited. Therefore, in order to make full use of the information of high-resolution remote sensing images, efforts should be made to solve the problems of strong sample dependence, high training cost, and weak generalization ability in the existing recognition models based on deep learning, which are prone to low extraction efficiency and accuracy. Summary of the invention

[0004] The purpose of the present invention is to solve the problems existing in the prior art and to provide a method and system for extracting winter wheat planting areas from remote sensing images with coordinated large and small models, so as to improve the extraction efficiency and accuracy of the boundaries of crop planting areas.

[0005] In order to achieve the above-mentioned invention object, the present invention specifically adopts the following technical solutions:

[0006] In a first aspect, the present invention provides a method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration, comprising the following steps:

[0007] S1: Obtain the original remote sensing image and planting information survey data, perform preliminary processing on the original remote sensing image, crop the preliminarily processed remote sensing image according to the range of the area to be extracted, extract the spectral reflectance information of the remote sensing image pixel in the cropped remote sensing image, and calculate the normalized vegetation index and enhanced vegetation index based on the corresponding band information;

[0008] S2: Projecting the cropped remote sensing image and the planting information survey data to the same geographic coordinate system and resampling them to the same resolution to obtain the preprocessed remote sensing image and the preprocessed planting information survey data;

[0009] S3: replacing the kernel function in the deep vector data description method with a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat, using the preprocessed remote sensing image and the normalized vegetation index and enhanced vegetation index in S1 as training samples, and using the preprocessed planting information survey data as reference true value labels, and training the small model for pixel-level recognition of winter wheat based on the training samples and the reference true value labels;

[0010] S4: Input the remote sensing image of the target extraction area into the trained winter wheat pixel-level recognition model to estimate the crop distribution, take the prediction score corresponding to each remote sensing image pixel in the target extraction area as the crop distribution estimation result, filter the crop distribution estimation results according to the prediction score, and obtain a point data set in which all the crop distribution estimation results are winter wheat;

[0011] S5: Using the nearest neighbor index as the spatial distribution clustering discrimination index, the crop spatial distribution clustering is judged for the point data set obtained in S4. According to the judgment result of crop spatial distribution clustering, the corresponding large and small model collaborative strategy is adopted to extract the boundary information of the winter wheat planting area.

[0012] Based on the above scheme, each step can be implemented in the following preferred specific manner.

[0013] As a preferred embodiment of the above-mentioned first aspect, in step S1, the specific process of the preliminary processing is: obtaining the corresponding original remote sensing image according to the area to be extracted and the planting time of winter wheat, performing radiation correction, geometric correction, atmospheric correction and image fusion on the original remote sensing image, and obtaining the remote sensing image after preliminary processing.

[0014] As a preferred embodiment of the above-mentioned first aspect, in step S3, the fully connected neural network is composed of an input layer, three hidden layers with activation functions and an output layer cascaded in sequence, the number of nodes in the output layer is consistent with the number of nodes in the input layer, the input layer takes the band information of the positive pixel and the feature vector composed of the vegetation index as input, and the number of nodes in the input layer is the feature vector dimension of the positive pixel.

[0015] As a preferred embodiment of the first aspect, in step S3, the loss function Loss used in the training of the winter wheat pixel-level recognition model is as follows:

[0016]

[0017]

[0018]

[0019] Where R is the radius of the hypersphere; v is the weight factor; mean is the average value; scores is the prediction score output by the small model of winter wheat pixel-level recognition; dist is the distance from the target point to the center of the hypersphere; x is the distance from the target point to the center of the hypersphere; i represents the output vector of the small model for winter wheat pixel-level recognition; N represents the number of target points; c is the center vector of the hypersphere.

[0020] As a preferred embodiment of the first aspect, in step S4, the specific process of screening the crop distribution estimation results according to the prediction scores is as follows: when the radius of the hypersphere is set to 1, the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores in the interval (-1,0] are represented as winter wheat, and the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores outside the interval (-1,0] are represented as non-winter wheat, and the point data set is generated by the remote sensing image pixels with crop distribution estimation results of winter wheat.

[0021] As a preferred embodiment of the above-mentioned first aspect, in step S5, if the judgment result of the clustering of crop spatial distribution is that the crop distribution has clustering characteristics, the observed average distance is used as a guide for the spatially constrained multivariate clustering method. After obtaining the spatial clustering results of the point data set, a fuzzy position hint of the winter wheat planting area is generated based on the cluster center and the sampling points within the class. The SAM-based image segmentation large model is used to extract the boundary information of the winter wheat planting area under the constraint of the fuzzy position hint to complete the winter wheat planting area extraction task.

[0022] As a preferred embodiment of the above-mentioned first aspect, the specific process of extracting the boundary information of the winter wheat planting area by the SAM-based image segmentation model under the constraint of the fuzzy position hint is: the remote sensing image of the target extraction area is input into the image segmentation model in the form of a sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the fuzzy position hint of the winter wheat planting area is input into the image segmentation model, the hint encoder of the image segmentation model converts the fuzzy position hint into a feature representation matching the high-dimensional feature representation, and the mask decoder of the image segmentation model combines the two feature representations to generate a segmentation mask that is closely aligned with the edge of the target crop field, and the segmentation mask is used as the boundary information of the winter wheat planting area.

[0023] As a preferred embodiment of the above-mentioned first aspect, in step S5, if the judgment result of the clustering of crop spatial distribution is that the crop distribution does not have clustering characteristics, the spatial constraint information of the winter wheat planting area without semantic information is generated by the SAM-based image segmentation model, and for each plot in the spatial constraint information, the proportion of remote sensing image pixels in each plot whose crop distribution estimation result is winter wheat is counted; if the proportion of remote sensing image pixels is higher than a preset threshold, the plot is considered to be a winter wheat planting area, and all winter wheat planting areas are summarized to obtain the boundary information of the winter wheat planting area, thereby completing the winter wheat planting area extraction task.

[0024] As a preferred embodiment of the above-mentioned first aspect, the specific process of generating spatial constraint information of winter wheat planting areas without semantic information by the SAM-based image segmentation large model is: the remote sensing image of the target extraction area is input into the image segmentation large model in the form of a sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation large model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the automatic mask generator generates a full coverage point set as prompt information in a spatial sliding window manner; the prompt encoder of the image segmentation large model processes the prompt information and converts it into a feature representation matching the high-dimensional feature representation; the mask decoder of the image segmentation large model combines the two feature representations to generate a global semantic-free segmentation mask, and uses the segmentation mask as the spatial constraint information.

[0025] In a second aspect, the present invention provides a remote sensing image winter wheat planting area extraction system with large and small models coordinated, comprising:

[0026] The first processing module is used to obtain the original remote sensing image and the planting information survey data, and after the original remote sensing image is preliminarily processed, the preliminarily processed remote sensing image is cropped according to the range of the area to be extracted, and the remote sensing image pixels in the cropped remote sensing image are taken as the object to extract the spectral reflectance information of the remote sensing image pixels, and the normalized vegetation index and the enhanced vegetation index are calculated based on the corresponding band information;

[0027] The second processing module is used to project the cropped remote sensing image and the planting information survey data to the same geographic coordinate system and resample them to the same resolution to obtain the preprocessed remote sensing image and the preprocessed planting information survey data;

[0028] A model optimization module is used to replace the kernel function in the deep vector data description method with a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat, use the preprocessed remote sensing image and the normalized vegetation index and the enhanced vegetation index as training samples, use the preprocessed planting information survey data as a control true value label, and train the small model for pixel-level recognition of winter wheat based on the training samples and the control true value label;

[0029] The crop distribution estimation and screening module is used to input the remote sensing image of the target extraction area into the trained winter wheat pixel-level recognition model to estimate the crop distribution, and use the prediction score corresponding to each remote sensing image pixel in the target extraction area as the crop distribution estimation result. The crop distribution estimation results are screened according to the prediction score to obtain a point data set in which all the crop distribution estimation results are winter wheat.

[0030] The result acquisition module is used to use the nearest neighbor index as the spatial distribution aggregation discrimination index to judge the crop spatial distribution aggregation of the obtained point data set, and adopt the corresponding large and small model collaborative strategy according to the judgment result of the crop spatial distribution aggregation to extract the boundary information of the winter wheat planting area.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. The present invention provides a small model for pixel-level recognition of winter wheat based on deep vector data description, which utilizes the natural advantages of fully connected neural networks in extracting high-dimensional data features, replaces traditional kernel functions with fully connected neural networks, realizes the projection of pixel spectral information from the original space to the feature space, and then obtains the binary discrimination constraint of the positive pixel features based on the hypersphere model, and realizes remote sensing crop recognition relying only on single-label samples. The characteristic of only requiring single-label sample data alleviates the dependence of the small model for pixel-level recognition of winter wheat on sample diversity to a certain extent. The small model for pixel-level recognition of winter wheat designed by the present invention only requires the characteristics of positive samples, that is, the label data of winter wheat planting areas, which greatly reduces its dependence on label data. In summary, the small model for pixel-level recognition of winter wheat proposed by the present invention has the advantages of low sample dependence, short training time, and high recognition accuracy.

[0033] 2. The present invention provides a method for extracting winter wheat planting areas from remote sensing images with large and small models in coordination. The method is based on the idea of ​​large and small model coordination, and different large and small model coordination strategies are designed for the clustering differences in crop spatial distribution, thereby achieving the complementary advantages of large and small models. The small winter wheat pixel-level recognition model described based on depth vector data has the advantages of simple structure, low sample dependence, fast training time and high recognition accuracy, but it also faces the problems of poor continuity of crop recognition results, high degree of spatial distribution fragmentation and insufficient spatial generalization ability. The large image segmentation model based on SAM has the ability to accurately extract the boundary features of the planting area, but its mask segmentation ability is based on a good spatial prompt, so if it is directly segmented without accurate prompts, the obtained mask data will lack semantic information and cannot extract the planting area boundary of the specified crop. Therefore, the present invention achieves the complementary advantages of large and small models, makes up for the shortcomings of the fragmentation of the small winter wheat pixel-level recognition model results and the lack of semantic information of the large image segmentation model, and not only ensures the recognition accuracy, but also improves the spatial continuity of the recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a structural block diagram of the method of the present invention;

[0035] Figure 2 This is a graph showing the results of the winter wheat pixel-level recognition model of the present invention identifying the Sentinel-2 remote sensing images of study area A and study area B in April 2020;

[0036] Figure 3 The present invention generates a distribution map of fuzzy location prompts of winter wheat planting areas based on cluster centers and sampling points within the cluster;

[0037] Figure 4 Generate a spatial constraint information result map without semantic information for the SAM-based image segmentation model of the present invention;

[0038] Figure 5 This is a result diagram of the boundary information extraction of winter wheat planting areas in study area A and study area B in the present invention;

[0039] Figure 6 is a system block diagram of the present invention;

[0040] Figure 7 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purpose, features and advantages of the present invention more obvious and easy to understand, the specific implementation mode of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in each embodiment of the present invention can be combined accordingly without conflicting with each other.

[0042] In the description of the present invention, it should be understood that the terms "first" and "second" are only used for the purpose of distinguishing descriptions, and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features.

[0043] like Figure 7 As shown, in a preferred implementation of the present invention, the above-mentioned large and small model coordinated remote sensing image winter wheat planting area extraction method includes the following steps S1 to S5. The specific implementation process is described in detail below.

[0044] 1. Extract spectral reflectance information of remote sensing image pixels and calculate vegetation index

[0045] S1: Obtain the original remote sensing image and planting information survey data, perform preliminary processing on the original remote sensing image, crop the preliminarily processed remote sensing image according to the scope of the area to be extracted, take the remote sensing image pixels in the cropped remote sensing image as the object, extract the spectral reflectance information of the remote sensing image pixels, and calculate the normalized vegetation index and enhanced vegetation index based on the corresponding band information.

[0046] It should be noted that in step S1 of the present invention, there are many sources of planting information survey data, which can be obtained from data published by government departments such as the Statistics Bureau, or from papers or public data sets, which is not limited in this embodiment.

[0047] It should be noted that in step S1 of the present invention, the specific process of the above-mentioned preliminary processing is: obtaining the corresponding original remote sensing image according to the area to be extracted and the planting time of winter wheat, performing radiation correction, geometric correction, atmospheric correction and image fusion on the original remote sensing image, and obtaining the remote sensing image after preliminary processing.

[0048] In step S1 of the present embodiment, first, the corresponding original remote sensing image is obtained according to the area to be extracted and the planting time of winter wheat. The study area A and the study area B selected in this embodiment should avoid areas with large terrain undulations and densely populated cities. The growth cycle of winter wheat is from the autumn of the current year to the summer of the following year. Among them, the heading period with the most significant reflectivity characteristics of winter wheat is mainly distributed in April. The original remote sensing image corresponding to the study area in April is obtained as the research data, and the Sentinel-2 remote sensing image in April 2020 is further selected from the original remote sensing image as the target remote sensing image. After the acquired original remote sensing image is subjected to radiation correction, geometric correction, atmospheric correction and image fusion on Google Earth Engine, it is cropped according to the scope of the study area. The cropped remote sensing image is used to calculate the normalized vegetation index (NDVI) and the enhanced vegetation index (EVI) as auxiliary identification features. Among them, the calculation methods of the normalized vegetation index and the enhanced vegetation index belong to the common knowledge in the field. In order to facilitate the technical personnel in this field to understand the present invention, the specific calculation methods of the normalized vegetation index and the enhanced vegetation index will be introduced below.

[0049] The Normalized Difference Vegetation Index (NDVI) is the most widely used vegetation index in the study of vegetation changes. Its basic principle is that the spectral information of vegetation shows strong reflection of the near-infrared band and obvious absorption of the red light band. Therefore, changes in vegetation coverage will cause differences in reflection of red light and near-infrared light. The corresponding vegetation index can be obtained by combining and calculating the reflectivity of vegetation to the near-infrared band and the red light band. It is extremely sensitive to changes in surface vegetation. The calculation formula is as follows:

[0050]

[0051] In the formula, RED represents the reflectivity of the red light band, and NIR represents the reflectivity of the near infrared band. Taking the Sentinel-2 remote sensing image corresponding to this embodiment as an example, RED corresponds to the B4 band information of the Sentinel-2 remote sensing image, and NIR corresponds to the B8 band information of the Sentinel-2 remote sensing image.

[0052] The enhanced vegetation index (EVI) inherits the characteristics of the normalized vegetation index (NDVI), and takes into account the influence of the atmosphere and soil, which improves the defects of the normalized vegetation index (NDVI) to a certain extent. The calculation formula is as follows:

[0053]

[0054] Wherein, G is a gain factor, which is used to amplify the result of the index so that the value of the enhanced vegetation index is easier to interpret and compare. In this embodiment, G is set to 2.5; C1 and C2 are both atmospheric correction factors, which are used to offset the influence of atmospheric scattering on imaging, and help to improve the stability of the enhanced vegetation index in different regions and at different times. In this embodiment, the atmospheric correction factor C1 is set to 6 and C2 is set to 7.5; BLUE represents the reflectivity of the blue light band, which corresponds to the B2 band information of the Sentinel-2 remote sensing image; L is a ground reflection adjustment factor, which is used to reduce the influence of surface features such as soil type and surface humidity on imaging, and help to improve the reliability and applicability of the enhanced vegetation index. In this embodiment, it is set to 1.

[0055] S2: After the cropped remote sensing image and the planting information survey data are projected to the same geographic coordinate system and resampled to the same resolution, the preprocessed remote sensing image and the preprocessed planting information survey data are obtained.

[0056] In step S2 of the present embodiment, the cropped remote sensing image and the planting information survey data are projected to the WGS84 geographic coordinate system in ArcGIS software, and the remote sensing image and the planting information survey data in the same geographic coordinate system are resampled to a resolution of 20 m.

[0057] 2. Training a small model for winter wheat pixel-level recognition based on depth vector data description

[0058] S3: The kernel function in the deep vector data description method is replaced by a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat. The preprocessed remote sensing image and the normalized vegetation index NDVI and enhanced vegetation index EVI in S1 are used as training samples, and the preprocessed planting information survey data are used as reference true value labels. The small model for pixel-level recognition of winter wheat is trained based on the training samples and the reference true value labels.

[0059] In this embodiment, when training the small model for pixel-level recognition of winter wheat, Band 2 to Band 8, Band 11, Band 12 of the remote sensing image preprocessed in S1 and the calculated normalized vegetation index NDVI and enhanced vegetation index EVI are used as training samples, and the preprocessed planting information survey data in S2 is used as the reference true value label. Based on the above training samples and reference true value labels, the small model for pixel-level recognition of winter wheat is trained.

[0060] In step S3 of the present invention, a fully connected neural network is used to replace the kernel function in the deep vector data description method, and it is used as a medium for converting pixel information from the original space to the hypersphere space. A fully connected neural network is one of the most basic network types for deep learning, which is mainly composed of an input layer, a hidden layer, and an output layer.

[0061] In this embodiment, the fully connected neural network is composed of an input layer, three hidden layers with activation functions, and an output layer, which are cascaded in sequence, and the number of nodes in the output layer is consistent with the number of nodes in the input layer. The components of the fully connected neural network are as follows:

[0062] 1) Input layer: The feature vector 𝑃=(𝑝1,𝑝2,𝑝3,…,𝑝𝑛) composed of the band information of the positive pixel and the vegetation index is used as input for subsequent hidden layer training and learning. The number of nodes in the input layer is the dimension of the feature vector of the positive pixel. Among them, 𝑝1,𝑝2,𝑝3,…,𝑝𝑛 represent different feature information of the positive pixel respectively.

[0063] In this embodiment, the number of nodes in the input layer is 11.

[0064] 2) Hidden layer: The fully connected neural network consists of three hidden layers. The feature variable set in the training sample is interconnected in the neurons of the hidden layer and processed by the ReLU activation function. The ReLU activation function passes the newly obtained features to the next layer.

[0065] The ReLU activation function in the hidden layer can effectively prevent the fully connected neural network from overfitting and speed up the learning speed of the fully connected neural network. It is defined as , where Represents the independent variable of the activation function.

[0066] 3) Output layer: The number of nodes in the output layer is consistent with the number of nodes in the input layer to obtain the distance from the positive pixel to the center of the hypersphere.

[0067] In step S3 of the present invention, the loss function of the winter wheat pixel-level recognition model is constructed according to the distance from the target point to the center of the hypersphere, and the remote sensing image pixels beyond the radius of the hypersphere are regarded as abnormal. The calculation formula of the above loss function Loss is:

[0068]

[0069]

[0070]

[0071] Where R is the radius of the hypersphere; v is the weight factor used to amplify the distance score; mean is the average value; scores is the predicted score output by the small model of winter wheat pixel-level recognition; dist is the distance from the target point to the center of the hypersphere; x is the distance between the target point and the center of the hypersphere; i represents the output vector of the small model for winter wheat pixel-level recognition; N represents the number of target points; c is the center vector of the hypersphere.

[0072] 3. Estimation of winter wheat crop distribution based on a small winter wheat pixel-level recognition model

[0073] S4: The remote sensing image of the target extraction area is input into the trained winter wheat pixel-level recognition model to estimate the crop distribution. The prediction score corresponding to each remote sensing image pixel in the target extraction area is used as the crop distribution estimation result. The crop distribution estimation results are screened according to the prediction score to obtain a point data set in which all the crop distribution recognition results are winter wheat.

[0074] It should be noted that in step S4 of the present invention, the specific process of screening the crop distribution estimation results according to the prediction scores is as follows: when the radius of the hypersphere is set to 1, the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores in the interval (-1,0] are represented as winter wheat, and the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores outside the interval (-1,0] are represented as non-winter wheat, and the above-mentioned point data set is generated by the remote sensing image pixels whose crop distribution estimation results are winter wheat.

[0075] In step S4 of this embodiment, the remote sensing image of the target extraction area is first input into the trained winter wheat pixel-level recognition model, and the trained winter wheat pixel-level recognition model is used to predict the remote sensing images of study area A and study area B, and the prediction score corresponding to each remote sensing image pixel in the entire study area is calculated, and it is used as the crop distribution recognition result. The crop distribution recognition results are then screened to create a point data set in which all crop distribution recognition results are winter wheat. In this embodiment, because the radius of the hypersphere is set to 1, when the prediction score of the remote sensing image pixel by the winter wheat pixel-level recognition model is less than or equal to 0 and greater than or equal to -1, it means that the crop distribution recognition result is winter wheat. According to this rule, the distribution recognition results can be screened, and the screening results are as follows. Figure 2 shown.

[0076] 4. Determine the clustering of winter wheat crop distribution and obtain boundary information of winter wheat planting areas

[0077] S5: Figure 1 As shown in the figure, the nearest neighbor index is used as the spatial distribution clustering discrimination index to judge the crop spatial distribution clustering of the point data set obtained by S4. According to the judgment result of the crop spatial distribution clustering, the corresponding large and small model collaborative strategy is adopted to extract the boundary information of the winter wheat planting area.

[0078] It should be noted that in step S5 of the present invention, the nearest neighbor index is a commonly used indicator for measuring the distribution pattern of spatial points. It describes the degree of aggregation of the spatial point set based on the difference between the average distance between the spatial point and its nearest neighbor point under the current distribution and the expected distance of the point under the random distribution. Among them, if the nearest neighbor index is less than 1, the distribution of the spatial point set tends to be aggregated; if the nearest neighbor index is greater than 1, it does not have the aggregation characteristics. The calculation formula of the nearest neighbor index NNI is as follows:

[0079]

[0080]

[0081]

[0082] in, is the observed average distance, i.e., the average distance from each point in the spatial point set to its nearest point; E(d) is the expected average distance, i.e., the average nearest neighbor distance under completely random conditions, which is determined by the number of points M and the coverage area A of the point set; d m Represents the distance from the mth point in the spatial point set to its nearest point.

[0083] In this embodiment, ArcGIS analysis tools are used to calculate the nearest neighbor index. Figure 2 In the data, the observed average distance of the point data set corresponding to the crop distribution estimation results in study area A is 13.80, the expected average distance is 13.96, and its nearest neighbor index is 0.98, which has clustering characteristics; the observed average distance of the point data set corresponding to the crop distribution estimation results in study area B is 13.71, the expected average distance is 12.93, and its nearest neighbor index is 1.06, which does not have clustering characteristics.

[0084] It should be noted that in step S5 of the present invention, if the judgment result of the clustering of crop spatial distribution is that the crop distribution has clustering characteristics, the observed average distance is used as a guide for the spatially constrained multivariate clustering method. After obtaining the spatial clustering result of the point data set, a fuzzy position prompt of the winter wheat planting area is generated based on the cluster center and the sampling points within the class. The SAM-based image segmentation model is used to extract the boundary information of the winter wheat planting area under the constraint of the fuzzy position prompt to complete the winter wheat planting area extraction task.

[0085] In step S5 of this embodiment, if the result of the judgment of crop spatial distribution aggregation is that the crop distribution has aggregation characteristics, the prompts required for the image segmentation model are generated based on the crop distribution estimation results. The observed average distance (i.e., the average distance from each point in the spatial point set to its nearest point) is used as a guide for the spatially constrained multivariate clustering method. Then, the spatial clustering results of the point data set are obtained using ArcGIS's spatially constrained multivariate clustering tool, and the fuzzy location prompts of the winter wheat planting area are generated based on the cluster center, such as Figure 3 As shown, the number of sampling points within a class is increased to avoid missing crop-growing areas.

[0086] The above-mentioned spatially constrained multivariate clustering method (Spatially Constrained MultivariateClustering) is a method that considers spatial location information in the data clustering process. It is particularly suitable for processing data sets with obvious clustering characteristics in spatial distribution. The formula is expressed as follows:

[0087]

[0088] Among them, C represents the set of clusters; K represents the total number of clusters; represents the kth cluster; As a weight coefficient, it balances the influence of attribute distance and spatial distance; Represents the distance measure on the attributes; Represents the attribute value of the i-th element; Representation clustering Center on the property; Represents a distance measure in space; Represents the spatial value of the i-th element; Representation clustering The center in space.

[0089] It should be noted that, in the present invention, the specific process of extracting the boundary information of the winter wheat planting area under the constraint of the fuzzy position prompt by the SAM-based image segmentation model is as follows: the remote sensing image of the target extraction area is input into the image segmentation model in the form of a 256×256 sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the fuzzy position prompt of the winter wheat planting area is input into the image segmentation model, the prompt encoder of the image segmentation model converts the fuzzy position prompt into a feature representation matching the high-dimensional feature representation, and the mask decoder of the image segmentation model combines the two feature representations to generate a segmentation mask that is closely aligned with the edge of the target crop field, and the segmentation mask is used as the boundary information of the winter wheat planting area.

[0090] In this embodiment, compared with the crop distribution estimation result based only on the winter wheat pixel-level recognition model ( Figure 2 ),like Figure 5 As shown in Figure 1, the extraction results of study area A have clear crop planting area boundary information, which greatly improves the fragmentation problem in large winter wheat planting areas. Based on the planting information survey data as the true value control, the statistics are as follows: Figure 5The winter wheat crop recognition accuracy of all remote sensing image pixels within the study area A is shown as follows: accuracy is 93.83%, precision is 86.36%, and recall is 83.54%.

[0091] It should be noted that in step S5 of the present invention, if the judgment result of the clustering of crop spatial distribution is that the crop distribution does not have clustering characteristics, the spatial constraint information of the winter wheat planting area without semantic information is generated by the SAM-based image segmentation model, and for each plot in the spatial constraint information, the proportion of remote sensing image pixels in each plot whose crop distribution estimation result is winter wheat is counted. If the proportion of remote sensing image pixels is higher than the preset threshold, the plot is considered to be a winter wheat planting area, and all winter wheat planting areas are summarized to obtain the boundary information of the winter wheat planting area, thereby completing the winter wheat planting area extraction task.

[0092] It should be noted that, in the present invention, the specific process of generating the spatial constraint information of the winter wheat planting area without semantic information by the SAM-based image segmentation model is as follows: the remote sensing image of the target extraction area is input into the image segmentation model in the form of a 256×256 sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the automatic mask generator generates a full coverage point set as prompt information in a spatial sliding window manner; the prompt encoder of the image segmentation model processes the prompt information and converts it into a feature representation matching the high-dimensional feature representation; the mask decoder of the image segmentation model combines the two feature representations to generate a global semantic-free segmentation mask (i.e., the global plot boundary information of the unspecified target crop), and uses the segmentation mask as the spatial constraint information.

[0093] In this embodiment, the mask decoder of the above-mentioned image segmentation model generates a global semantic-free segmentation mask of the study area B and uses it as spatial constraint information, such as Figure 4 As shown. Secondly, the proportion of remote sensing image pixels whose crop distribution estimation result is winter wheat in each plot is counted, and 60% is set as the threshold for screening. If the percentage is higher than the threshold, the plot is considered to be a winter wheat planting area. The plots belonging to the winter wheat planting area are summarized to obtain the boundary information of the winter wheat planting area.

[0094] In this embodiment, compared with the crop distribution estimation result based only on the winter wheat pixel-level recognition model ( Figure 2 ),like Figure 5 As shown in Figure 1, the extraction results of study area B have clear crop planting area boundary information, eliminating most of the misjudged pixels outside the winter wheat planting area. Based on the planting information survey data as the true value control, the statistics are as follows: Figure 5The winter wheat crop recognition accuracy of all remote sensing image pixels within the study area B is shown as follows: accuracy is 90.56%, precision is 85.24%, and recall is 87.69%.

[0095] It should also be noted that the large-small model coordinated remote sensing image winter wheat planting area extraction method in the above embodiment can essentially be executed by a computer program or module. Therefore, similarly, based on the same inventive concept, another preferred embodiment of the present invention also provides a large-small model coordinated remote sensing image winter wheat planting area extraction system corresponding to the large-small model coordinated remote sensing image winter wheat planting area extraction method provided in the above embodiment, such as Figure 6 As shown, it includes:

[0096] The first processing module is used to obtain the original remote sensing image and the planting information survey data, and after the original remote sensing image is preliminarily processed, the preliminarily processed remote sensing image is cropped according to the range of the area to be extracted, and the remote sensing image pixels in the cropped remote sensing image are taken as the object to extract the spectral reflectance information of the remote sensing image pixels, and the normalized vegetation index and the enhanced vegetation index are calculated based on the corresponding band information;

[0097] The second processing module is used to project the cropped remote sensing image and the planting information survey data to the same geographic coordinate system and resample them to the same resolution to obtain the preprocessed remote sensing image and the preprocessed planting information survey data;

[0098] A model optimization module is used to replace the kernel function in the deep vector data description method with a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat, use the preprocessed remote sensing image and the normalized vegetation index and the enhanced vegetation index as training samples, use the preprocessed planting information survey data as a control true value label, and train the small model for pixel-level recognition of winter wheat based on the training samples and the control true value label;

[0099] The crop distribution estimation and screening module is used to input the remote sensing image of the target extraction area into the trained winter wheat pixel-level recognition model to estimate the crop distribution, and use the prediction score corresponding to each remote sensing image pixel in the target extraction area as the crop distribution estimation result. The crop distribution estimation results are screened according to the prediction score to obtain a point data set in which all the crop distribution estimation results are winter wheat.

[0100] The result acquisition module is used to use the nearest neighbor index as the spatial distribution aggregation discrimination index to judge the crop spatial distribution aggregation of the obtained point data set, and adopt the corresponding large and small model collaborative strategy according to the judgment result of the crop spatial distribution aggregation to extract the boundary information of the winter wheat planting area.

[0101] The above-described embodiment is only a preferred solution of the present invention, but it is not intended to limit the present invention. A person skilled in the relevant technical field may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent replacement or equivalent transformation falls within the protection scope of the present invention.

Claims

1. A method for extracting winter wheat planting areas from remote sensing images using large and small models, characterized in that: The following steps are involved: S1: Obtain the original remote sensing image and planting information survey data, perform preliminary processing on the original remote sensing image, crop the preliminarily processed remote sensing image according to the range of the area to be extracted, extract the spectral reflectance information of the remote sensing image pixel in the cropped remote sensing image, and calculate the normalized vegetation index and enhanced vegetation index based on the corresponding band information; S2: Projecting the cropped remote sensing image and the planting information survey data to the same geographic coordinate system and resampling them to the same resolution to obtain the preprocessed remote sensing image and the preprocessed planting information survey data; S3: replacing the kernel function in the deep vector data description method with a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat, using the preprocessed remote sensing image and the normalized vegetation index and enhanced vegetation index in S1 as training samples, and using the preprocessed planting information survey data as reference true value labels, and training the small model for pixel-level recognition of winter wheat based on the training samples and the reference true value labels; S4: Input the remote sensing image of the target extraction area into the trained winter wheat pixel-level recognition model to estimate the crop distribution, take the prediction score corresponding to each remote sensing image pixel in the target extraction area as the crop distribution estimation result, filter the crop distribution estimation results according to the prediction score, and obtain a point data set in which all the crop distribution estimation results are winter wheat; S5: Using the nearest neighbor index as the spatial distribution clustering discrimination index, the crop spatial distribution clustering is judged for the point data set obtained in S4. According to the judgment result of crop spatial distribution clustering, the corresponding large and small model collaborative strategy is adopted to extract the boundary information of the winter wheat planting area.

2. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 1, characterized in that: In step S1, the specific process of the preliminary processing is: obtaining the corresponding original remote sensing image according to the area to be extracted and the planting time of winter wheat, performing radiation correction, geometric correction, atmospheric correction and image fusion on the original remote sensing image, and obtaining the remote sensing image after preliminary processing.

3. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 1, characterized in that: In step S3, the fully connected neural network is composed of an input layer, three hidden layers with activation functions and an output layer cascaded in sequence, the number of nodes in the output layer is consistent with the number of nodes in the input layer, the input layer takes the band information of the positive pixel and the feature vector composed of the vegetation index as input, and the number of nodes in the input layer is the feature vector dimension of the positive pixel.

4. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 1, characterized in that: In step S3, the loss function Loss used in the training of the winter wheat pixel-level recognition model is as follows: In the formula, R is the radius of the hypersphere; v is the weight factor; mean represents the average value; represents the prediction score output by the small model for pixel-level recognition of winter wheat; Indicates the distance from the target point to the center of the hypersphere; represents the output vector of the small model for pixel-level recognition of winter wheat; Indicates the number of target points; is the center vector of the hypersphere.

5. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 4, characterized in that: In step S4, the specific process of screening the crop distribution estimation results according to the prediction scores is as follows: when the radius of the hypersphere is set to 1, the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores in the interval (-1,0] are represented as winter wheat, and the crop distribution estimation results corresponding to the remote sensing image pixels with prediction scores outside the interval (-1,0] are represented as non-winter wheat, and the point data set is generated from the remote sensing image pixels with crop distribution estimation results of winter wheat.

6. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 1, characterized in that: In step S5, if the judgment result of the clustering of crop spatial distribution is that the crop distribution has clustering characteristics, the observed average distance is used as a guide for the spatially constrained multivariate clustering method. After obtaining the spatial clustering result of the point data set, a fuzzy position prompt of the winter wheat planting area is generated based on the cluster center and the sampling points within the class. The SAM-based image segmentation large model is used to extract the boundary information of the winter wheat planting area under the constraint of the fuzzy position prompt to complete the winter wheat planting area extraction task.

7. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 6, characterized in that: The specific process of extracting the boundary information of the winter wheat planting area by the SAM-based image segmentation model under the constraint of the fuzzy position hint is as follows: the remote sensing image of the target extraction area is input into the image segmentation model in the form of a sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the fuzzy position hint of the winter wheat planting area is input into the image segmentation model, the hint encoder of the image segmentation model converts the fuzzy position hint into a feature representation matching the high-dimensional feature representation, and the mask decoder of the image segmentation model combines the two feature representations to generate a segmentation mask that is closely aligned with the edge of the target crop field, and the segmentation mask is used as the boundary information of the winter wheat planting area.

8. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 1, characterized in that: In step S5, if the judgment result of the clustering of crop spatial distribution is that the crop distribution does not have clustering characteristics, the spatial constraint information of the winter wheat planting area without semantic information is generated by the SAM-based image segmentation model. For each plot in the spatial constraint information, the proportion of remote sensing image pixels in each plot whose crop distribution estimation result is winter wheat is counted. If the proportion of remote sensing image pixels is higher than the preset threshold, the plot is considered to be a winter wheat planting area. All winter wheat planting areas are summarized to obtain the boundary information of the winter wheat planting area, thereby completing the winter wheat planting area extraction task.

9. The method for extracting winter wheat planting areas from remote sensing images using large and small models in collaboration as claimed in claim 8, characterized in that: The specific process of generating the spatial constraint information of the winter wheat planting area without semantic information by the SAM-based image segmentation model is as follows: the remote sensing image of the target extraction area is input into the image segmentation model in the form of a sliding window, and the sliding step is 50% of the window side length; the image encoder of the image segmentation model converts the remote sensing image of the target extraction area into a high-dimensional feature representation; the automatic mask generator generates a full coverage point set as prompt information in a spatial sliding window manner; the prompt encoder of the image segmentation model processes the prompt information and converts it into a feature representation matching the high-dimensional feature representation; the mask decoder of the image segmentation model combines the two feature representations to generate a global semantic-free segmentation mask, and uses the segmentation mask as the spatial constraint information.

10. A remote sensing image winter wheat planting area extraction system with large and small models, characterized in that: include: The first processing module is used to obtain the original remote sensing image and the planting information survey data, and after the original remote sensing image is preliminarily processed, the preliminarily processed remote sensing image is cropped according to the range of the area to be extracted, and the remote sensing image pixels in the cropped remote sensing image are taken as the object to extract the spectral reflectance information of the remote sensing image pixels, and the normalized vegetation index and the enhanced vegetation index are calculated based on the corresponding band information; The second processing module is used to project the cropped remote sensing image and the planting information survey data to the same geographic coordinate system and resample them to the same resolution to obtain the preprocessed remote sensing image and the preprocessed planting information survey data; A model optimization module is used to replace the kernel function in the deep vector data description method with a fully connected neural network to obtain a small model for pixel-level recognition of winter wheat, use the preprocessed remote sensing image and the normalized vegetation index and the enhanced vegetation index as training samples, use the preprocessed planting information survey data as a control true value label, and train the small model for pixel-level recognition of winter wheat based on the training samples and the control true value label; The crop distribution estimation and screening module is used to input the remote sensing image of the target extraction area into the trained winter wheat pixel-level recognition model to estimate the crop distribution, and use the prediction score corresponding to each remote sensing image pixel in the target extraction area as the crop distribution estimation result. The crop distribution estimation results are screened according to the prediction score to obtain a point data set in which all the crop distribution estimation results are winter wheat. The result acquisition module is used to use the nearest neighbor index as the spatial distribution aggregation discrimination index to judge the crop spatial distribution aggregation of the obtained point data set, and adopt the corresponding large and small model collaborative strategy according to the judgment result of the crop spatial distribution aggregation to extract the boundary information of the winter wheat planting area.

Images