Field intelligent extraction and fusion method for high-resolution satellite image

Abstract

The invention relates to the technical field of agricultural remote sensing classification, and provides an intelligent field extraction and fusion method for high-resolution satellite images. The method comprises the following steps: step 1, collecting high-resolution satellite images of different areas and preprocessing the high-resolution satellite images to obtain a high-resolution satellite image data set; 2, constructing a high-resolution image field intelligent extraction and fusion network, and training the high-resolution image field intelligent extraction and fusion network by using the high-resolution satellite image data set to obtain a high-resolution image field intelligent extraction and fusion model; wherein the high-resolution image field intelligent extraction fusion network comprises a cultivated land information intelligent extraction module, a cultivated land boundary multi-task deep learning module and an improved mean shift multi-scale segmentation module. According to the method, layered extraction of cultivated land information extraction, cultivated land block boundary determination and crop field block segmentation is carried out in the cultivated land fragmentation area by fusing multiple remote sensing methods, and the refined extraction requirement of precision agriculture can be met.

Description

Technical Field

[0001] The present invention relates to the technical field of agricultural remote sensing classification, and particularly relates to an intelligent extraction and fusion method for field blocks oriented to high-resolution satellite images. Background Art

[0002] A field block refers to a closed area formed by boundaries resulting from differences in planted crops, crop growth, or planting status. The land within it is planted with only one type of crop, a mixture of crops, or is in a fallow state, and has the characteristic of crop planting consistency, being a more refined cultivated land plot. Different from the general cultivated land plot, the general cultivated land plot refers to a closed farmland within cultivated land surrounded by natural or artificial boundaries that are easy to identify and relatively stable over a period of time, such as a piece of land with independent geographical locations divided by farm roads, relatively wide field ridges, etc. A cultivated land plot may contain one or more field blocks.

[0003] Most of the previous methods for delineating field block boundaries were completed based on medium-resolution images, mainly used in regions or countries with intensive agriculture and large farmlands. The commonly used methods mainly include edge detection, region growth and splitting, and multi-scale segmentation algorithms. However, the applicability of these methods in regions such as developing countries represented by China has great uncertainty. The arable land in these regions mainly consists of small and scattered farmlands, with a high degree of fragmentation of cultivated land, and there are significant differences in the area, shape, and distribution of farmlands in Europe and the Americas.

[0004] At present, the gradual opening of sub-meter remote sensing data and the rapid development of deep learning technology provide good data and technical bases for the extraction of high-precision cultivated land products (Cai Zhiwen et al., 2022). For example, Garcia-Pedrero et al. (2019) used the deep learning semantic segmentation model U-Net to extract farmland plots based on 207 image tiles with a spatial resolution of 0.25 m and a size of 9384×1368 pixels within the territory of Spain. The research results show that the powerful feature learning ability of deep learning can extract richer semantic information with the assistance of high-resolution data, and this research further confirms the advantages of deep learning technology in the extraction of farmland plots. Cheng et al. (2020) tried to determine farmland boundaries from WorldView-2 / 3 with sub-meter spatial resolution and 3-meter Planet satellite images by integrating the spatial and temporal information boundary delineation method (DESTIN), and conducted method tests and result evaluations on four sub-regions in the eastern part of Jiangsu Province, China, so as to prove the great potential of using high-spatial-resolution satellite images and multi-temporal data to delineate farmland boundaries. Cheng Rui et al. (2022) constructed a set of cultivated land plot extraction methods integrating four semantic segmentation models (FCN, PSPNet, SegNet, U-Net) based on WorldView satellite images with high spatial resolution. The experimental results show that integrating multiple deep learning models has higher extraction accuracy than a single convolutional neural network. However, most of the test areas of these studies are mainly in plain areas, and there are few studies on the applicability of these methods in fragmented cultivated land areas such as hilly or mountainous areas. This is attributed to the complex agricultural structure, fragmented plots, and complex spectra and textures presented in the images in hilly and mountainous areas. Using only a single method and data source cannot meet the refined extraction requirements of cultivated land plot boundaries, which brings certain challenges to the development of precision agriculture in the region. Summary of the Invention

[0005] Aiming at the problem that the existing field extraction technology cannot meet the refined extraction requirements of precision agriculture due to the complex agricultural structure, fragmented plots, and complex spectra and textures presented in the images in hilly and mountainous areas, the present invention provides a fusion method for intelligent extraction of field blocks for high-resolution satellite images, which conducts hierarchical extraction from the extraction of cultivated land information to the determination of cultivated land plot boundaries and then to the segmentation of crop field blocks by integrating a variety of remote sensing methods in fragmented cultivated land areas, and can meet the refined extraction requirements of precision agriculture.

[0006] A fusion method for intelligent extraction of field blocks for high-resolution satellite images provided by the present invention includes:

[0007] Step 1: Collect high-resolution satellite images of different regions and perform preprocessing to obtain a high-resolution satellite image dataset;

[0008] Step 2: Construct a high-resolution image field intelligent extraction fusion network, and use the high-resolution satellite image dataset to train the high-resolution image field intelligent extraction fusion network to obtain a high-resolution image field intelligent extraction fusion model;

[0009] Among them, the high-resolution image field intelligent extraction fusion network includes a cultivated land information intelligent extraction module, a cultivated land plot boundary multi-task deep learning module, and an improved mean shift multi-scale segmentation module; the cultivated land information intelligent extraction module is used to extract the cultivated land of the pre-processed high-resolution satellite image to obtain a preliminary cultivated land plot; the cultivated land plot boundary multi-task deep learning module extracts the roads therein based on the preliminary cultivated land plot to obtain independent cultivated land plots; the improved mean shift multi-scale segmentation module is used to mask the independent cultivated land plots and the corresponding high-resolution satellite images to obtain a cultivated land image, and perform multi-scale segmentation on the cultivated land image to obtain a field extraction result.

[0010] Further, in Step 1, the spatial resolution of the high-resolution satellite image dataset is 1m.

[0011] Further, the high-resolution satellite image includes a multi-spectral image and a panchromatic image.

[0012] Further, the pre-processing includes:

[0013] After radiometric calibration, atmospheric correction, and orthorectification of the multi-spectral image, an optimized multi-spectral image is obtained;

[0014] After radiometric calibration and orthorectification of the panchromatic image, an optimized panchromatic image is obtained;

[0015] After image fusion, geometric correction, and mosaicking of the optimized multi-spectral image and the optimized panchromatic image, a pre-processed high-resolution satellite image is obtained.

[0016] Further, the construction and training steps of the cultivated land information intelligent extraction module are as follows:

[0017] Replace the encoding layer of the U-Net model with a pre-trained Resnet34 model to obtain a cultivated land information intelligent extraction module;

[0018] Calculate the derived vegetation indices based on the original bands of the high-resolution satellite image dataset, use the original bands and the vegetation indices as the features of the high-resolution satellite image, and perform feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset; among them, the original bands include blue light, green light, red light, and near-infrared light, and the vegetation indices include: normalized difference vegetation index, normalized difference water index, ratio vegetation index, difference vegetation index, green normalized difference vegetation index, and optimized soil-adjusted vegetation index;

[0019] Train the intelligent cultivated land information extraction module using the optimal feature subset to complete the training of the intelligent cultivated land information extraction module.

[0020] Further, perform feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset, specifically including:

[0021] Calculate the permutation feature importance of all features in the high-resolution satellite image and sort them;

[0022] Calculate the correlation coefficients of all features in the high-resolution satellite image and sort them;

[0023] Perform feature screening according to the sorting results of permutation feature importance and the sorting results of correlation coefficients to obtain the optimal feature subset;

[0024] Among them, the calculation method of the permutation feature importance is as follows:

[0025] Use the high-resolution satellite image dataset as the input data to calculate the original prediction error e of the intelligent cultivated land information extraction module orig :

[0026]

[0027] Among them, X represents the feature matrix, which is the feature set of the high-resolution satellite image, represents the intelligent cultivated land information extraction module, and y represents the target vector;

[0028] Randomly permute the feature a in the feature matrix X to generate a feature permutation matrix X perm , and use the feature permutation matrix X perm as the input feature to calculate the permutation prediction error e of the intelligent cultivated land information extraction module perm :

[0029]

[0030] Among them, X perm represents the feature permutation matrix;

[0031] Based on the results of the original prediction error e orig and the permutation prediction error e perm to obtain the permutation feature importance of feature a, and the permutation feature importance calculation formula is as follows:

[0032] FI a = e perm / e orig

[0033] Among them, FI aIndicates the permutation feature importance of feature a;

[0034] The correlation coefficient is calculated by the following formula:

[0035]

[0036] Where r represents the correlation coefficient, cov(a,b) represents the covariance of features a and b, and σ a and σ b respectively represent the standard deviations of features a and b; a i and b i respectively represent the i-th eigenvalue of features a and b, and respectively represent the means of features a and b, and n is the number of eigenvalues of a and b.

[0037] Furthermore, the cultivated land plot boundary multi-task deep learning module includes a shared encoder module, multiple iterative multi-branch fusion modules, and a stacked multi-branch prediction module; the shared encoder module and the iterative multi-branch fusion module are connected by residual block skip connections, and the output of the iterative multi-branch fusion module is connected to the input of the stacked multi-branch prediction module;

[0038] Among them, the shared encoder module is used to extract multi-scale features common to different tasks; the iterative multi-branch fusion module is used to learn the multi-scale features extracted by the shared encoder module, predict the road direction and road pixels to construct an intermediate prediction; the stacked multi-branch prediction module is used to fuse the intermediate prediction to realize information interaction based on the direction learning task and the road segmentation task, so as to complete the extraction of the road.

[0039] Furthermore, the processing process of the improved mean shift multi-scale segmentation module is as follows:

[0040] Based on the independent cultivated land plots, the corresponding high-resolution satellite images are masked to obtain cultivated land images;

[0041] Based on the elevation, the cultivated land images are divided into geomorphic zones, and the geomorphic zone division results are segmented based on the strip-shaped ground object data to obtain irregular image blocks;

[0042] The optimal segmentation scale is determined by calculating the mean variance of the irregular image blocks in different geomorphic regions;

[0043] All the irregular image blocks are segmented using the multi-scale segmentation algorithm corresponding to the optimal segmentation scale, and the segmentation results of all the irregular image blocks are merged to obtain the final cultivated land plot extraction result.

[0044] Furthermore, the determination of the optimal segmentation scale by calculating the mean variance of the irregular image blocks in different geomorphic regions specifically includes:

[0045] Arbitrarily select an irregular image block from the irregular image blocks included in each geomorphic division, calculate the mean variance at different segmentation scales, analyze the trend of the mean variance changing from small to large with the segmentation scale, and when the turning point where the mean variance shows a change from large to small occurs, the corresponding segmentation scale is determined as the optimal segmentation scale;

[0046] Among them, the mean variance is calculated by the following formula:

[0047]

[0048]

[0049] Among them, represents the brightness value of the i-th pixel within the object in the T-th band, n represents the number of pixels in the object, and D T represents the brightness mean of a single image object in the T-th band; represents the brightness mean of all objects in the image in the T-th band, m represents the total number of objects in the image, and S 2 represents the mean variance.

[0050] The beneficial effects of the present invention are as follows:

[0051] (1) Based on the original U-Net model, the present invention uses the pre-trained model ResNet to replace the encoding layer for feature extraction, and expands the flexibility of the data channel number and bit depth at the input end of the model, no longer being limited to the limitations of the three RGB channels and 8-bit depth; and the present invention introduces a feature selection method, which fully exerts the ability of vegetation index features to extract cultivated land information in complex geomorphic areas. This module is not only applicable to the high-precision extraction of plain cultivated land, but also applicable to the scattered and fragmented cultivated land plots in hilly areas.

[0052] (2) The present invention improves the mean shift multi-scale segmentation module based on the fusion of geomorphic feature partitioning and the mean variance method, solves the seam problem that often occurs when splicing the results due to the block processing of high-resolution remote sensing images, improves the efficiency of determining the optimal segmentation scale, and avoids the process of frequently trying wrong segmentation parameters. That is, on the basis of the mean shift multi-scale segmentation algorithm, the zoning and segmentation of images are realized by means of linear features and geomorphic information, and the optimal segmentation scale of different geomorphic areas is dynamically determined in combination with the mean variance method. Finally, the extraction of field boundaries is realized on high-resolution remote sensing images with a spatial resolution of 1 m.

[0053] (3) The intelligent extraction and fusion model of high-resolution image fields provided by the present invention integrates a variety of remote sensing methods, and can carry out hierarchical research on the extraction of cultivated land information, the determination of cultivated land parcel boundaries, and the segmentation of crop fields in fragmented cultivated land areas. It effectively gives play to the advantages of deep learning and traditional methods in remote sensing fine segmentation. The field data extracted by the method provided by the present invention can provide the smallest recognition unit that matches the geographical entity boundary for crop model construction and precise recognition, and provide important data support for carrying out regional and even large-scale crop fine recognition research at the field level. Description of the Drawings

[0054] Figure 1 It is a schematic framework diagram of a method for intelligent extraction and fusion of fields for high-resolution satellite images provided by an embodiment of the present invention;

[0055] Figure 2 It is a schematic structural diagram of a multi-task deep learning module for cultivated land parcel boundaries provided by an embodiment of the present invention;

[0056] Figure 3 It is a schematic structural diagram of an iterative multi-branch fusion module provided by an embodiment of the present invention;

[0057] Figure 4 It is a schematic diagram of multi-scale segmentation sub-regions and region numbers provided by an embodiment of the present invention;

[0058] Figure 5 It is a schematic diagram of the change of the mean-variance curve of cultivated land in different geomorphic regions at different segmentation scales provided by an embodiment of the present invention;

[0059] Figure 6 It is a schematic diagram of the correlation analysis result between each feature provided by an embodiment of the present invention;

[0060] Figure 7 It is a schematic diagram of the relationship between the permutation feature importance score of each feature, the number of features participating in model training, and the model accuracy provided by an embodiment of the present invention;

[0061] Figure 8 It is a schematic diagram of the visualization effect of cultivated land extraction in the study area and the local magnification effect of cultivated land in different geomorphic regions provided by an embodiment of the present invention;

[0062] Figure 9 It is a schematic diagram of the cultivated land parcel result after correcting the field road provided by an embodiment of the present invention;

[0063] Figure 10 It is a schematic diagram of the shape and size distribution of fields in different geomorphic regions of the study area provided by an embodiment of the present invention;

[0064] Figure 11 It is the segmentation evaluation index S of the fields extracted in different regions provided by an embodiment of the present invention over 、Sunder Schematic diagram of spatial mapping results

[0065] Figure 12 Schematic diagram of the result of the internal type uniformity (Con) of the fields extracted from different regions provided by the embodiments of the present invention Detailed implementation manners

[0066] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part rather than all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention

[0067] As Figure 1 shown, a method for intelligent extraction and fusion of fields oriented to high-resolution satellite images provided by the embodiments of the present invention includes:

[0068] Step 1: Collect high-resolution satellite images of different regions and perform preprocessing to obtain a high-resolution satellite image dataset; wherein, the high-resolution satellite images include multispectral images and panchromatic images

[0069] Specifically, the preprocessing includes:

[0070] After radiometric calibration, atmospheric correction and orthorectification of the multispectral images, optimized multispectral images are obtained

[0071] After radiometric calibration and orthorectification of the panchromatic images, optimized panchromatic images are obtained

[0072] After image fusion, geometric correction and mosaicking of the optimized multispectral images and the optimized panchromatic images, preprocessed high-resolution satellite images are obtained. The spatial resolution of the preprocessed high-resolution satellite images is 1m

[0073] Step 2: Construct a high-resolution image field intelligent extraction and fusion network, and use the high-resolution satellite image dataset to train the high-resolution image field intelligent extraction and fusion network to obtain a high-resolution image field intelligent extraction and fusion model

[0074] Among them, the intelligent extraction and fusion network for high-resolution image farmland blocks includes an intelligent extraction module for cultivated land information, a multi-task deep learning module for cultivated land block boundaries, and an improved mean shift multi-scale segmentation module; the intelligent extraction module for cultivated land information is used to extract the cultivated land from the pre-processed high-resolution satellite images to obtain preliminary cultivated land blocks; the multi-task deep learning module for cultivated land block boundaries extracts the roads therein based on the preliminary cultivated land blocks to obtain independent cultivated land blocks; the improved mean shift multi-scale segmentation module masks the independent cultivated land blocks and the corresponding high-resolution images to obtain cultivated land images, and performs multi-scale segmentation on the cultivated land images to obtain the farmland block extraction results.

[0075] The method provided by the present invention conducts a hierarchical extraction study on the extraction of cultivated land information to the determination of cultivated land block boundaries and then to the segmentation of crop farmland in fragmented cultivated land areas by integrating various remote sensing methods, effectively leveraging the advantages of deep learning and traditional methods in the field of remote sensing fine classification. The produced farmland data can provide the smallest recognition unit that matches the geographical entity boundaries for crop model construction and precise recognition, and provide important data support for carrying out regional and even large-scale crop fine recognition research at the farmland level.

[0076] As an implementable manner, the construction and training steps of the intelligent extraction module for cultivated land information in the intelligent extraction and fusion network for high-resolution image farmland blocks are as follows:

[0077] Replace the encoding layer of the U-Net model with the pre-trained Resnet34 model to obtain the intelligent extraction module for cultivated land information;

[0078] Furthermore, expand the flexibility of the input data of the U-Net model to make it meet the input of various image bit depths (8bit, 16bit, 32bit) and any number of bands. To improve the model training speed, it is recommended to use 8bit as the main one;

[0079] Calculate the derived vegetation indices based on the original bands of the high-resolution satellite image dataset. Take the original bands and vegetation indices as the features of the high-resolution satellite image, and perform feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset; among them, the original bands include blue light (Blue), green light (Green), red light (Red), and near-infrared light (NIR); the vegetation indices include: normalized difference vegetation index (NDVI), normalized difference water index (NDWI), ratio vegetation index (RVI), difference vegetation index (DVI), green normalized difference vegetation index (GNDVI), and optimized soil-adjusted vegetation index (OSAVI). The calculation methods are shown in Table 1.

[0080] Table 1 Vegetation indices participating in feature optimization

[0081]

[0082] Train the intelligent extraction module of cultivated land information using the optimal feature subset to complete the training of the intelligent extraction module of cultivated land information.

[0083] Furthermore, perform feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset, specifically including:

[0084] Calculate the permutation feature importance of all features in the high-resolution satellite image and sort them;

[0085] Calculate the correlation coefficients of all features in the high-resolution satellite image and sort them;

[0086] Perform feature screening according to the sorting results of permutation feature importance and correlation coefficients to obtain the optimal feature subset;

[0087] Among them, the calculation method of permutation feature importance is as follows:

[0088] Use the high-resolution satellite image dataset as the input data to calculate the original prediction error e of the intelligent extraction module of cultivated land information orig :

[0089]

[0090] Among them, X represents the feature matrix, which is the feature set of the high-resolution satellite image, represents the intelligent extraction module of cultivated land information, and y represents the target vector;

[0091] Randomly permute the feature a in the feature matrix X to generate the feature permutation matrix X perm , and use the feature permutation matrix X perm as the input feature to calculate the permutation prediction error e of the intelligent extraction module of cultivated land information perm :

[0092]

[0093] Among them, X perm represents the feature permutation matrix;

[0094] Based on the results of the original prediction error e orig and the permutation prediction error e perm to obtain the permutation feature importance of feature a, and the permutation feature importance calculation formula is as follows:

[0095] FI a = e perm / e orig

[0096] Among them, FI a represents the permutation feature importance of feature a. The lower the permutation feature importance, the weaker the classification ability;

[0097] The correlation coefficient is calculated by the following formula:

[0098]

[0099] where r represents the correlation coefficient, and its value range is [-1, +1]. A negative correlation coefficient indicates a negative correlation between a and b, a positive one indicates a positive correlation, and 0 indicates no correlation. The closer the correlation is to 0, the weaker the correlation; the closer it is to -1 or +1, the stronger the correlation. cov(a, b) represents the covariance of features a and b, and σ a and σ b represent the standard deviations of features a and b respectively; a i and b i represent the i-th eigenvalue of features a and b respectively. and represent the means of features a and b respectively, and n is the number of eigenvalues of a and b. The stronger the correlation, the greater the information redundancy.

[0100] Based on the original bands, the present invention introduces multiple vegetation indices as features of high-resolution satellite images, and screens the features through permutation feature importance analysis and correlation analysis to obtain the optimal feature subset for participating in the training of the cultivated land information intelligent extraction module. The permutation feature importance method is a measure of the increase in model error when feature information is destroyed, and it is a model-independent feature importance calculation method with global interpretability.

[0101] As an implementable mode, such as Figure 2As shown in the figure, the multi-task deep learning module for cultivated land plot boundaries in the high-resolution image plot intelligent extraction and fusion network includes a shared encoder module, multiple iterative multi-branch fusion modules, and a stacked multi-branch prediction module; the shared encoder module and the iterative multi-branch fusion module are connected by residual block skip connections, and the output of the iterative multi-branch fusion module is connected to the input of the stacked multi-branch prediction module. It should be noted that the core network architecture used in the multi-task deep learning module for cultivated land plot boundaries is from (Batra, A., Singh S., Pang G., et al. Improved Road Connectivity by Joint Learning of Orientation and Segmentation[J]. Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2019: 10377-10385.). The only difference is that the present invention expands the number of input channels and the data bit depth of the network involved in the module, making it not limited to specific 8-bit and three-channel RGB data; in addition, the original loss function of the network is replaced with an improved cross-entropy loss function (Focal Loss) to solve the problem of serious imbalance in the ratio of positive and negative samples.

[0102] Among them, the shared encoder module is used to extract multi-scale features common to different tasks; the iterative multi-branch fusion module is used to learn the multi-scale features extracted by the shared encoder module, predict the road direction and road pixels to construct an intermediate prediction; the stacked multi-branch prediction module is used to fuse the intermediate prediction to achieve information interaction based on the orientation learning task and the road segmentation task, so as to complete the extraction of the road.

[0103] Specifically, the shared encoder module adopts a Resnet34 network; the iterative multi-branch fusion module includes a multi-branch block, a convolutional block, and a fusion block; the stacked multi-branch prediction module includes a multi-branch block, a convolutional block, a transposed convolutional block, and a per-pixel classifier.

[0104] Among them, the improved cross-entropy loss function adds a factor γ to the original binary cross-entropy loss function, so that the model reduces the loss of easy-to-separate samples during the training process, and prompts the model to focus more on difficult samples, as shown specifically below:

[0105]

[0106] Among them, y’ is the output (predicted probability) after passing through the activation function, and its value is between 0 and 1. y is the label, corresponding to 0 and 1 in binary classification. When γ is 0, it is the cross-entropy loss function. It can be seen that for the positive samples, the larger the output probability, the smaller the loss. For the negative samples, the smaller the output probability, the smaller the loss. At this time, the loss function is relatively slow during the iteration process of a large number of simple samples and may not be optimized to the optimal. For the Focal Loss function, when γ is greater than 0, if the predicted probability of the easy-to-separate samples is very large, the value of the loss function will be relatively smaller than that of the cross-entropy loss function, so that the model can focus more on the difficult-to-separate samples; for the difficult-to-separate samples, the predicted probability is small, then the corresponding value of the loss function will be relatively large, and the model will also focus more on the difficult-to-separate samples. In the embodiment of the present invention, the γ value is optimally set to 2.

[0107] As Figure 3 shown, the structure of the iterative multi-branch fusion module provided by the embodiment of the present invention. After the multi-branch block outputs, a convolutional block is used to extract from each branch, and a fusion block is used for merging. The embodiment of the present invention uses multiple iterative multi-branch fusion modules for iterative fusion. Through sufficient learning of road features, the road direction and road pixels are predicted, enabling the network to obtain the ability to complete discontinuous roads.

[0108] The purpose of the cultivated land plot boundary multi-task deep learning module provided by the embodiment of the present invention is to correct the results of the cultivated land information extraction module, solve the problem that the cultivated land plots with independent actual spatial positions are stuck together due to incomplete extraction of field roads, and obtain cultivated land plots with independent actual spatial positions. In the cultivated land area, the field roads can divide the cultivated land into discrete cultivated land plots, thereby limiting the classification process within the independent cultivated land plots. This not only helps to reduce the transmission of classification errors, but also helps to improve the neatness of the cultivated land edges and reduce the misclassification and incorrect classification phenomena in the area adjacent to the cultivated land edge and the road. The main problem faced by the existing road extraction methods is the problem of road disconnection.

[0109] As an implementable manner, the processing process of the mean shift multi-scale segmentation module in the intelligent extraction fusion network of high-resolution image field plots is as follows:

[0110] Mask the corresponding high-resolution satellite image based on the independent cultivated land plot to obtain the cultivated land image;

[0111] Based on the elevation, the cultivated land image is divided into geomorphic zones (plain 0 - 200m, hill 200 - 500m, and mountain above 500m), and the geomorphic zone division result is segmented based on the linear feature data to obtain irregular image blocks; As Figure 4 shown, it is the final division result of the geomorphic and linear feature segmentation of the area using the embodiment of the present invention.

[0112] Determine the optimal segmentation scale by calculating the mean variance of irregular image patches in different geomorphic regions;

[0113] Specifically, randomly select an irregular image patch from the irregular image patches included in each geomorphic sub-region, calculate the mean variance at different segmentation scales, analyze the trend of the mean variance changing as the segmentation scale increases from small to large. When the turning point where the mean variance changes from large to small appears, the corresponding segmentation scale is determined as the optimal segmentation scale. As Figure 5 shown, for the mean variance curves of cultivated land in different geomorphic regions at different segmentation scales, as the number of pure objects in the image layer increases and the spectral variation between adjacent objects increases, the mean variance of the objects increases; on the contrary, when the number of mixed objects increases, the spectral variation between adjacent objects decreases, and the mean variance of the objects changes. Therefore, the present invention selects the segmentation scale corresponding to the change from large to small of the mean variance as the optimal segmentation scale.

[0114] Among them, the mean variance is calculated by the following formula:

[0115]

[0116] Among them, represents the brightness value of the i-th pixel within the object in the T-th band, n represents the number of pixels in the object, D T represents the brightness mean of a single image object in the T-th band; represents the brightness mean of all objects in the image in the T-th band, m represents the total number of objects in the image, S 2 represents the mean variance.

[0117] Segment all the irregular image patches using the multi-scale segmentation algorithm corresponding to the optimal segmentation scale, and merge the segmentation results of all the irregular image patches to obtain the final result of field block extraction.

[0118] As Figure 6As shown, the correlation analysis among the 10 features of the embodiments of the present invention is presented, and the correlation results on cultivated land and non-cultivated land are also given. It can be seen that the correlations between spectral features and vegetation indices are relatively low (r < 0.6). Among the four spectral features, strong correlations are shown among the red, green, and blue bands, and the correlations between these three and the near-infrared are relatively low. In previous machine learning classification studies based on remote sensing images, the participation of the near-infrared band has played a positive role in improving the classification accuracy and performance of the model (Liu et al., 2021; Liu et al., 2022; Unnikrishnan et al., 2019; Wang et al., 2022). Among the six vegetation index features, strong positive correlations are shown between NDVI and OSAVI and GNDVI, and a strong negative correlation is shown between NDVI and NDWI; strong negative correlations are shown between NDWI and OSAVI and GNDVI. The correlations between RVI and DVI and other features are relatively low.

[0119] As Figure 7 shown, the importance ranking of each feature is carried out by using the permutation feature importance method in the embodiments of the present invention. The LSTM neural network model is used for model training and prediction, and then the permutation feature importance method is used for the importance ranking of each feature. Figure 7 (a) shows the permutation feature importance results of model prediction on the test set, represented by the mean squared error (MSE). To exclude randomness, the final importance score of each feature is the mean value obtained after randomly shuffling 10 times. For the convenience of comparison, the results of the present invention are uniformly multiplied by a constant value of 1000. From Figure 7 (a), it can be seen that the importance of the three index features of OSAVI, GNDVI, and NDWI is the lowest. In addition, as Figure 7 (b) shows, when the number of features participating in model training is greater than 6, the rising speed of the model classification accuracy slows down. When the number of features is 8, the model classification accuracy is the highest. Beyond this number, the model classification accuracy decreases slightly.

[0120] Considering the results of correlation analysis and importance scoring comprehensively, according to the principle that "the stronger the correlation, the greater the information redundancy, the lower the feature importance score, and the weaker the classification ability" (Zhang Peng et al., 2019), the features that may need to be removed include NDWI, GNDVI, and OSAVI. Since OSAVI is sensitive to changes in vegetation canopy cover, when extracting cultivated land information based on multi-temporal data or long-term time series data, this index has a good indication effect, so it is retained. In addition, although the correlation of Red, Green, and Blue in the spectral bands is relatively high, the feature importance of these three is relatively high, so they are retained. To sum up, the features used for cultivated land information extraction in this paper include 8, namely Blue, Green, Red, NIR, NDVI, RVI, DVI, and OSAVI.

[0121] As Figure 8 shown, it shows the preliminary cultivated land parcel spatial distribution information of the study area in 2022 extracted by the intelligent cultivated land information extraction module, indicating the effective cultivated land extraction ability in flat and hilly areas. And the details of the cultivated land show clear information about the field roads, especially obvious in flat areas. However, the magnified results show that some very small parcels are missed, especially near the roads. This further illustrates the necessity of constructing a field boundary extraction module to correct this result.

[0122] As Figure 9 shown, it shows the spatial distribution and details of independent cultivated land parcels after the correction of field roads. Figure 3 It shows that the neatness of the cultivated land parcel boundaries has been improved before and after the correction. The field roads are relatively stable within a certain period of time. Using the field roads as the defined boundaries to correct the preliminary cultivated land parcel extraction results can obtain relatively stable and spatially independent cultivated land parcel results, which provides a good data basis for dividing the cultivated land parcels into pure parcels with a single internal land type.

[0123] Figure 10 It is the distribution of the shapes and sizes of the field parcels in different geomorphic regions of the study area. In the embodiment of the present invention, the field parcels in the study area are divided into the following three grades, namely extremely small field parcels (field parcel area < 0.64 hm 2 ), small field parcels (0.64 hm 2 < field parcel area < 2.56 hm 2 ), and medium-sized field parcels (2.56 hm 2 < field parcel area < 16 hm 2 ). Most of the field parcels in the entire study area are small field parcels, and the number and area of medium-sized field parcels are both low, further indicating a high degree of fragmentation of the cultivated land in the entire study area. The proportion of extremely small field parcels is the highest in the hilly areas in the north and south of the study area, while in the central plain area, small field parcels are the main ones, followed by medium-sized field parcels, with an area of 0.64 hm 2The following fields are the fewest.

[0124] Figure 11 For the segmentation evaluation index S of a single field object in each evaluation sub-region over 、S under Spatial mapping results. The S of each evaluation sub-region under and S over The index situations are all different, and this difference is more obvious in the hilly area. In the plain area, due to the large and regular-shaped plots and little influence of vegetation, the segmentation quality of the plots is relatively high. In the hilly area, the segmentation quality is relatively poor due to the shadows caused by vegetation, light, etc. Over-segmentation in the plain area mostly occurs in plots with a large area, while under-segmentation mostly concentrates in plots with a small area, and this rule is not significant in the hilly area.

[0125] Figure 12 For the comparison result of the internal type uniformity (Con) of the fields extracted from each evaluation sub-region. The internal type uniformity index of the field is an intuitive index to judge whether there are multiple crops inside the field and whether it is a pure plot. The result of this index can provide a reference for crop classification research. It can be seen from the figure that the proportion of the number of pure plots in each evaluation area is above 96%, which provides good basic data for crop classification facing plots. Among them, the highest is in the Central Plain Area_1, and this result is consistent with the good segmentation effect reflected by its S over and S under The proportion of the number of pure plots in the Central Plain Area_2 is the lowest. Although the degree of under-segmentation in the Southern Hilly Area is the highest, its degree of over-segmentation is higher than the former. This shows that the higher the degree of under-segmentation, the higher the possibility of generating mixed plots. Although the higher the degree of over-segmentation will increase the proportion of pure plots and reduce the probability of generating mixed plots, at the same time, it will also damage the original integrity of the plots. Therefore, in practical applications, an appropriate segmentation scale should be selected according to the application purpose and actual needs.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent extraction and fusion method for field plots oriented to high-resolution satellite images, characterized in that, Including: Step 1: Collect high-resolution satellite images from different regions and perform preprocessing to obtain a high-resolution satellite image dataset; Step 2: Construct a high-resolution image field intelligent extraction fusion network, and use the high-resolution satellite image dataset to train the high-resolution image field intelligent extraction fusion network to obtain a high-resolution image field intelligent extraction fusion model; Among them, the high-resolution image field intelligent extraction fusion network includes a cultivated land information intelligent extraction module, a cultivated land plot boundary multi-task deep learning module, and an improved mean shift multi-scale segmentation module; the cultivated land information intelligent extraction module is used to extract the cultivated land of the preprocessed high-resolution satellite image to obtain a preliminary cultivated land plot; the cultivated land plot boundary multi-task deep learning module extracts the roads therein based on the preliminary cultivated land plot to obtain independent cultivated land plots; the improved mean shift multi-scale segmentation module is used to mask the independent cultivated land plots and the corresponding high-resolution satellite images to obtain a cultivated land image, and perform multi-scale segmentation on the cultivated land image to obtain a field extraction result.

2. The intelligent extraction and fusion method of field plots for high-resolution satellite images according to claim 1, characterized in that In Step 1, the spatial resolution of the high-resolution satellite image dataset is 1m.

3. A method for intelligent extraction and fusion of field plots for high-resolution satellite images according to claim 1, characterized in that, The high-resolution satellite images include multi-spectral images and panchromatic images.

4. A method for intelligent extraction and fusion of field plots for high-resolution satellite images according to claim 3, characterized in that The preprocessing includes: After radiometric calibration, atmospheric correction, and orthorectification of the multi-spectral image, an optimized multi-spectral image is obtained; After radiometric calibration and orthorectification of the panchromatic image, an optimized panchromatic image is obtained; After image fusion, geometric correction, and mosaicking of the optimized multi-spectral image and the optimized panchromatic image, a preprocessed high-resolution satellite image is obtained.

5. A method for intelligent extraction and fusion of field blocks for high-resolution satellite images according to claim 1, characterized in that, The construction and training steps of the cultivated land information intelligent extraction module are as follows: Replace the encoding layer of the U-Net model with a pre-trained Resnet34 model to obtain a cultivated land information intelligent extraction module; Calculate the derived vegetation indices based on the original bands of the high-resolution satellite image dataset, use the original bands and the vegetation indices as the features of the high-resolution satellite image, and perform feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset; among them, the original bands include blue light, green light, red light, and near-infrared light, and the vegetation indices include: normalized difference vegetation index, normalized difference water index, ratio vegetation index, difference vegetation index, green normalized difference vegetation index, and optimized soil-adjusted vegetation index; Use the optimal feature subset to train the cultivated land information intelligent extraction module to complete the training of the cultivated land information intelligent extraction module.

6. The intelligent extraction and fusion method of field blocks for high-resolution satellite images according to claim 5, characterized in that Performing feature screening based on permutation feature importance analysis and correlation analysis to determine the optimal feature subset specifically includes: Calculate the permutation feature importance of all features in the high-resolution satellite image and sort them; Calculate the correlation coefficients of all features in the high-resolution satellite image and sort them; Perform feature screening according to the sorting results of permutation feature importance and the sorting results of correlation coefficients to obtain the optimal feature subset; Among them, the calculation method of the permutation feature importance is as follows: Using the high-resolution satellite image dataset as input data, calculate the original prediction error e of the intelligent cultivated land information extraction module orig : Among them, X represents the feature matrix, which is the feature set of high-resolution satellite images, represents the intelligent extraction module for cultivated land information, and y represents the target vector; Randomly permute feature a in the feature matrix X to generate a feature permutation matrix X perm , and use the feature permutation matrix X perm as the input feature to calculate the permutation prediction error e of the cultivated land information intelligent extraction module perm : Among them, X perm represents a feature permutation matrix; Based on the original prediction error e orig and the permutation prediction error e perm the permutation feature importance of feature a is obtained, and the calculation formula of the permutation feature importance is as follows: FI a = e perm / e orig Among them, FI a represents the permutation feature importance of feature a; The correlation coefficient is calculated by the following formula: Among them, r represents the correlation coefficient, cov(a, b) represents the covariance of features a and b, and σ a and σ b represent the standard deviations of features a and b respectively; a i and b i represent the i-th eigenvalues of features a and b respectively, and represent the means of features a and b respectively, and n is the number of eigenvalues of a and b.

7. A method for intelligent extraction and fusion of field plots for high-resolution satellite images according to claim 1, characterized in that The cultivated land plot boundary multi-task deep learning module includes a shared encoder module, multiple iterative multi-branch fusion modules, and a stacked multi-branch prediction module; the shared encoder module and the iterative multi-branch fusion module are connected by a residual block skip connection, and the output of the iterative multi-branch fusion module is connected to the input of the stacked multi-branch prediction module; Among them, the shared encoder module is used to extract multi-scale features common to different tasks; the iterative multi-branch fusion module is used to learn the multi-scale features extracted by the shared encoder module, predict the road direction and road pixels to construct an intermediate prediction; the stacked multi-branch prediction module is used to fuse the intermediate prediction to achieve information interaction based on the direction learning task and the road segmentation task, so as to complete the extraction of the road.

8. A method for intelligent extraction and fusion of field plots for high-resolution satellite images according to claim 1, characterized in that The processing process of the improved mean shift multi-scale segmentation module is as follows: Based on the corresponding high-resolution satellite image of the independent cultivated land plot, a mask is obtained to get the cultivated land image; Based on the elevation, the cultivated land image is divided into geomorphic zones, and based on the strip-shaped ground object data, the geomorphic zone division result is segmented to obtain irregular image blocks; The optimal segmentation scale is determined by calculating the mean variance of the irregular image blocks in different geomorphic regions; All the irregular image blocks are segmented by using the multi-scale segmentation algorithm corresponding to the optimal segmentation scale, and the segmentation results of all the irregular image blocks are merged to obtain the final cultivated land block extraction result.

9. A method for intelligent extraction and fusion of field plots for high-resolution satellite images according to claim 8, characterized in that The determination of the optimal segmentation scale by calculating the mean variance of the irregular image blocks in different geomorphic regions specifically includes: Arbitrarily select an irregular image block in the irregular image blocks included in each geomorphic zone, calculate the mean variance at different segmentation scales, analyze the trend of the mean variance changing with the segmentation scale from small to large, and when the mean variance shows a turning point from large to small, the corresponding segmentation scale is determined as the optimal segmentation scale; Among them, the mean variance is calculated by the following formula: Among them, represents the brightness value of the i-th pixel within the object in the T-th band, n represents the number of pixels in the object, D T represents the brightness mean of a single image object in the T-th band; represents the brightness mean of all objects in the image in the T-th band, m represents the total number of objects in the image, S 2 represents the mean variance.

Images