Intelligent identification method for cultivated land non-grain crops based on high-resolution remote sensing image

Abstract

The invention discloses a cultivated land non-grain crop intelligent identification method based on a high-resolution remote sensing image. The method comprises the following steps: step 1, image acquisition and processing; 2, constructing a sample library; and step 3, model training and optimization, comprising the following sub-steps: (1) constructing a deep learning network structure; (2) model training; (3) model prediction; (4) evaluating the precision of a model result; (5) carrying out model iteration and optimization; and step 4, automatic extraction and post-processing. According to the invention, through a deep learning technology, an end-to-end identification process from an image to a result is realized, and various non-grain targets involved in the invention are automatically identified; secondly, the algorithm processing efficiency is evaluated in combination with a result evaluation index, and algorithm optimization and parameter adjustment are carried out in the aspects of non-grain pattern spot extraction precision, accuracy and the like.

Description

Technical Field

[0001] This invention relates to the field of intelligent identification technology for non-grain crops, and in particular to an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing images. Background Technology

[0002] The rapid development of land cover identification technology based on high-resolution satellites and unmanned aerial vehicles (UAVs) has provided strong technical support for the precise management of arable land resources. Satellite remote sensing and UAV-based land cover identification technology is an advanced method that combines satellite remote sensing and UAV technologies to accurately identify and analyze land cover types. Satellite remote sensing acquires large-scale, high-resolution surface images from high-altitude platforms, providing macroscopic land cover information. UAVs, with their flexibility, high precision, and low-altitude flight capabilities, can conduct more detailed, real-time observations of local areas. The combination of the two enables comprehensive, accurate, and dynamic monitoring of land cover types, providing crucial data support for agricultural management, urban planning, environmental protection, and other fields.

[0003] "Non-grain conversion" refers to the use of arable land for planting non-grain crops (such as fruits, tea, and flowers) or for non-grain-related activities (such as digging ponds for fish farming or building factories), violating the policy requirement that arable land should primarily be used for grain production. Traditional identification of non-grain crops on arable land is based on crop extraction from remote sensing images. Extraction methods typically combine field surveys with visual interpretation of remote sensing images, which is labor-intensive and time-consuming. Because different land features exhibit diverse spectral characteristics and complex textures in remote sensing images, shallow network machine learning techniques are prone to causing land use classification confusion. Summary of the Invention

[0004] To address the problems existing in the prior art, this invention provides an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery, improving the accuracy and efficiency of non-grain crop extraction. This invention first acquires relevant data within the region, including high-resolution orthophoto remote sensing imagery from UAVs, sample annotation data, and other auxiliary data, establishing spatial and temporal relationships between the data to form a non-grain identification sample set. Based on this, deep learning technology is used to achieve an end-to-end identification process from image to result, automatically identifying various non-grain targets involved in the invention. Secondly, the algorithm's processing efficiency is evaluated using result evaluation indicators, and algorithm optimization and parameter adjustments are made in terms of accuracy and precision in extracting non-grain patches.

[0005] This invention provides a method for intelligent identification of non-grain crops in cultivated land based on high-resolution remote sensing imagery, comprising the following steps: Step 1, Image Acquisition and Processing; includes the following steps: Delineation of areas for identifying non-grain use of arable land; Delineation of crop sample calibration and collection areas: Delineating crop sample calibration and collection area units; Orthophoto production operation: On a quarterly basis, aerial photography data with pixel size of 0.1 meters to 0.03 meters is collected in different regions, at different times and under different weather conditions within the crop sample calibration collection area. Based on the crop sample calibration collection area, aerial triangulation modeling and mosaicking are performed to produce sample images. Step two, sample library construction, includes the following steps: determining sample production specifications, establishing sample classification, constructing sample interpretation rules, and enriching sample image data; sample annotation, labeling non-grain crop target categories on image slices according to sample annotations; sample data collection and annotation; sample quality inspection; sample image data processing, uniformly performing bit depth conversion on sample images to convert them into 8-bit depth image data; sample set construction. Step 3, Model Training and Optimization, includes the following sub-steps: (1) Deep learning network structure construction: The model is a deep learning network framework for UAV DOM data constructed using the SegFormer semantic segmentation model; the model mainly includes a hierarchical Transformer encoder module and a lightweight multilayer perceptron decoding module, which are used to generate the final semantic segmentation mask; according to the SegFormer construction strategy, the model is adapted to three-band or multi-band UAV data by modifying and adding the parameters of the model when reading images. (2) Model training: During the training process, the input ground object samples will pass through each layer of the model to generate a pixel-level segmentation result with the same size as the input image. The loss value between the result and the true value is calculated through the loss function. Subsequently, backpropagation is used to modify and update the weight parameters. During this process, the various running parameters and system resources in the algorithm model can be configured according to the sample dataset so that the model can be used for training on the sample data, achieving faster training speed and higher training accuracy. (3) Model prediction: The test data randomly segmented from the sample dataset is input into the trained network model to obtain the recognition results of the changing regions in the test data; the test dataset is randomly sampled from the sample dataset. The performance of the model is evaluated by parameters such as accuracy and recall in the prediction results, the error is analyzed, and the model with the highest prediction accuracy is selected. (4) Evaluation of model result accuracy: 10% of the test samples were randomly selected from the constructed sample library, and the accuracy of the algorithm was evaluated by the F1 score, specifically as follows: ; Where tp represents the number of samples correctly predicted as positive; fp represents the number of false positives, i.e. the number of negative samples incorrectly predicted as positive; and fn represents the number of false negatives, i.e. the number of positive samples incorrectly predicted as negative. (5) Model Iteration and Optimization: After the sample data and model parameters are processed, the entire algorithm model is trained. Based on the loss, training accuracy and other information fed back during the model training process, the model structure and model parameters are readjusted and the training is run again. By repeatedly adjusting the training model, sample data or model hyperparameters, the training results are made to meet the accuracy requirements. The model with the best training effect is saved for testing. Based on the accuracy evaluation of the results, actual needs and sample supplementation, the model is continuously iterated and optimized until the accuracy of the model results can be used for actual production. Step four, automatic extraction and post-processing; includes the following sub-steps: (1) Automatic extraction of patches: Before using the semantic segmentation model for recognition, the UAV data to be predicted is sliced ​​to make its size match the input of the model; when the model is performing recognition, the optimal model weight file will be loaded first, and the sliced ​​data will be calculated and transformed step by step through each layer of the model to obtain a segmentation result with the same size as the input slice; the predicted slices are spliced ​​to obtain a semantic segmentation mask with the same size as the UAV data, and the result is vectorized to obtain the extracted patch vector result; (2) Post-processing of extraction results: After the remote sensing image is automatically extracted, there may be noise in the extracted patches. The results need to be post-processed to eliminate the noise of the predicted fragments as much as possible, fill the gaps in the patches, and at the same time ensure that the outline of the predicted patches is clear and connected. The gap filling algorithm and the filtering algorithm are used for processing. (3) Manual verification and correction: Based on relevant business knowledge, manual optimization is used to ensure that the extracted results are free of misidentified patches and that the boundaries of the patches are accurate, ultimately forming the final results of automated extraction.

[0006] Furthermore, the main technical requirements for the sample images are as follows: a. The overall image tone is consistent, with moderate contrast, natural transitions, clear textures, rich layers, and no obvious distortion; b. The image can accurately reflect the spectral characteristics of the land cover type; c. The image has no obvious distortion or loss of features, no ghosting or blurring, and the boundaries of features are clear; d. The transitions at image mosaicking and edge joining are natural, and the same feature should be kept intact; e. The image bit depth can be 8-bit, 16-bit, or 32-bit; f. The image color mode can be panchromatic, color, or multispectral; g. The NoData area of ​​the image is set to 0.

[0007] Furthermore, step two specifies the sample preparation standards as follows: (1) Sample classification system: In line with the goal of the "non-grain" identification work, several non-grain crops were selected to establish a sample classification; (2) Constructing sample interpretation rules: refine the scene, determine the correspondence between the features of ground objects on the captured images and the actual ground object categories based on the results of the field survey, and then determine the interpretation rules through the image features; for categories that cannot be determined, field surveys are required to determine them. (3) Sample image data augmentation: The sample selection should be evenly distributed across different dimensions as much as possible, and additional data should be added for difficult cases; the dimensions include season, terrain, and category; the data source used is aerial photography data from 0.1 meters to 0.03 meters. (4) Sample iteration and supplementation: Adopt the production and accumulation mode, check the production results simultaneously and analyze the omission and error of extraction, make supplementary positive and negative samples in a targeted manner, and then introduce the new samples into the training to update the production model and iterate. (5) Sample image data processing: uniformly perform bit depth conversion to convert it into 8-bit depth image data; Furthermore, the main technical requirements for sample annotation in step two are as follows: (1) Sufficient samples need to be labeled for each type of non-grain crop; when labeling based on image slices, due to inconsistent image resolution, each image slice must be at least 3000*3000 in size. The target categories of non-grain crops on the image slices must be labeled without error or omission. (2) Within the same area of ​​the sample, there should be no obvious errors or omissions in the label collection. Within the same area of ​​the sample, the sum of the areas of the erroneous or omitted patches should not exceed 5% of the total area. (3) Within the same region, the error rate of sample attribute assignment should not exceed 5%; (4) The boundary between the sample label and the target land type should be aligned. For artificial features with obvious boundaries or targets with small areas, the alignment difference should not exceed 2 pixels. For natural features with indistinct boundaries, the alignment difference should not exceed 5 pixels. (5) The sample labels are raster data with a bit depth of 8 bits. The label index value of the target land type is an integer from 1 to 255, and the label index value of the non-target land type is 0. (6) The region sample labels are vector data and there are no topological errors such as face overlap, sharp corners and face cracks.

[0008] Furthermore, the sample image data processing in step two specifically involves: A linear stretching method is used to normalize the pixels distributed in [inputmin, inputmax] in each band, and then uniformly stretch them linearly to [outputmin, outputmax]. The formula is as follows: ; In the formula, value(x,y) represents each pixel in the band; when inputmin and inputmax are the minimum and maximum values ​​of the band pixel statistics, it is called extreme value stretching; when inputmin and inputmax are the pixel values ​​of the cumulative percentile position of the pixel histogram, it is called percentile stretching. `outputmin` and `outputmax` are the range of pixel values ​​for the stretched output. When they are 0 and 255 respectively, it is an 8-bit deep output. `result` is the stretched output result.

[0009] Furthermore, the sample set construction in step two is specifically as follows: (1) Define the requirements for sample set creation: Sample size: The sample size needs to take into account the features of different sample categories and the structure of deep learning networks, and is set to a regular square size with a side length of no less than 512 pixels; Sample overlap: Sample overlap represents the degree of overlap between individual sample pieces, and is set to half the sample size; Valid percentage of samples: The valid percentage of samples is the proportion of the specified sample category area in the label piece to the size of the entire sample piece. During production, sample pieces with a valid percentage higher than 5% are selected. Label Category: The category labeling of the label patch is based on standard onehot encoding. If the sample is a single-category land cover sample, label 0 is the background and 1 is the target land cover. If the sample is a multi-category land cover sample, the labels are numbered starting from 0, where 0 is the background and 1 is the label for the first category, and so on. Sample segment name: The sample segment name is usually composed of the sample set name + sample segment number. The sample segment number starts from 1, and is padded at the beginning if the number of digits is insufficient. Sample data format: Image patches are in .tif or .jpg format. .jpg format requires an accompanying .aux.xml file with the same name to record the spatial range information of the patch and ensure spatial positioning of the data; Label patches are in .png format and require an accompanying .aux.xml file with the same name to record the spatial range information of the label patch and ensure spatial positioning. (2) Sample set creation: Based on the labeled samples of each category, independently create a sample set for each category; (3) Sample set quality check: After the sample set is completed, the sample set data needs to be checked for quality. If there are problems with the quality check of the sample set data, the sample set data should be modified or regenerated until all quality check requirements are met and the sample quality check is completed. The sample set quality check items mainly include spatial reference system, metadata, and logical consistency content. (4) Sample set results organization: The sample set consists of n sets of sample fragments and sample set metadata; Sample Patch: A set of sample patches includes image patches, tag patches, and patch metadata. Image patches are regular-sized image data after cropping. Surface change image patches include two images from different periods. Tag patches are rasterized sample category label data corresponding to the image patches. Patch metadata records the attribute information of related sample patches, including the name of the individual patch data and the effective percentage of the sample. Sample set metadata records the correspondence of the entire sample set and the corresponding image and classification information for database management. The sample set metadata file is in .xml format.

[0010] Furthermore, in step three, the layered Transformer encoder module specifically comprises: The Transformer encoder module is used to input an image of a given resolution. The image has a resolution of [resolution value], and the output resolution is [resolution value]. Hierarchical feature map , ;in The layered feature map contains coarse and fine features at different resolutions, corresponding to the channel dimension of the feature map. A key component of the hierarchical Transformer encoder module is the self-attention layer. In the original multi-head self-attention process, each head... , , Having the same dimensions ,in Let be the length of the sequence, then the estimated value of self-attention is: ; Its computational complexity is To reduce the computational complexity of self-attention estimation at high resolution, the model uses a reduction ratio. To reduce the sequence length, the formula is as follows: ; ; in, The sequence to be reduced, It is Reshape into shape sequence, It is to put one A dimensional tensor is used as input and outputs a Linear layers of dimensional tensors, new Dimension is This reduces the complexity of the self-attention mechanism to ; The model mixes in the feedforward network The convolutional and MLP components form the Mix-FNN module to provide positional information for the Transformer. The Mix-FNN can be defined as follows: ; In the formula, These are features derived from the self-attention module. It is an activation function based on Gaussian error.

[0011] Further, in step three, the lightweight multilayer perceptron decoding module processes the feature maps at different resolutions output by the Transformer encoder and obtains the final segmentation mask. This mainly includes four processes: 1) Unifying the channel dimensions of the feature maps at different resolutions output by the Transformer encoder using an MLP layer; 2) Upsampling the feature maps processed in step 1) to one-quarter of their original size and concatenating them; 3) Fusing the concatenated feature maps using an MLP layer; 4) Predicting the final segmentation mask using the fused feature map output in step 3) through another MLP layer. The above process can be represented as: ; ; ; ; In the formula, Multi-layer feature maps at different resolutions The number of channel dimensions after feature map unification. This is the feature map after unifying the channel dimensions. This is the feature map after upsampling. This is the spliced ​​and merged feature map. For the output category, This is the final segmentation mask.

[0012] Furthermore, in step four, the gap-filling algorithm utilizes the idea of ​​morphological reconstruction to fill the holes, and the principle is as follows: ; Wherein, dst is the image after gap filling, Masker is the continuously dilating image, SE is the dilating structuring element, and Mask is the image constraining the dilating image.

[0013] Furthermore, in step four, the filtering process is to address the issue of isolated pixels in the classified images; the category filtering method is used to eliminate isolated classified pixels; if the number of pixels in a patch is less than the input threshold, the patch will be removed from that category.

[0014] In sub-step (3) of step four, the manual optimization work mainly includes: deleting erroneous patches, supplementing missed patches, and modifying the boundaries of correct patches.

[0015] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention provides a self-service convolutional neural network model parameter setting system for intelligent interpretation of farmland remote sensing. Users can set the number of input image bands, image patch size, and model layers according to sample availability and computing resources, enabling rapid model training. By supplementing missed and falsely detected objects in the prediction results as positive and negative samples, iterative learning is performed to improve the model's accuracy.

[0016] 2. This invention creates a multi-layer semantic structure model of "land parcel-object" through two segmentations. The land parcel layer focuses on potentially changing land parcels based on statistical knowledge reasoning, while the object layer provides evidence of change through land cover classification using an object convolutional neural network model. The land parcel layer and the object layer jointly detect at different scales and corroborate each other, thereby improving the accuracy of change detection.

[0017] 3. This invention provides two detection modes for intelligent interpretation of farmland remote sensing: vector-to-image and image-to-image. Users can use them in combination based on available data resources, or choose any mode to achieve change detection. Vector-to-image change detection is suitable for unchanged samples, situations requiring high detection accuracy but with moderate speed requirements. Image-to-image change detection requires a certain amount of sample accumulation before it can be used; its advantages are better generalization ability and faster prediction speed.

[0018] 4. In the intelligent target detection and recognition based on massive remote sensing data, how to fuse the extracted features to assist in multi-source information extraction is a key technology that needs to be overcome. Therefore, this invention proposes a multi-source target intelligent detection and recognition technology based on feature-level fusion. On the basis of feature extraction from multi-source visible light images, a feature-level fusion algorithm is adopted to make full use of the data features of natural resource monitoring targets in multi-source images, which can effectively improve the target detection rate and provide necessary support for remote sensing monitoring of natural resources.

[0019] 5. Traditional methods for extracting land features based on remote sensing imagery typically combine field surveys with visual interpretation of remote sensing images, which is labor-intensive and time-consuming. Due to the varied spectral and complex texture features of different land features in remote sensing images, shallow network machine learning techniques are prone to causing land type confusion. To improve the accuracy and efficiency of land feature extraction, this invention implements an automated land feature extraction model based on deep learning technology by performing spectral, geometric, and textural analyses on remote sensing data with different spatiotemporal differences. This invention collects, creates, and uses semantic segmentation samples of different land features such as cultivated land, forest land, water bodies, buildings, roads, greenhouses, and bare soil, and uses the SegFormer semantic segmentation framework to extract these land features. SegFormer is an efficient and powerful semantic segmentation framework composed of a Transformer and a lightweight multilayer perceptron (MLP). SegFormer has the advantages of high efficiency, high accuracy, and strong robustness. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0021] Figure 1 This is a flowchart of an embodiment of the present invention of an intelligent identification method for non-grain crops in arable land based on high-resolution remote sensing images; Figure 2 This is an organizational catalog of sample collection results for an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing images, according to an embodiment of the present invention. Figure 3 This is a sample set organization catalog diagram of an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing images according to an embodiment of the present invention; Figure 4 This is a model network structure diagram of an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing images, according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] In the description of this invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0024] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Furthermore, the technical features involved in the different embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0025] like Figure 1 As shown in the figure, this embodiment presents an intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing images. The method specifically includes the following steps: Step 1, Image Acquisition and Processing; includes the following steps: Delineate areas for identifying non-grain use of arable land; based on the local historical arable land topography, create buffer zone aggregated surface vectors, and divide and generate areas of 5 to 15 square kilometers according to the processing capabilities of the graphics computing cluster's hardware and software.

[0026] Delineation of crop sample calibration and collection areas: Delineate crop sample calibration and collection area units; refer to the local farmland hidden outflow layer to understand the local crop planting behavior and the distribution of crops in the townships, and delineate crop sample calibration and collection area units.

[0027] Orthophoto production operation: On a quarterly basis, aerial photography data with pixel size of 0.1 meters to 0.03 meters is collected in different regions, at different times and under different weather conditions within the crop sample calibration collection area. Based on the crop sample calibration collection area, aerial triangulation modeling and mosaicking are performed to produce sample images. The main technical requirements for sample images are as follows: a. The overall image tone is consistent, with moderate contrast, natural transitions, clear textures, rich layers, and no obvious distortion; b. The image can accurately reflect the spectral characteristics of the land cover type; c. The image has no obvious distortion or loss of features, no ghosting or blurring, and the boundaries of features are clear; d. The transitions at image mosaicking and edge joining are natural, and the same feature should be kept intact; e. The image bit depth can be 8-bit, 16-bit, or 32-bit; f. The image color mode can be panchromatic, color, or multispectral; g. The NoData area of ​​the image is set to 0.

[0028] Step two, sample library construction, includes the following steps: Sample image data processing: uniformly perform bit depth conversion on the sample images to convert them into 8-bit depth image data; sample set construction; 1. Define sample preparation specifications, establish sample classification, construct sample interpretation rules, and enrich sample image data; the specific sample preparation specifications are as follows: (1) Sample classification system: In line with the goal of the "non-grain" identification work, several non-grain crops were selected to establish a sample classification; (2) Constructing sample interpretation rules: refine the scene, determine the correspondence between the features of ground objects on the captured images and the actual ground object categories based on the results of the field survey, and then determine the interpretation rules through the image features; for categories that cannot be determined, field surveys are required to determine them. (3) Sample image data augmentation: The sample selection should be evenly distributed across different dimensions as much as possible, and additional data should be added for difficult cases; the dimensions include season, terrain, and category; the data source used is aerial photography data from 0.1 meters to 0.03 meters. (4) Sample iteration and supplementation: Adopt the production and accumulation mode, check the production results simultaneously and analyze the omission and error of extraction, make supplementary positive and negative samples in a targeted manner, and then introduce the new samples into the training to update the production model and iterate. (5) Sample image data processing: uniformly perform bit depth conversion to convert it into 8-bit depth image data; 2. Sample labeling: Label the image slices with non-grain crop target categories based on sample labels; the main technical requirements for sample labeling are as follows: (1) Sufficient samples need to be labeled for each type of non-grain crop; when labeling based on image slices, due to inconsistent image resolution, each image slice must be at least 3000*3000 in size. The target categories of non-grain crops on the image slices must be labeled without error or omission. (2) Within the same area of ​​the sample, there should be no obvious errors or omissions in the label collection. Within the same area of ​​the sample, the sum of the areas of the erroneous or omitted patches should not exceed 5% of the total area. (3) Within the same region, the error rate of sample attribute assignment should not exceed 5%; (4) The boundary between the sample label and the target land type should be aligned. For artificial features with obvious boundaries or targets with small areas, the alignment difference should not exceed 2 pixels. For natural features with indistinct boundaries, the alignment difference should not exceed 5 pixels. (5) The sample labels are raster data with a bit depth of 8 bits. The label index value of the target land type is an integer from 1 to 255, and the label index value of the non-target land type is 0. (6) The region sample labels are vector data and there are no topological errors such as face overlap, sharp corners and face cracks.

[0029] 3. Sample data collection and labeling; When collecting samples, it is necessary to collect all sample elements of this category in the sample area in accordance with the principle of "collecting all that should be collected" to ensure that there are no omissions; the sample patches must strictly comply with the boundary accuracy requirements of each type of sample land cover without obvious topological errors; for the sample vectors obtained by semi-automatic collection, the patch data that does not meet the sample collection accuracy requirements needs to be manually edited to meet the accuracy requirements.

[0030] The collected sample data, such as Figure 2 As shown, the file directories are organized and stored according to the collection tasks and sample area hierarchy: (1) First, collect the sample folders according to the task name. The folders are named after the task name and each sample area folder is stored in the folder.

[0031] (2) Each sample area folder is named after the sample area, and the data files of the sample collection area are stored in the folder.

[0032] (3) Sample collection data includes sample vector data, sample image data, sample metadata file, and sample collection effective range file.

[0033] Files are distinguished by sample names plus a suffix, and the file organization requirements are as follows: a. Sample vector files are in .shp format.

[0034] b. Sample image files use the common .tif or .img format. If the data source is an image service, the image service address is recorded in a file and saved as a .txt document.

[0035] c. Sample metadata is in .xml format.

[0036] d. The valid range file for sample collection stores the vector file of the sample area in .shp format.

[0037] The sample vector data includes various samples drawn on different images. The classification system is written in the form of fields in the vector data. The metadata description mainly explains what the classification fields are and how the categories are represented. For example, the classification field is "type", where 1 represents sugarcane, 2 represents mulberry, 3 represents eucalyptus, and 4 represents banana, etc.

[0038] 4. Sample Quality Inspection: Throughout the sample collection and preparation process, comprehensive quality control is implemented, including self-inspection, mutual inspection, and specialized inspection. 100% inspection of each step's output is conducted; only after the previous step's output has been verified as correct can the next step proceed. Ensure the sample attribute list is complete, sample patch boundaries are accurate and reasonable, low-quality data is removed, and the sample area image is free from cloud and snow cover and interference factors. For data with quality issues identified during sample collection quality inspection, the error type and location must be recorded, and the data returned to the sample collection workstation for correction until all quality inspection requirements are met, thus completing the sample quality inspection.

[0039] 5. Sample image data, the specific processing is as follows: The pixel depth of UAV DOM imagery may vary and may not meet the input requirements of deep learning models. Therefore, a unified bit depth conversion is needed to transform it into 8-bit imagery data. This embodiment of the invention employs a linear stretching method, normalizing the pixels distributed in [inputmin, inputmax] within each band, and then uniformly stretching them linearly to [outputmin, outputmax]. The formula is as follows: ; In the formula, value(x,y) represents each pixel in the band; when inputmin and inputmax are the minimum and maximum values ​​of the band pixel statistics, it is called extreme value stretching; when inputmin and inputmax are the pixel values ​​at the cumulative percentile position of the pixel histogram, it is called percentile stretching; outputmin and outputmax are the range of pixel values ​​for the stretched output, and when they are 0 and 255, it is called 8-bit deep output; result is the stretched output result.

[0040] 6. Sample set construction, specifically: (1) Define the requirements for sample set creation: Sample size: The sample size needs to take into account the features of different sample categories and the structure of deep learning networks, and is set to a regular square size with a side length of no less than 512 pixels; Sample overlap: Sample overlap represents the degree of overlap between individual sample pieces, and is set to half the sample size; Valid percentage of samples: The valid percentage of samples is the proportion of the specified sample category area in the label piece to the size of the entire sample piece. During production, sample pieces with a valid percentage higher than 5% are selected. Label Category: The category labeling of the label patch is based on standard onehot encoding. If the sample is a single-category land cover sample, label 0 is the background and 1 is the target land cover. If the sample is a multi-category land cover sample, the labels are numbered starting from 0, where 0 is the background and 1 is the label for the first category, and so on. Sample segment name: The sample segment name is usually composed of the sample set name + sample segment number. The sample segment number starts from 1, and is padded at the beginning if the number of digits is insufficient. Sample data format: Image patches are in .tif or .jpg format and must be accompanied by a .aux.xml file with the same name to record the spatial range information of the patch and ensure spatial positioning of the data; Label patches are in .png format and must be accompanied by a .aux.xml file with the same name to record the spatial range information of the label patch and ensure spatial positioning. (2) Sample set creation: Based on the labeled samples of each category, independently create a sample set for each category; (3) Sample set quality check: After the sample set is completed, the sample set data needs to be checked for quality. If there are problems with the quality check of the sample set data, the sample set data should be modified or regenerated until all quality check requirements are met and the sample quality check is completed. The sample set quality check items mainly include spatial reference system, metadata, and logical consistency content. (4) Sample set results organization: The sample set consists of n sets of sample patches and sample set metadata; Sample patch: A set of sample patches includes image patches, tag patches and patch metadata; Image patches are regular-sized image data after cropping, and surface change image patches include two image patches from the previous and previous periods; Tag patches are: rasterized sample category tag data corresponding to the image patches; Patch metadata is: records of relevant sample patch attribute information, including the name of a single set of patch data and the effective proportion of the sample; Sample set metadata is: records of the correspondence of the entire sample set and the corresponding image information and classification information of the sample set metadata information, so as to facilitate database construction and management, and the sample set metadata file is in .xml format.

[0041] like Figure 3 As shown, the sample set is organized and stored in the form of folders. Image patches, tag patches, and patch metadata are stored in different folders with the same name. The sample set is named with the acquisition task + sample area.

[0042] Step 3, model training and optimization, includes the following steps: (1) Deep Learning Network Structure Construction: The model is a deep learning network framework for UAV DOM data constructed using the SegFormer semantic segmentation model. The model mainly includes a hierarchical Transformer encoder module and a lightweight multilayer perceptron decoding module to generate the final semantic segmentation mask. Based on the SegFormer construction strategy, the model is adapted to three-band or multi-band UAV data by modifying and adding parameters when reading images. The SegFormer semantic segmentation model is used to identify different land cover types such as corn, sugarcane, rice, eucalyptus, fir, citrus, mulberry, banana, and camellia. The deep learning network framework for UAV DOM data is constructed using the SegFormer semantic segmentation model. The network structure is as follows: Figure 4 As shown.

[0043] The layered Transformer encoder module is specifically as follows: The Transformer encoder module is used to input an image of a given resolution. The image has a resolution of [resolution value], and the output resolution is [resolution value]. Hierarchical feature map , ;in The layered feature map contains coarse and fine features at different resolutions, corresponding to the channel dimension of the feature map. A key component of the hierarchical Transformer encoder module is the self-attention layer. In the original multi-head self-attention process, each head... , , Having the same dimensions ,in Let be the length of the sequence, then the estimated value of self-attention is: ; Its computational complexity is To reduce the computational complexity of self-attention estimation at high resolution, the model uses a reduction ratio. To reduce the sequence length, the formula is as follows: ; ; in, The sequence to be reduced, It is Reshape into shape sequence, It is to put one A dimensional tensor is used as input and outputs a Linear layers of dimensional tensors, new Dimension is This reduces the complexity of the self-attention mechanism to ; The model mixes in the feedforward network The convolutional and MLP components form the Mix-FNN module to provide positional information for the Transformer. The Mix-FNN can be defined as follows: ; In the formula, These are features derived from the self-attention module. It is an activation function based on Gaussian error.

[0044] The lightweight multilayer perceptron decoding module processes feature maps at different resolutions output by the Transformer encoder and obtains the final segmentation mask. This mainly includes four processes: 1) Unifying the channel dimensions of the feature maps at different resolutions output by the Transformer encoder using an MLP layer; 2) Upsampling the feature maps processed in step 1) to one-quarter of their original size and concatenating them; 3) Fusing the concatenated feature maps using an MLP layer; 4) Predicting the final segmentation mask using the fused feature map output in step 3) through another MLP layer. This process can be represented as: ; ; ; ; In the formula, Multi-layer feature maps at different resolutions The number of channel dimensions after feature map unification. This is the feature map after unifying the channel dimensions. This is the feature map after upsampling. This is the spliced ​​and merged feature map. For the output category, This is the final segmentation mask.

[0045] (2) Model Training: The training samples for the model need to contain as much diverse ground feature information as possible, so that the model can learn more ground feature information and improve the model's ability to detect complex ground features. During the training process, the input ground feature samples will pass through each layer of the model to generate a pixel-level segmentation result with the same size as the input image. The loss value between the result and the ground truth is calculated through the loss function. Subsequently, backpropagation is used to modify and update the weight parameters. During this process, the various running parameters and system resources in the algorithm model can be configured according to the sample dataset so that the model can be used for training on the current sample data, achieving faster training speed and higher training accuracy. For example, training parameters such as batch_size and learn_rate can be adjusted to obtain the optimal weight file.

[0046] (3) Model prediction: The test data randomly segmented from the sample dataset is input into the trained network model to obtain the recognition results of the changing regions in the test data; the test dataset is randomly sampled from the sample dataset. The performance of the model is evaluated by parameters such as accuracy and recall in the prediction results, the error is analyzed, and the model with the highest prediction accuracy is selected.

[0047] (4) Evaluation of model result accuracy: 10% of the test samples were randomly selected from the constructed sample library, and the accuracy of the algorithm was evaluated by the F1 score, specifically as follows: ; Where tp represents the number of samples correctly predicted as positive; fp represents the number of false positives, i.e. the number of negative samples incorrectly predicted as positive; and fn represents the number of false negatives, i.e. the number of positive samples incorrectly predicted as negative. (5) Model Iteration and Optimization: After the sample data and model parameters are processed, the entire algorithm model is trained. Based on the loss, training accuracy and other information fed back during the model training process, the model structure and model parameters are readjusted and the training is run again. By repeatedly adjusting the training model, sample data or model hyperparameters, the training results are made to meet the accuracy requirements. The model with the best training effect is saved for testing. Based on the accuracy evaluation of the results, actual needs and sample supplementation, the model is continuously iterated and optimized until the accuracy of the model results can be used for actual production. Step four, automatic extraction and post-processing; includes the following steps: (1) Automatic extraction of patches: Before using the semantic segmentation model for recognition, the UAV data to be predicted is sliced ​​to make its size match the input of the model; when the model is performing recognition, the optimal model weight file will be loaded first, and the sliced ​​data will be calculated and transformed step by step through each layer of the model to obtain a segmentation result with the same size as the input slice; the predicted slices are spliced ​​to obtain a semantic segmentation mask with the same size as the UAV data, and the result is vectorized to obtain the extracted patch vector result; (2) Post-processing of extraction results: After automatic extraction of patches from remote sensing images, noise may exist in the extracted patches, so post-processing of the results is required. It is necessary to eliminate predicted patch noise as much as possible, fill gaps in patches, and at the same time ensure that the predicted patch outlines are clear and connected. Gap filling algorithm and filtering algorithm are used for processing. The gap filling algorithm uses the idea of ​​morphological reconstruction to fill holes. The principle is as follows: ; Wherein, dst is the image after gap filling, Masker is the continuously dilating image, SE is the dilating structuring element, and Mask is the image constraining the dilating image.

[0048] The filtering process is designed to address the issue of isolated pixels in classified images. It utilizes a category-based filtering method to eliminate isolated classified pixels. If the number of pixels in a pixel is less than the input threshold, the pixel will be removed from that category.

[0049] (3) Manual verification and correction: Based on relevant business knowledge, manual optimization is used to ensure that the extracted results are free of erroneous map features and that the map feature boundaries are accurate, ultimately forming the final automated extraction results. The manual optimization work mainly includes: deleting erroneous map features, supplementing missing map features, and modifying the boundaries of correct map features.

[0050] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for intelligent identification of non-grain crops in cultivated land based on high-resolution remote sensing imagery, characterized in that, Includes the following steps: Step 1, Image Acquisition and Processing; includes the following steps: Delineation of areas for identifying non-grain use of arable land; Delineation of crop sample calibration and collection areas: Delineating crop sample calibration and collection area units; Orthophoto production operation: On a quarterly basis, aerial photography data with pixel size of 0.1 meters to 0.03 meters is collected in different regions, at different times and under different weather conditions within the crop sample calibration collection area. Based on the crop sample calibration collection area, aerial triangulation modeling and mosaicking are performed to produce sample images. Step two, sample library construction, includes the following steps: determining sample production specifications, establishing sample classification, constructing sample interpretation rules, and enriching sample image data; sample annotation, labeling non-grain crop target categories on image slices according to sample annotations; sample data collection and annotation; sample quality inspection; sample image data processing, uniformly performing bit depth conversion on sample images to convert them into 8-bit depth image data; sample set construction. Step 3, Model Training and Optimization, includes the following sub-steps: (1) Deep learning network structure construction: The model is a deep learning network framework for UAV DOM data constructed using the SegFormer semantic segmentation model; the model mainly includes a hierarchical Transformer encoder module and a lightweight multilayer perceptron decoding module, which are used to generate the final semantic segmentation mask; according to the SegFormer construction strategy, the model is adapted to three-band or multi-band UAV data by modifying and adding the parameters of the model when reading images. (2) Model training: During the training process, the input ground object samples will pass through each layer of the model to generate a pixel-level segmentation result with the same size as the input image. The loss value between the result and the true value is calculated through the loss function. Subsequently, backpropagation is used to modify and update the weight parameters. During this process, the various running parameters and system resources in the algorithm model can be configured according to the sample dataset so that the model can be used for training on the sample data, achieving faster training speed and higher training accuracy. (3) Model prediction: The test data randomly segmented from the sample dataset is input into the trained network model to obtain the recognition results of the changing regions in the test data; the test dataset is randomly sampled from the sample dataset. The performance of the model is evaluated by parameters such as accuracy and recall in the prediction results, the error is analyzed, and the model with the highest prediction accuracy is selected. (4) Evaluation of model result accuracy: 10% of the test samples were randomly selected from the constructed sample library, and the accuracy of the algorithm was evaluated by the F1 score, specifically as follows: ； Where tp represents the number of samples correctly predicted as positive; fp represents the number of false positives, i.e. the number of negative samples incorrectly predicted as positive; and fn represents the number of false negatives, i.e. the number of positive samples incorrectly predicted as negative. (5) Model Iteration and Optimization: After the sample data and model parameters are processed, the entire algorithm model is trained. Based on the loss, training accuracy and other information fed back during the model training process, the model structure and model parameters are readjusted and the training is run again. By repeatedly adjusting the training model, sample data or model hyperparameters, the training results are made to meet the accuracy requirements. The model with the best training effect is saved for testing. Based on the accuracy evaluation of the results, actual needs and sample supplementation, the model is continuously iterated and optimized until the accuracy of the model results can be used for actual production. Step four, automatic extraction and post-processing; includes the following sub-steps: (1) Automatic extraction of patches: Before using the semantic segmentation model for recognition, the UAV data to be predicted is sliced ​​to make its size match the input of the model; when the model is performing recognition, the optimal model weight file will be loaded first, and the sliced ​​data will be calculated and transformed step by step through each layer of the model to obtain a segmentation result with the same size as the input slice; the predicted slices are spliced ​​to obtain a semantic segmentation mask with the same size as the UAV data, and the result is vectorized to obtain the extracted patch vector result; (2) Post-processing of extraction results: After the remote sensing image is automatically extracted, there may be noise in the extracted patches. The results need to be post-processed to eliminate the noise of the predicted fragments as much as possible, fill the gaps in the patches, and at the same time ensure that the outline of the predicted patches is clear and connected. The gap filling algorithm and the filtering algorithm are used for processing. (3) Manual verification and correction: Based on relevant business knowledge, manual optimization is used to ensure that the extracted results are free of misidentified patches and that the boundaries of the patches are accurate, ultimately forming the final results of automated extraction.

2. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, The main technical requirements for the sample images are as follows: a. The overall image has a consistent tone, moderate contrast, natural transitions, clear textures, rich layers, and no obvious distortion; b. The image can accurately reflect the spectral characteristics of land cover types. c. The image shows no obvious distortion or loss of features, no ghosting or blurring, and the boundaries of features are clear; d. The transitions at the image mosaicking and edge joints are natural, and the same feature should be kept intact; e. The image bit depth can be 8-bit, 16-bit, or 32-bit; f. The image color mode can be panchromatic, color, or multispectral; g. The NoData area of ​​the image is set to 0.

3. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, Step two, which determines the specific sample preparation specifications, is as follows: (1) Sample classification system: In line with the goal of the "non-grain" identification work, several non-grain crops were selected to establish a sample classification; (2) Constructing sample interpretation rules: refine the scene, determine the correspondence between the features of ground objects on the captured images and the actual ground object categories based on the results of the field survey, and then determine the interpretation rules through the image features; for categories that cannot be determined, field surveys are required to determine them. (3) Sample image data augmentation: The sample selection should be evenly distributed across different dimensions as much as possible, and additional data should be added for difficult cases; the dimensions include season, terrain, and category; the data source used is aerial photography data from 0.1 meters to 0.03 meters. (4) Sample iteration and supplementation: Adopt the production and accumulation mode, check the production results simultaneously and analyze the omission and error of extraction, make supplementary positive and negative samples in a targeted manner, and then introduce the new samples into the training to update the production model and iterate. (5) Sample image data processing: uniformly perform bit depth conversion to convert it into 8-bit depth image data.

4. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, The main technical requirements for sample annotation in step two are as follows: (1) Sufficient samples need to be labeled for each type of non-grain crop; when labeling based on image slices, due to inconsistent image resolution, each image slice must be at least 3000*3000 in size. The target categories of non-grain crops on the image slices must be labeled without error or omission. (2) Within the same area of ​​the sample, there should be no obvious errors or omissions in the label collection. Within the same area of ​​the sample, the sum of the areas of the erroneous or omitted patches should not exceed 5% of the total area. (3) Within the same region, the error rate of sample attribute assignment should not exceed 5%; (4) The boundary between the sample label and the target land type should be aligned. For artificial features with obvious boundaries or targets with small areas, the alignment difference should not exceed 2 pixels. For natural features with indistinct boundaries, the alignment difference should not exceed 5 pixels. (5) The sample labels are raster data with a bit depth of 8 bits. The label index value of the target land type is an integer from 1 to 255, and the label index value of the non-target land type is 0. (6) The region sample labels are vector data and there are no topological errors such as face overlap, sharp corners and face cracks.

5. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, The specific steps for processing the sample image data in step two are as follows: A linear stretching method is used to normalize the pixels distributed in [inputmin, inputmax] in each band, and then uniformly stretch them linearly to [outputmin, outputmax]. The formula is as follows: ； In the formula, value(x,y) represents each pixel in the band; when inputmin and inputmax are the minimum and maximum values ​​of the band pixel statistics, it is called extreme value stretching; when inputmin and inputmax are the pixel values ​​of the cumulative percentile position of the pixel histogram, it is called percentile stretching. `outputmin` and `outputmax` are the range of pixel values ​​for the stretched output. When they are 0 and 255 respectively, it is an 8-bit deep output. `result` is the stretched output result.

6. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, The specific steps for constructing the sample set in step two are as follows: (1) Define the requirements for sample set creation: Sample size: The sample size needs to take into account the features of different sample categories and the structure of deep learning networks, and is set to a regular square size with a side length of no less than 512 pixels; Sample overlap: Sample overlap represents the degree of overlap between individual sample pieces, and is set to half the sample size; Valid percentage of samples: The valid percentage of samples is the proportion of the specified sample category area in the label piece to the size of the entire sample piece. During production, sample pieces with a valid percentage higher than 5% are selected. Label Category: The category labeling of the label patch is based on standard onehot encoding. If the sample is a single-category land cover sample, label 0 is the background and 1 is the target land cover. If the sample is a multi-category land cover sample, the labels are numbered starting from 0, where 0 is the background and 1 is the label for the first category, and so on. Sample segment name: The sample segment name is usually composed of the sample set name + sample segment number. The sample segment number starts from 1, and is padded at the beginning if the number of digits is insufficient. Sample data format: Image patches are in .tif or .jpg format. .jpg format requires an accompanying .aux.xml file with the same name to record the spatial range information of the patch and ensure spatial positioning of the data; Label patches are in .png format and require an accompanying .aux.xml file with the same name to record the spatial range information of the label patch and ensure spatial positioning. (2) Sample set creation: Based on the labeled samples of each category, independently create a sample set for each category; (3) Sample set quality check: After the sample set is completed, the sample set data needs to be checked for quality. If there are problems with the quality check of the sample set data, the sample set data should be modified or regenerated until all quality check requirements are met and the sample quality check is completed. The sample set quality check items mainly include spatial reference system, metadata, and logical consistency content. (4) Sample set results organization: The sample set consists of n sets of sample fragments and sample set metadata; Sample Patch: A set of sample patches includes image patches, tag patches, and patch metadata. Image patches are regular-sized image data after cropping. Surface change image patches include two images from different periods. Tag patches are rasterized sample category label data corresponding to the image patches. Patch metadata records the attribute information of related sample patches, including the name of the individual patch data and the effective percentage of the sample. Sample set metadata records the correspondence of the entire sample set and the corresponding image and classification information for database management. The sample set metadata file is in .xml format.

7. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, In step three, the layered Transformer encoder module specifically consists of: The Transformer encoder module is used to input an image of a given resolution. The image has a resolution of [resolution value], and the output resolution is [resolution value]. Hierarchical feature map , ;in The layered feature map contains coarse and fine features at different resolutions, corresponding to the channel dimension of the feature map. A key component of the hierarchical Transformer encoder module is the self-attention layer. In the original multi-head self-attention process, each head... , , Having the same dimensions ,in Let be the length of the sequence, then the estimated value of self-attention is: ； Its computational complexity is To reduce the computational complexity of self-attention estimation at high resolution, the model uses a reduction ratio. To reduce the sequence length, the formula is as follows: ； ； in, The sequence to be reduced, It is Reshape into shape sequence, It is to put one A dimensional tensor is used as input and outputs a Linear layers of dimensional tensors, new Dimension is This reduces the complexity of the self-attention mechanism to ; The model mixes in the feedforward network The convolutional and MLP components form the Mix-FNN module to provide positional information for the Transformer. The Mix-FNN can be defined as follows: ； In the formula, These are features derived from the self-attention module. This is an activation function based on Gaussian error.

8. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, In step three, the lightweight multilayer perceptron decoding module processes the feature maps at different resolutions output by the Transformer encoder and obtains the final segmentation mask. This mainly includes four processes: (1) unifying the channel dimensions of the feature maps at different resolutions output by the Transformer encoder through an MLP layer; (2) upsampling the feature maps processed in step (1) to one-quarter of their original size and stitching them together; (3) fusing the stitched feature maps using an MLP layer; and (4) predicting the final segmentation mask using the fused feature map output in step (3) through another MLP layer. The above process can be represented as follows: ； ； ； ； In the formula, Multi-layer feature maps at different resolutions The number of channel dimensions after feature map unification. This is the feature map after unifying the channel dimensions. This is the feature map after upsampling. This is the spliced ​​and merged feature map. For the output category, This is the final segmentation mask.

9. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, In step four, the gap-filling algorithm utilizes the idea of ​​morphological reconstruction to fill the holes. The principle is as follows: ； Wherein, dst is the image after gap filling, Masker is the continuously dilating image, SE is the dilating structuring element, and Mask is the image constraining the dilating image.

10. The intelligent identification method for non-grain crops in cultivated land based on high-resolution remote sensing imagery according to claim 1, characterized in that, In step four, the filtering process is to address the issue of isolated pixels in the classified images; the category filtering method is used to eliminate isolated classified pixels; if the number of pixels in a patch is less than the input threshold, the patch will be removed from that category.

Images