A regional scale raster grid segmentation method based on ArcPy

Abstract

This invention discloses a regional-scale raster grid segmentation method based on ArcPy, belonging to the field of geospatial data processing. The method comprises the following steps: Step 1: Creating fishnet data; Step 2: Segmenting administrative district vector data; Step 3: Extracting fishnet grids by administrative district; Step 4: Segmenting fishnet grids by administrative district; and Step 5: Clipping raster grids by administrative district fishnet grids. The invention significantly improves the efficiency of fine-grained segmentation of raster data at the regional scale, ensuring that the segmented raster grid data has accurate boundaries and geographic coordinate systems, facilitating researchers' further data analysis and application.

Description

Technical Field

[0001] The present invention relates to the field of geospatial data processing, and in particular to a regional scale raster grid segmentation method based on ArcPy. Background Art

[0002] In geospatial data processing and analysis, it is often necessary to segment the entire raster data into a series of grid images according to a preset grid size. For example, remote sensing images captured by drones are usually large in size and usually need to be split into grid images of uniform size for easy storage. When calculating landscape pattern indices, it is usually necessary to segment landscape type data into ecological risk areas according to a preset grid size. When predicting climate change, it is usually necessary to split multiple years of ground meteorological data into grid images of fixed size to reduce the consumption of computer memory when reading data. It can be seen that raster grid segmentation is widely involved in many fields.

[0003] Currently, the main methods for raster grid segmentation include software and programmatic methods. The software method uses image processing software to segment raster data at a specified spatial resolution. Conventional image processing software provides a ruler and grid for setting grid size, and clipping tools can be used to clip raster data into grids. However, this method is labor-intensive and can typically only clip small, spatially discontinuous raster grids. Furthermore, the boundaries of the clipped grid data cannot be accurately ensured, and the data lacks a geographic coordinate system. Geographic information system software provides fishing net tools, clipping tools, and modeling tools to segment raster data to ensure spatial continuity based on grid size. While this method reduces the workload, it is limited in the amount of data it can process, and the resulting iterative clipping model can only be used for small-scale raster data and coarse-grained raster grid segmentation. The programmatic method uses existing image processing libraries, such as GDAL, OpenCV, and Pillow, to create programs in computer programming languages ​​to segment raster grids. This method is highly targeted, but the programming threshold is high, requiring strong professional knowledge and programming skills. The image processing libraries it relies on are cumbersome to install, and the quality of the final segmented data (including the geographic coordinate system) is closely related to the quality of the written program. For most researchers, especially those who are not proficient in programming, starting from scratch is not an effective method.

[0004] It can be seen that the performance of existing grid segmentation methods is difficult to meet the needs of researchers when processing regional or continental scale raster data. There is an urgent need to propose a simple and efficient segmentation method to address the shortcomings of existing methods. Summary of the Invention

[0005] In response to the problems existing in the above-mentioned prior art, the present invention provides a regional-scale raster grid segmentation method based on ArcPy to improve the efficiency of fine-grained segmentation of raster grids at the regional scale or even the continental scale, ensure the quality of the segmentation results, and effectively solve the problems of conventional methods such as excessive memory consumption, inaccurate data boundaries, and lack of or inaccurate geographic coordinates when processing regional-scale or continental-scale data.

[0006] To achieve the above object, the present invention adopts a technical solution: a regional scale grid segmentation method based on ArcPy, comprising the following steps:

[0007] Step 1: Create fishing net data;

[0008] Step 2: Segment administrative district vector data;

[0009] Step 3: Extract fishing net grids by administrative district;

[0010] Step 4: Divide the fishing net grid by administrative district;

[0011] Step 5: Clip the raster according to the administrative district fishing net grid.

[0012] Preferably, the step 1 is specifically:

[0013] (1) Collect raster data to be segmented and administrative district vector data

[0014] Collect regional scale raster data to be segmented with geographic coordinates and corresponding administrative district vector data, and import them into geographic information system software respectively, wherein the raster data to be segmented refers to the raster data in the geographic information system;

[0015] (2) Create a fishing net

[0016] Use the "Data Management Tools" -> "Feature Class" -> "Create Fishnet" tool in the geographic information system software to generate fishnet polygon data, where the template range is set to "Same as Administrative District Vector Data", the pixel width is set to "W", the pixel height is set to "H", and the geometry type is set to "Polygon". W and H are integer multiples of the spatial resolution of the raster data to be segmented.

[0017] (3) Remove invalid grids from the fishing net

[0018] Use the "Select" -> "Select by Location" tool in the geographic information system software to select the fishing net data grid that intersects with the administrative district vector data; then export the selected grid data as corresponding vector data and name it net_pure.shp;

[0019] (4) Add fields to fishing net data

[0020] Use geographic information system software to open the attribute data of the fishing net data net_pure.shp with invalid grids removed, add a field name "pid" to it, and the field type is string; use the field calculator tool to batch assign values ​​for pid, and the general format of pid is agreed to be: "p" + corresponding feature code, such as "p10".

[0021] Preferably, the step 2 is specifically as follows:

[0022] (1) Add fields for administrative district data

[0023] Use geographic information system software to open the attribute data of administrative district vector data, add a field name "cid" to it, and set the field type to string; use the field calculator tool to batch assign values ​​to cid, and the general format of cid is agreed to be: "c" + corresponding administrative code, such as "c310000";

[0024] (2) Create a geographic database of segmentation results

[0025] Create a new file geodatabase using geographic information system software and name it city_split.gdb;

[0026] (3) Administrative district data segmentation

[0027] Use the "Analysis Tools" -> "Extraction Analysis" -> "Split" tool of the geographic information system software to split the administrative district vector data according to the split field "cid"; the segmentation results are saved in the file geographic database city_split.gdb.

[0028] Preferably, the step three is specifically as follows:

[0029] (1) Create a geographic database to store the block fishing net

[0030] First, determine whether the geographic database net_city_split.gdb used to save the block fishing net exists. If it exists, use the arcpy.Delete_management() function to delete the geographic database; if it does not exist, use the arcpy.CreateFileGDB_management() function to create the geographic database;

[0031] (2) Read the administrative district geographic database

[0032] Use arcpy.da.Walk() function to read the administrative district geographic database city_split.gdb, traverse the polygon feature set in the database, and return a walk containing data sets such as path, directory name, and file name;

[0033] (3) Extract fishing net grids by administrative district location

[0034] First, use a for loop to traverse the path, directory name, file name and other data in the administrative district geographic database, and then use the arcpy.Exists() function to determine whether the layer data net_lyr exists. If it exists, use the arcpy.Delete_management() function to delete the layer data; if it does not exist, use the arcpy.MakeFeatureLayer_management() function to convert the fishing net data net_pure.shp into the layer data net_lyr; then, use the arcpy.SelectLayerByLocation_management() function to select the grid data from the layer data net_lyr according to the current administrative district location; finally, use the arcpy.CopyFeatures_management() function to save the selected grid data in the layer data net_lyr to the specified geographic database.

[0035] Preferably, the step 4 is specifically as follows:

[0036] (1) Read the administrative district fishing net geographic database

[0037] Use arcpy.da.Walk() function to read the administrative district fishing net geographic database net_city_split.gdb, traverse the polygon feature set in the database, and return a walk containing data sets such as path, directory name, and file name;

[0038] (2) Divide the administrative district fishing net grid

[0039] First, use a for loop to iterate over the paths, directory names, and file names in the administrative district fishnet geodatabase. Then, set the geodatabase file name tmp_gdb_name and the storage location tmp_gdb_out to store the split fishnet. Use the arcpy.Exists() function to determine whether the geodatabase exists. If so, use the arcpy.Delete_management() function to delete it. If not, use the arcpy.CreateFileGDB_management() function to create a new geodatabase. Finally, use the arcpy.Split_analysis() function to split the fishnet grid in the current administrative district according to the split field "pid". The split results are saved in the file geodatabase tmp_gdb_name.

[0040] Preferably, the step five is specifically as follows:

[0041] (1) Define the fishnet clipping raster function

[0042] Define the function extract_raster_by_net() to clip the raster using the fishnet grid. The input parameters include the raster data to be clipped raster_file, the fishnet data net_gdb used for clipping, and the clipped raster output path raster_out_path. The implementation steps of the function include: first, using the arcpy.da.Walk() function to traverse the features in the fishnet database net_gdb used for clipping, and returning a walk containing a set of database path, directory name, and file name; then, setting the directory for saving the clipped raster and using the os.mkdir() function to create the directory; then, using the for outer loop to traverse the walk, and using the for inner loop to traverse the file name, and finally using the arcpy.gp.ExtractByMask_sa() function to use the current fishnet grid data to clip the raster image and save it;

[0043] (2) Perform the cropping operation

[0044] Set the location of the raster data to be clipped, the location of the fishnet geographic database used to clip the raster, and the output path of the raster data after clipping according to the fishnet grid; then use the get_all_gdbs() function to obtain all the geographic databases at the specified location; then use a for loop to traverse the database, and finally use the extract_raster_by_net() function to clip the raster data using the fishnet grid and save it.

[0045] Compared with some existing raster data segmentation methods, the beneficial effects of the present invention include the following points.

[0046] (1) The present invention significantly improves the efficiency of fine-grained segmentation of raster grids at the regional scale. By downscaling regional-scale raster data according to administrative district fishing nets, the present invention reduces the consumption of ordinary computer memory during data processing and avoids errors such as memory overflow or insufficient memory.

[0047] (2) The present invention ensures the quality of the segmented raster data. Based on ArcPy's standard interface for segmenting raster data, the present invention ensures that the segmented raster data is accurate in terms of segmentation boundaries and data content, while also ensuring that each raster data has an accurate geographic coordinate system. This greatly facilitates researchers in conducting further data analysis and data application work. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the technical process of the present invention.

[0049] Figure 2 This is a rendering of the basic data collected in an embodiment of the present invention.

[0050] Figure 3 Create a fishing net effect diagram in an embodiment of the present invention.

[0051] Figure 4 This is a diagram showing the effect of dividing administrative regions in an embodiment of the present invention.

[0052] Figure 5 This is an effect diagram of extracting fishing net grids according to administrative district locations in an embodiment of the present invention.

[0053] Figure 6 This is a diagram showing the effect of a fishing net grid for dividing administrative districts in an embodiment of the present invention.

[0054] Figure 7 This is a grid effect diagram of the segmentation grid in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. However, it should be understood that the specific embodiments described herein are only used to illustrate the present invention and are not intended to limit the scope of the present invention.

[0056] Unless otherwise defined, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention pertains. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0057] The technical solution of the present invention is further described in detail below with reference to specific embodiments and drawings. Specifically, the present invention is a regional scale grid segmentation method based on ArcPy. The technical process is as follows: Figure 1 As shown, the specific steps include:

[0058] Step 1: Create fishnet data

[0059] (1) Collect raster data to be segmented and administrative district vector data

[0060] Collect regional scale raster data to be segmented with geographic coordinates and corresponding administrative district vector data, and import them into geographic information system software respectively. The raster data to be segmented refers to raster data in the geographic information system, such as meteorological / climate data, land use / land cover data, remote sensing data, ecosystem data, soil data, vegetation data, air quality data, topography / geomorphology / geology data, productivity data, etc. In this method, land use / land cover data is taken as an example. The level of administrative district vector data can be city level, district level or county level. In this method, city level is taken as an example. The basic data collected in this embodiment is as follows: Figure 2 shown.

[0061] (2) Create a fishing net

[0062] Use the "Data Management Tools" -> "Feature Class" -> "Create Fishnet" tool in the geographic information system software to generate fishnet polygon data, where the template range is set to "Same as Administrative District Vector Data", the pixel width is set to "W", the pixel height is set to "H", and the geometry type is set to "Polygon". W and H are integer multiples of the spatial resolution of the land use / land cover raster data. The initial fishnet data created in this embodiment is as follows Figure 3 As shown in a.

[0063] (3) Remove invalid grids from the fishing net

[0064] Use the "Select" -> "Select by Location" tool in the geographic information system software to select the fishing net data grid that intersects with the administrative district vector data. Then export the selected grid data as corresponding vector data and name it net_pure.shp. Figure 3 As shown in b.

[0065] (4) Add fields to fishing net data

[0066] Use geographic information system software to open the attribute data of the fishing net data net_pure.shp with invalid grids removed, and add a field name "pid" to it, with the field type as string. Use the field calculator tool to batch assign values ​​for pid. The general format of pid is agreed to be: "p" + corresponding element code, such as "p10". In this embodiment, the effect of adding fields to the fishing net data is as follows: Figure 3 As shown in c.

[0067] Step 2: Segment administrative district vector data

[0068] (1) Add fields for administrative district data

[0069] Use geographic information system software to open the attribute data of the administrative district vector data and add a field named "cid" with a string type. Use the field calculator tool to batch assign values ​​for CID. The general format of CID is: "c" + the corresponding administrative code, such as "c310000".

[0070] (2) Create a geographic database of segmentation results

[0071] Use geographic information system software to create a new file geographic database and name it city_split.gdb.

[0072] (3) Administrative district data segmentation

[0073] Use the "Analysis Tools" -> "Extraction Analysis" -> "Split" tool in the geographic information system software to split the administrative district vector data according to the segmentation field "cid". The segmentation results are saved in the file geographic database city_split.gdb. The data effect after executing the segmentation of administrative districts in this embodiment is as follows Figure 4 shown.

[0074] Step 3: Extract fishing net grids by administrative district

[0075] (1) Create a geographic database to store the block fishing net

[0076] First, determine whether the geodatabase net_city_split.gdb used to store the split fishing net exists. If it does, use the arcpy.Delete_management() function to delete the geodatabase. If it does not exist, use the arcpy.CreateFileGDB_management() function to create the geodatabase. The relevant code for this step is:

[0077] net_city_gdb_path=". / "

[0078] net_city_gdb_name="net_city_split.gdb"

[0079] net_city_gdb_out=net_city_gdb_path+' / '+net_city_gdb_name#Save the database of the block fishing net

[0080] if arcpy.Exists(net_city_gdb_out):#Determine whether the geographic database exists

[0081] print("\tWarning: File geodatabase "+net_city_gdb_out+" already exists!")

[0082] print("\tPrepare to delete the file geodatabase...")

[0083] arcpy.Delete_management(net_city_gdb_out)#Delete database

[0084] ####

[0085] arcpy.CreateFileGDB_management(net_city_gdb_path,net_city_gdb_name)#Create database.

[0086] (2) Read the administrative district geographic database

[0087] Use the arcpy.da.Walk() function to read the administrative district geographic database city_split.gdb, traverse the polygon feature set in the database, and return a walk containing a data set of paths, directory names, and file names. The relevant code for this step is:

[0088] walk = arcpy.da.Walk("city_split.gdb", datatype = "Any", type = "Polygon") # Traverse the elements in the database;

[0089] (3) Extract fishing net grids by administrative district location

[0090] First, use a for loop to traverse the path, directory name, file name and other data in the administrative district geographic database, and then use the arcpy.Exists() function to determine whether the layer data net_lyr exists. If it exists, use the arcpy.Delete_management() function to delete the layer data. If it does not exist, use the arcpy.MakeFeatureLayer_management() function to convert the fishing net data net_pure.shp into the layer data net_lyr. Next, use the arcpy.SelectLayerByLocation_management() function to select the grid data from the layer data net_lyr according to the current administrative district location. Finally, use the arcpy.CopyFeatures_management() function to save the grid data selected in the layer data net_lyr to the specified geographic database. The relevant code for this step is:

[0091]

[0092] After the above steps, the effect of extracting fishing net grids by administrative district location is as follows Figure 5 shown.

[0093] Step 4: Divide the fishing net grid by administrative district

[0094] (1) Read the administrative district fishing net geographic database

[0095] Use the arcpy.da.Walk() function to read the administrative district fishnet geographic database net_city_split.gdb, traverse the polygon feature set in the database, and return a walk containing a data set of paths, directory names, and file names. The relevant code for this step is:

[0096] walk=arcpy.da.Walk("net_city_split.gdb",datatype="Any",type="Polygon")#Traverse the Features in the database

[0097] (2) Divide the administrative district fishing net grid

[0098] First, use a for loop to traverse the path, directory name, file name and other data in the administrative district fishnet geodatabase, and then set the geodatabase file name tmp_gdb_name and storage location tmp_gdb_out to store the fishnet segmentation. Use the arcpy.Exists() function to determine whether the geodatabase exists. If it exists, use the arcpy.Delete_management() function to delete it. If it does not exist, use the arcpy.CreateFileGDB_management() function to create a new geodatabase. Finally, use the arcpy.Split_analysis() function to split the fishnet grid in the current administrative district according to the segmentation field "pid". The segmentation results are saved in the file geodatabase tmp_gdb_name. The relevant code for this step is:

[0099]

[0100]

[0101] After the above steps, the effect of dividing the fishing net grid by administrative district is as follows Figure 6 shown.

[0102] Step 5: Clip the raster to the administrative district fishing grid

[0103] (1) Define the fishnet clipping raster function

[0104] Define the function extract_raster_by_net() for clipping raster using fishnet grid. The input parameters include the raster data to be clipped raster_file, the fishnet data net_gdb used for clipping, and the clipped raster output path raster_out_path. The implementation steps of the function include: first, use the arcpy.da.Walk() function to traverse the features in the fishnet database net_gdb used for clipping, and return a walk containing a set of database path, directory name, and file name. Then set the directory for saving the clipped raster, and use the os.mkdir() function to create the directory. Next, use the for outer loop to traverse the walk, use the for inner loop to traverse the file name, and finally use the arcpy.gp.ExtractByMask_sa() function to use the current fishnet grid data to clip the raster image and save it. The relevant code for this operation is:

[0105]

[0106]

[0107] (2) Perform the cropping operation

[0108] Set the location of the raster data to be clipped, the location of the fishnet geodatabase used to clip the raster, and the output path of the raster data clipped by the fishnet grid. Then use the get_all_gdbs() function to obtain all geodatabases at the specified location. Next, use a for loop to iterate through the databases, and finally use the extract_raster_by_net() function to clip the raster data using the fishnet grid and save it. The relevant code for this step is:

[0109]

[0110]

[0111] After the above steps, the regional scale raster data is clipped according to the administrative district fishing net grid. Figure 7 This method significantly improves the efficiency of fine-grained segmentation of raster data at the regional scale, ensuring that the segmented raster grid data has accurate boundaries and geographic coordinates, making it easier for researchers to conduct further data analysis and data applications.

[0112] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A regional scale grid segmentation method based on ArcPy, characterized in that: The following steps are involved: Step 1: Create fishing net data; Step 2: Segment administrative district vector data; The step 2 is specifically as follows: (1) Add fields for administrative district data Use geographic information system software to open the attribute data of administrative district vector data, add a field name "cid" to it, and set the field type to string; use the field calculator tool to batch assign values ​​to cid, and the general format of cid is agreed to be: "c" + corresponding administrative code; (2) Create a segmentation result geodatabase Create a new file geodatabase using geographic information system software and name it city_split.gdb; (3) Split administrative district data Use the "Analysis Tools" -> "Extraction Analysis" -> "Split" tool in the geographic information system software to split the administrative district vector data according to the segmentation field "cid"; the segmentation results are saved in the file geographic database city_split.gdb; Step 3: Extract fishing net grids by administrative district; The step three is specifically as follows: (1) Create a geographic database to store the block fishing net First, determine whether the geographic database net_city_split.gdb used to save the block fishing net exists. If it exists, use the arcpy.Delete_management() function to delete the geographic database; if it does not exist, use the arcpy.CreateFileGDB_management() function to create the geographic database; (2) Read the administrative district geographic database Use arcpy.da.Walk() function to read the administrative district geographic database city_split.gdb, traverse the polygon feature set in the database, and return a walk containing data sets such as path, directory name, and file name; (3) Extract fishing net grids by administrative district location First, use a for loop to traverse the path, directory name, file name and other data in the administrative district geographic database, then use the arcpy.Exists() function to determine whether the layer data net_lyr exists. If it exists, use the arcpy.Delete_management() function to delete the layer data; if it does not exist, use the arcpy.MakeFeatureLayer_management() function to convert the fishing net data net_pure.shp into the layer data net_lyr; then, use the arcpy.SelectLayerByLocation_management() function to select the grid data from the layer data net_lyr according to the current administrative district location; finally, use the arcpy.CopyFeatures_management() function to save the selected grid data in the layer data net_lyr to the specified geographic database; Step 4: Divide the fishing net grid by administrative district; Step 5: Clip the raster according to the administrative district fishing net grid.

2. The method for regional scale grid segmentation based on ArcPy according to claim 1, characterized in that: The step 1 specifically comprises: (1) Collect raster data to be segmented and administrative district vector data Collect regional scale raster data to be segmented with geographic coordinates and corresponding administrative district vector data, and import them into geographic information system software respectively, wherein the raster data to be segmented refers to the raster data in the geographic information system; (2) Create a fishing net Use the "Data Management Tools" -> "Feature Class" -> "Create Fishnet" tool in the geographic information system software to generate fishnet polygon data. Set the template range to "Same as Administrative District Vector Data", the pixel width to "W", the pixel height to "H", and the geometry type to "Polygon". W and H are integer multiples of the spatial resolution of the land use / land cover raster data. (3) Remove invalid grids from the fishing net Use the "Select" -> "Select by Location" tool in the GIS software to select the fishing net data grid that intersects with the administrative district vector data; then export the selected grid data as corresponding vector data and name it net_pure.shp; (4) Add fields to fishing net data Use geographic information system software to open the attribute data of the fishing net data net_pure.shp with invalid grids removed, add a field name "pid" to it, and set the field type to string; use the field calculator tool to batch assign values ​​for pid, and the general format of pid is agreed to be: "p" + corresponding feature code.

3. The method for regional scale grid segmentation based on ArcPy according to claim 1, characterized in that: The step 4 is specifically as follows: (1) Read the administrative district fishing net geographic database Use arcpy.da.Walk() function to read the administrative district fishing net geographic database net_city_split.gdb, traverse the polygon feature set in the database, and return a walk containing data sets such as path, directory name, and file name; (2) Divide the administrative district fishing net grid First, use a for loop to traverse the path, directory name, file name and other data in the administrative district fishnet geodatabase, then set the geodatabase file name tmp_gdb_name and storage location tmp_gdb_out to store the fishnet segmentation; use the arcpy.Exists() function to determine whether the geodatabase exists. If it exists, use the arcpy.Delete_management() function to delete it; if it does not exist, use the arcpy.CreateFileGDB_management() function to create a new geodatabase; finally, use the arcpy.Split_analysis() function to split the fishnet grid in the current administrative district according to the segmentation field "pid"; The segmentation results are saved in the file geodatabase tmp_gdb_name.

4. The method for regional scale grid segmentation based on ArcPy according to claim 1, wherein: The step five is specifically as follows: (1) Define the fishnet clipping raster function Define the function extract_raster_by_net() to clip the raster using the fishnet grid. The input parameters include the raster data to be clipped raster_file, the fishnet data net_gdb used for clipping, and the clipped raster output path raster_out_path. The implementation steps of the function include: first, using the arcpy.da.Walk() function to traverse the features in the fishnet database net_gdb used for clipping, and returning a walk containing a set of database path, directory name, and file name; then, setting the directory for saving the clipped raster and using the os.mkdir() function to create the directory; then, using the for outer loop to traverse the walk, and using the for inner loop to traverse the file name, and finally using the arcpy.gp.ExtractByMask_sa() function to use the current fishnet grid data to clip the raster image and save it; (2) Perform cropping operations Set the location of the raster data to be clipped, the location of the fishnet geographic database used to clip the raster, and the output path of the raster data after clipping according to the fishnet grid; then use the get_all_gdbs() function to obtain all the geographic databases at the specified location; then use a for loop to traverse the database, and finally use the extract_raster_by_net() function to clip the raster data using the fishnet grid and save it.

Images