Hydrological network parameter automatic generation system based on multi-open source GIS platform integration

Abstract

The present application relates to the technical field of geographic information system (GIS) and hydrological modeling, more particularly to a hydrological network parameter automatic generation system based on multi-open source GIS platform integration, integrating multiple open source geographic information systems QGIS, PROJ, GDAL, GRASS GIS, SAGA NextGen and TauDEM, realizing zero-length river channel topology repair through an improved StreamNet algorithm, combining a dynamic soil depth model and a multi-format output engine, and significantly improving the efficiency and accuracy of data preparation of a watershed source element management model (CNMM). The present application enables a user to completely get rid of ESRI ARC GIS geographic information system which is strictly protected by American property rights, and can be widely applied in the fields of watershed management, hydrological and carbon-nitrogen cycle simulation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of geographic information system (GIS) and hydrological modeling, more particularly to a hydrological network parameter automatic generation system based on multi-open source GIS platform integration. BACKGROUND

[0002] Traditional hydrological network generation tools (such as ArcHydro) rely on a single ESRI ARC GIS platform, which has poor compatibility and high extension costs. Existing methods have deficiencies in river network topology optimization (such as 0-length river processing: four / five fork river problems), dynamic calculation of soil depth, and multi-format data collaborative output. There is a lack of special data processing procedures for CNMM (Catchment Management Model) models, which requires multiple manual intervention steps and is inefficient.

[0003] Patent CN118094968A proposes a dynamic watershed division method and system based on DEM data, but does not involve multi-tool integration and complex topology optimization. The TauDEM toolkit supports distributed hydrological analysis, but its native output format cannot directly adapt to the CNMM (Catchment Management Model) model requirements. SUMMARY

[0004] Therefore, the present application provides a hydrological network parameter automatic generation system based on multi-open source GIS platform integration, aiming to realize open source and cross-platform hydrological data processing, and to achieve one-key generation from raw DEM (Digital Elevation Model) to CNMM (Catchment Management Model) input data through an automated process, solving the problems of low efficiency and poor compatibility in the prior art.

[0005] To achieve the above purpose, the present application adopts the following technical solutions:

[0006] The hydrological network parameter automatic generation system based on multi-open source GIS platform integration comprises:

[0007] Integrating QGIS, TauDEM, GRASS, and SAGA open source tool chains, the system realizes collaborative processing through tool calling interfaces.

[0008] Topology repair is performed using a modified StreamNet algorithm, including deleting zero-length river segments and updating the USLINKNO3 field to resolve multi-branch river connection issues;

[0009] Soil depth is calculated based on slope data slpFile and flat-elevation data felFile using a dynamic weight formula.

[0010] The structured data files stream_map.dat and stream_network.dat for the CNMM model are output.

[0011] In one specific implementation, the modified StreamNet algorithm includes:

[0012] The river raster is extracted using the Threshold tool of TauDEM.

[0013] When converting the river raster to a vector network, river segments with a length less than 0.1 meters are automatically deleted.

[0014] For four-branch or five-branch river nodes, the USLINKNO3 field is activated in the topology relationship data to record the third upstream branch.

[0015] In one specific implementation, the dynamic weight formula is: soil depth = α × slope weight + β × elevation weight, where:

[0016] where α and β are weight coefficients.

[0017] α ∈ [0.6, 0.8], β = 1 - α.

[0018] Slope weight = 1 - normalized slope value.

[0019] Elevation weight = normalized elevation value.

[0020] In one specific implementation, the structured data files include:

[0021] stream_network.dat: records river segment ID, order ORDER, slope SLOPE, length LENGTH, and class CLASS.

[0022] stream_map.dat: records grid column Col, row Row, river channel projection length Segment Length, and direction Aspect.

[0023] In one specific implementation, the CLASS is divided into 18 categories according to the average catchment area meanmsq.

[0024] The ORDER adopts a recursive hierarchical algorithm, the source river section is level 1, and the downstream river section level is the maximum upstream level plus 1.

[0025] In one specific implementation, a watershed mask generation module is further included:

[0026] A binary mask mask.tif is generated based on the river grid srcFile generated by TauDEM.

[0027] The D8 flow direction data of TauDEM is converted into an ArcGIS compatible flow direction encoding format.

[0028] In one specific implementation, the soil depth calculation process includes:

[0029] The normalized slope value is calculated by the GRASS tool;

[0030] The normalized elevation value is calculated by the SAGA tool;

[0031] The soil depth grid soildepth.tif is generated according to the weight formula.

[0032] In one specific implementation, the system is implemented in parallel acceleration by:

[0033] Using the MPI parallel computing framework;

[0034] Assigning multiple processes to the PitRemove, D8 / Dinf flow direction calculation, and AreaD8 / AreaDinf flow accumulation steps of TauDEM;

[0035] The number of parallel processes can be configured to 4-8.

[0036] In one specific implementation, the tool calling interface includes:

[0037] QGIS calls the TauDEM command line tool through a Python script;

[0038] GRASS and SAGA realize data format conversion through the GDAL library;

[0039] GPKG format is used as the intermediate data exchange standard between tools.

[0040] In one specific implementation, the output data of the system directly meets the following CNMM model input requirements:

[0041] stream_network.dat provides river section topological relationship parameters;

[0042] stream_map.dat provides gridded river spatial parameters;

[0043] All parameter precision meets the data specification of the corresponding version (such as the v4.0 version) of CNMM.

[0044] Compared with the prior art, the hydrological network parameter automatic generation system based on multi-open source GIS platform integration has the following beneficial effects:

[0045] 1. In specific implementation, parallel computing can be realized by integrating MPI (Message Passing Interface), so that the processing efficiency of large-area watershed hydrological analysis is improved;

[0046] 2. Cross-mainstream operating system platform operation is supported, and through the integration and adaptation of the open source tool chain, the consistency of QGIS, TauDEM and other tools in different environments is ensured, and platform dependence is reduced;

[0047] 3. A special data output format is designed for the CNMM model, manual intervention steps are reduced, and the degree of automation of data processing is improved. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor on the basis of the provided drawings.

[0049] Figure 1 The overall flowchart of the hydrological network parameter automatic generation system based on multi-open source GIS platform integration. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present application will be described clearly and completely below. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] As shown in Figure 1 The hydrological network parameter automatic generation system based on multi-open source GIS platform integration includes core modules: input data module, TauDEM hydrological analysis module, river network post-processing module, watershed mask generation module, soil depth calculation module, row-column grid map generation module and final data output module.

[0052] An input data module is configured to receive original data such as DEM files, MASK / OUTLET vector files, and the like, and deliver the same to a TauDEM hydrological analysis module as basic data for subsequent processing. Exemplarily, the input data module receives files including DEM files, MASK or OUTLET vector files. The DEM file is a data file recording terrain elevation information, and contains elevation values of each cell in the study area, which is basic terrain data for hydrological analysis. The MASK vector file is used to define the known basin boundary, and through the file, the spatial range of the basin can be directly determined to limit the area for subsequent hydrological analysis. The OUTLET vector file marks the known outlet position as the outlet point of water flow, and is used to determine the final outflow position of water flow when generating a river network, so as to ensure the accuracy and pertinence of hydrological analysis.

[0053] The TauDEM hydrological analysis module is configured to perform hydrological analysis and processing (such as filling DEM pits, calculating flow direction / slope / flow accumulation, and generating a vector river network) on the original data, generate intermediate files such as felFile (elevation without pits), pFile (flow direction), and ad8File (flow accumulation), and deliver the intermediate files to a river network post-processing module, a basin mask generation module, and a soil depth calculation module to provide core analysis data for subsequent modules. Exemplarily, local pit points in the DEM are filled by PitFill operation to generate felFile, so as to ensure continuous outflow of water flow.

[0054] The river network post-processing module is configured to receive vector river network data output by the TauDEM hydrological analysis module, perform format conversion (Shapefile to GPKG) and attribute update (such as river section level and effective depth calculation), and deliver the processed river network data to a row-column grid map generation module and a final data output module to provide a basis for river spatial parameter calculation and final data integration.

[0055] The basin mask generation module is configured to receive river raster data (srcFile) and flow direction data (pFile) output by the TauDEM hydrological analysis module, generate basin mask data (maskFile) and convert flow direction format, and deliver the mask data to a soil depth calculation module and a final data output module to limit the basin calculation range and ensure data compatibility.

[0056] Soil depth calculation module, for receiving slope data (slpFile) output by TauDEM hydrological analysis module, non-depression elevation data (felFile) and mask data (maskFile) output by watershed mask generation module, calculating soil depth (soildepthFile) and transmitting to river network post-processing module to provide key parameters for effective depth calculation of river channel;

[0057] Row-column grid map generation module, for receiving DEM data output by TauDEM hydrological analysis module and optimized river network data output by river network post-processing module, generating grid polygon file and calculating effective length and direction of river channel in grid, and transmitting results to final data output module to supplement spatial attribute information of river channel;

[0058] Final data output module, for receiving various data output by river network post-processing module, watershed mask generation module and row-column grid map generation module, integrating to generate ASC format file (such as dem.asc, flowacc.asc) and structured data file (stream_map.dat, stream_network.dat), and providing standardized input data for CNMM model.

[0059] In this embodiment, there are three data processing paths, including:

[0060] Core hydrological network generation path, input data module→TauDEM hydrological analysis module→river network post-processing module→final data output module;

[0061] Watershed range and soil parameter support path, input data module→TauDEM hydrological analysis module→watershed mask generation module→soil depth calculation module→river network post-processing module→final data output module;

[0062] River spatial parameter supplement path, input data module→TauDEM hydrological analysis module→river network post-processing module→row-column grid map generation module→final data output module.

[0063] The three paths all start from input data module and end at final data output module, and form cooperation through data transmission between modules:

[0064] Path 1 is the core process, generating basic data of river network;

[0065] Path 2 provides watershed range limitation and soil depth parameters to optimize river attribute calculation;

[0066] Path 3 supplements spatial parameters (length, direction, etc.) of river channel in grid to improve the dimension of output data;

[0067] The processing results of the three paths are summarized to the final data output module to form the standardized input data required by the CNMM model.

[0068] On the basis of the above technical solution, the input data module receives input data, including a DEM file (terrain data), a MASK (known watershed boundary) or an OUTLET (known water outlet) vector file.

[0069] The received original data is directly transmitted to the TauDEM hydrological analysis module as initial data for processing of the module.

[0070] On the basis of the above technical solution, the data processing of the TauDEM hydrological analysis module specifically includes:

[0071] The PitFill operation is performed, the PitRemove algorithm of TauDEM is used, each cell of DEM is traversed, closed concave points (i.e. the surrounding elevation is higher than the point) are identified and gradually filled to the lowest neighborhood elevation, and a DEM data without concave points (felFile, i.e. fel.tif) is generated;

[0072] The flow direction and slope are calculated, the D8 algorithm (D8 single flow direction algorithm) is used, the flow direction is determined according to the maximum slope direction in the 8 neighborhoods of each cell, the flow direction grid (pFile, i.e. p.tif) and the slope data (sd8File, i.e. sd8.tif) are generated; or, the Dinf algorithm (Dinf infinite flow direction algorithm) is used, the flow direction is divided into two adjacent directions, the flow is allocated according to the slope ratio, the flow direction angle (angFile, i.e. ang.tif) and the slope data (slpFile, i.e. slp.tif) are generated;

[0073] The principle of the D8 algorithm is as follows: the flow direction of each cell is determined by the maximum slope direction in the 8 neighborhoods, the direction code is 1 (east) to 8 (northwest), and the formula is: flow direction = argmax (elevation difference / cell distance);

[0074] The principle of the Dinf algorithm is as follows: the flow direction is divided into two adjacent directions (for example, northeast and north), the flow is allocated according to the slope ratio, and the formula is: flow direction ratio = slope 1 / (slope 1 + slope 2);

[0075] The flow accumulation is performed, based on the D8 or Dinf flow direction data, the AreaD8 or AreaDinf method is used to accumulate the number of cells contributed by the upstream, and the flow accumulation grid (ad8File, i.e. ad8.tif) is generated, when the accumulation value exceeds the threshold value (such as source = 1500m²), the river starting point (srcFile, i.e. src.tif) is marked;

[0076] The principle of AreaD8 algorithm is as follows: based on the number of cells accumulated upstream by D8 flow direction, a flow accumulation raster (ad8.tif) is generated;

[0077] A river network is generated, and the Threshold tool is used to extract river raster data according to the flow accumulation value; the StreamNet algorithm is used to convert the river raster to a vector network (streamFile, i.e., net.shp), generate topological relationship data (tree.dat, coord.dat), and process zero-length river sections, delete invalid river sections, and update upstream and downstream links (USLINKNO3 field); for example, threshold=source / cellarea (cell accumulation threshold);

[0078] If the input is OUTLET, first adjust the outlet position by MoveOutlets, and then generate the river network; if the input is MASK, directly generate the complete river network;

[0079] The generated intermediate files such as felFile, pFile, and ad8File are transmitted to the river network post-processing module, the watershed mask generation module, and the soil depth calculation module.

[0080] On the basis of the above technical solutions, the data processing of the river network post-processing module specifically includes:

[0081] The river network format conversion is performed, the net.shp output by the TauDEM hydrological analysis module is read, the QGIS QgsVectorFileWriter.writeAsVectorFormat function is used to convert it to GPKG format (stream2File, i.e., net.gpkg), and the segorder (river section level) and effdepth (effective depth) fields are added, and the specific fields are shown in Table 1;

[0082] The river section level (segorder) is calculated: the source river section is initialized as level 1, and the downstream river section is recursively traversed, and if there are multiple inputs upstream, the downstream river section level is the maximum upstream level plus 1 (formula: segorder downstream=max(segorder upstream)+1);

[0083] The effective depth of the river (effdepth) is calculated: based on the soil depth data (soildepthFile) output by the soil depth calculation module and the minimum depth of the river section node, the formula effdepth=0.95×min(soil depth, node minimum depth) is used for calculation; the preset proportion coefficient is 0.95, and the effective depth data effdepth=0.95×min(soil depth data, node depth data);

[0084] Channel classification (chanclass): according to the meanmsq, the river channel category is determined by comparing with the preset 18 classification standards, and the corresponding hyddepth (hydrological depth) and effwidth (effective width) are determined. For specific classification standards, refer to Table 2.

[0085] The processed river network data file is transmitted to the row-column grid map generation module and the final data output module.

[0086] Table 1. Explanation of net.gpkg file header parameters of river network vector file

[0087]

[0088] Table 2. Explanation of river classification and hydrological parameters

[0089]

[0090]

[0091] On the basis of the above technical solutions, the data processing of the watershed mask generation module specifically includes:

[0092] Generating watershed mask data (maskFile): receiving the river raster file (srcFile) output by the TauDEM hydrological analysis module, generating a binary raster (mask.tif), where 1 represents the area within the watershed and 0 represents the outside, used to limit the subsequent calculation range; that is, a value of 0 represents a non-river and a value of 1 represents a river; the river raster srcFile extracted based on the flow accumulation threshold reflects the spatial distribution of the river network; the watershed boundary mask maskFile is used to limit the calculation range (such as soil depth, flow accumulation) in subsequent analysis;

[0093] Converting ArcGIS flow direction: receiving the flow direction data (pFile) output by the TauDEM hydrological analysis module, converting it into a raster file (arcflowdir.tif) that conforms to the ArcGIS flow direction coding standard, enhancing data compatibility; for example, the input data uses pFile (D8 flow direction raster of TauDEM): coded as 1 (east) to 8 (northwest), indicating 8 directions; D8 flow direction coding indicates the direction of water flow from the current cell to the adjacent cell, which is used for catchment analysis; ArcGIS coding uses power-of-2 encoding, which supports multi-directional flow accumulation calculation;

[0094] Transmitting maskFile to the soil depth calculation module and the final data output module.

[0095] On the basis of the above technical scheme, the data processing of the soil depth calculation module specifically includes:

[0096] Receiving the slope data (slpFile) and the depressionless elevation data (felFile) output by the TauDEM hydrological analysis module, and the maskFile output by the watershed mask generation module;

[0097] Calculating the soil depth through a weight formula (comprehensive slope weight and elevation weight) to generate a soil depth data file (soildepthFile, i.e., soildepth.tif);

[0098] Passing the soildepthFile to the river network post-processing module for calculating the effective depth (effdepth) of the river channel.

[0099] Illustratively, the weight formula is: soil depth = a x slope weight + b x elevation weight, where a and b are weight coefficients (a + b = 1, 0 < a, b < 1), and the slope weight is negatively correlated with the slope value, and the elevation weight is positively correlated with the elevation value.

[0100] On the basis of the above technical scheme, the data processing of the row-column grid map generation module specifically includes:

[0101] Receiving the DEM data output by the TauDEM hydrological analysis module and the optimized river network data output by the river network post-processing module;

[0102] Using the native:creategrid tool to create a grid polygon file (rowcol_polygon.gpkg) aligned with the DEM;

[0103] Using the native:intersection tool to intersect the river network with the grid polygon to generate a coverage file (outcover.gpkg) and calculate the projected length (Segment Length) and direction (Aspect) of the river channel in the grid;

[0104] Passing the calculation results to the final data output module.

[0105] Illustratively, receiving the DEM data output by the TauDEM hydrological analysis module and the optimized river network data output by the river network post-processing module;

[0106] Using the native:creategrid tool (QGIS native grid creation tool) to create a row-column grid polygon file (rowcol_polygon.gpkg) aligned with the DEM raster unit according to the spatial range and resolution of the DEM data;

[0107] The vector river network and the grid polygon are subjected to spatial intersection operation by the native:intersection tool (QGIS native spatial intersection tool), the river section attribute and the grid attribute of the intersection area are retained, the cover file (outcover.gpkg) is generated, and the projection length (SegmentLength) and direction (Aspect) of the river channel in the grid are calculated based on the intersection result;

[0108] The calculation result is transmitted to the final data output module.

[0109] On the basis of the above technical scheme, the data processing of the final data output module specifically includes:

[0110] Receiving various data output by the river network post-processing module, the watershed mask generation module and the row-column grid map generation module;

[0111] Outputting the ASC format file: including dem.asc (elevation), str.asc (river channel classification), flowacc.asc (flow accumulation);

[0112] Outputting the structured data file:

[0113] stream_map.dat: recording the river channel parameters of each grid, including column (Col), row (Row), ID, length (Segment Length), depth (Cut Height, i.e. effdepth), width (Cut Width, i.e. effwidth), direction (Aspect);

[0114] stream_network.dat: storing the river section topological relationship, including ID (corresponding to LINKNO), order (ORDER, corresponding to segorder), slope (SLOPE), length (LENGTH), class (CLASS, corresponding to chanclass) and the like;

[0115] These output files serve as the standardized input data of the CNMM model.

[0116] stream_map.dat is used to record the river channel parameters (column, row, ID, length, depth, width, direction) of each grid, and the specific fields are shown in Table 4;

[0117] stream_network.dat is used to store the river section topological relationship (ID, order, slope, length, class), and the specific fields are shown in Table 3.

[0118] Table 3 Parameter explanation table for file header of river network data file stream_network.dat

[0119]

[0120] Table 4. Explanation of stream_map.dat file header parameters

[0121]

[0122] The various embodiments described in this specification are presented by way of example, and each embodiment is not necessarily composed of all features described with respect to other embodiments. Each embodiment describes a distinct embodiment, and each embodiment can be implemented with respect to one or more features of other embodiments, without departing from the scope of the application. The foregoing description of various embodiments of the application will be better understood with reference to the appended drawings. While the application is described with reference to the drawings, the description is not intended to be limiting as to the scope of this application, as claimed, and any equivalents.

Claims

1. An automatic hydrological network parameter generation system based on the integration of multiple open-source GIS platforms, characterized in that, include: It integrates the QGIS, TauDEM, GRASS, and SAGA open-source toolchains, and achieves collaborative processing through tool call interfaces; An improved StreamNet algorithm is used to perform topology repair, including deleting zero-length river segments and updating the USLINKNO3 field to address the multi-branch river connectivity problem; Soil depth is calculated using a dynamic weighting formula based on slope data slpFile and elevation data without depressions felFile. Output structured data files stream_map.dat and stream_network.dat adapted to the CNMM model; The improved StreamNet algorithm includes: Extract river raster using the Threshold tool in TauDEM; When converting a river raster to a vector network, river segments shorter than 0.1 meters are automatically deleted; For the four-way or five-way nodes, activate the USLINKNO3 field in the topology data to record the third upstream branch; The system's output data directly meets the following CNMM model input requirements: stream_network.dat provides parameters for the topology of river segments; stream_map.dat provides gridded river spatial parameters; All parameters conform to the data specifications of the corresponding version of CNMM.

2. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms as described in claim 1, characterized in that, The dynamic weighting formula is: Soil depth = α × slope weight + β × elevation weight, where: Where α and β are weighting coefficients; α∈[0.6,0.8]，β=1-α； Slope weight = 1 - standardized slope value; Elevation weight = Standardized elevation value.

3. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms as described in claim 1, characterized in that, The structured data file includes: stream_network.dat: Records the segment ID, ORDER, SLOPE, LENGTH, and CLASS; stream_map.dat: Records grid column (Col), row (Row), segment length (Segment Length), and aspect (Aspect).

4. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms as described in claim 3, characterized in that, The CLASS is divided into 18 categories based on the average catchment area (meanmsq). The ORDER uses a recursive hierarchical algorithm, with the source river section being level 1 and the downstream river section being level 1, which is the maximum level of the upstream section plus 1.

5. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms according to claim 1, characterized in that, It also includes a watershed mask generation module: Generate a binary mask (mask.tif) from the river raster (srcFile) generated by TauDEM; Convert the TauDEM D8 flow direction data to an ArcGIS-compatible flow direction encoding format.

6. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms according to claim 1, characterized in that, The soil depth calculation process includes: Calculate the standardized slope value using the GRASS tool; Calculate standardized elevation values ​​using the SAGA tool; The soil depth raster, soildepth.tif, is generated based on the weighting formula.

7. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms according to claim 1, characterized in that, The system achieves parallel acceleration in the following ways: Use the MPI parallel computing framework; Multiple processes are allocated for the PitRemove, D8 / Dinf flow direction calculation, and AreaD8 / AreaDinf flow accumulation steps of TauDEM; The number of parallel processes is configured to be 4-8.

8. The automatic generation system for hydrological network parameters based on the integration of multiple open-source GIS platforms according to claim 1, characterized in that, The tool's API call interface includes: QGIS calls the TauDEM command-line tool via Python scripts; GRASS and SAGA use the GDAL library for data format conversion; The GPKG format is used as the intermediate data exchange standard between the various tools.

Images