Method and system for automatic processing of multi-source spatial data of water resources investigation

Abstract

The embodiment of the application discloses a kind of water resource survey multi-source spatial data automatic processing method and system, for significantly improving the efficiency and quality of multi-source spatial data processing of water resource survey.The embodiment of the application method includes the following steps that are automatically executed in sequence: accessing water resource survey data, water resource survey data at least includes initial reservoir DEM data and null data table, based on water resource survey data standard rules, automatically check the quality of water resource survey data, through attribute automatic association engine, the attribute mapping and hanging of water resource survey data that pass through quality inspection are carried out, generate fusion dataset, according to conversion rule, attribute normalizing and geometric transformation are carried out to fusion dataset, generate standard single reservoir DEM data and its metadata, based on water resource survey product catalog and product packaging rules, product packaging is carried out to single reservoir DEM data, and standard spatial data product is generated.

Description

[0001] This application claims priority to Chinese Patent Application No. 202512033594.7, filed with the Chinese Patent Office on December 30, 2025, entitled "An Automated Processing Method and System for Multi-Source Spatial Data for Water Resources Survey", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the fields of geographic information and water resources survey technology, and more specifically, to a method for automated processing of multi-source spatial data in water resources surveys, an apparatus for automated processing of multi-source spatial data in water resources surveys, an electronic device, a computer-readable storage medium, and a computer program product containing instructions. Background Technology

[0003] As the survey of reservoir water storage deepens, the demand for standardized spatial data products is increasing, thus requiring multi-source spatial data processing for water resource surveys.

[0004] Existing methods rely on manual switching between multiple software platforms such as ArcGIS, Excel, and FME. In ArcGIS, reservoir DEM (Digital Elevation Model) image data is imported and coordinates are manually transformed. In Excel, attribute tables containing normal water levels are organized and codes are manually checked. In FME, format conversion is manually configured. Then, the data is returned to ArcGIS for spatial overlay of DEM and attribute linking. Subsequently, the map sheet is manually determined, DEMs are cropped one by one, and metadata is manually filled in to generate map sheet reservoir DEM products. Finally, water level attributes are manually extracted, raster calculations are performed, and vector conversions are performed to generate thematic products of inundation range.

[0005] However, this method lacks automation. Generating final data products that meet the specifications usually requires tedious and repetitive operations such as manually judging the map sheets involved in the reservoir, manually creating catalogs, projecting and cropping each map sheet, and manually filling in metadata tables. Therefore, the efficiency and quality of multi-source spatial data processing in water resources surveys are low. Summary of the Invention

[0006] This application provides an automated processing method for multi-source spatial data from water resource surveys, an automated processing system for multi-source spatial data from water resource surveys, an electronic device, a computer-readable storage medium, and a computer program product containing instructions, which are used to significantly improve the efficiency and quality of multi-source spatial data processing in water resource surveys.

[0007] In a first aspect, embodiments of this application provide an automated processing method for multi-source spatial data from water resource surveys. This method, based on a visual data stream processing platform, creates a standardized processing template by dragging and dropping converter components, configuring component parameters, and establishing connections between components. The standardized processing template contains pre-defined water resource survey data standards and product specifications, enabling fully automated processing, including the following steps executed automatically in sequence:

[0008] After acquiring multi-source heterogeneous water resources survey data, the water resources survey data is accessed. The water resources survey data includes at least initial reservoir DEM data and null data tables.

[0009] Based on pre-defined water resources survey data standards and rules, the water resources survey data is subjected to automated quality checks.

[0010] Using an automatic attribute association engine, based on a preset unique identifier for the reservoir, water resource survey data that has passed quality inspection is mapped and linked to generate a fused dataset.

[0011] Based on the transformation rules built into the standardized processing template, the fused dataset is subjected to attribute normalization and geometric transformation to generate standard-compliant individual reservoir DEM data and its metadata.

[0012] Based on the preset water resources survey product catalog and product packaging rules, the DEM data of the individual reservoirs are productized and packaged to generate standardized spatial data products.

[0013] Secondly, embodiments of this application provide an automated processing system for multi-source spatial data from water resource surveys, comprising:

[0014] The access module is used to acquire multi-source heterogeneous water resources survey data and then access the water resources survey data, which at least includes initial reservoir DEM data and a null data table.

[0015] The quality inspection module is used to perform automated quality inspection on the water resources survey data based on preset water resources survey data standard rules;

[0016] The attribute mapping and linking module is used to automatically associate attributes with the reservoir based on the preset unique identifier code of the reservoir, and perform attribute mapping and linking on the water resource survey data that has passed the quality inspection to generate a fused dataset.

[0017] The attribute normalization and geometric transformation module is used to perform attribute normalization and geometric transformation on the fused dataset according to the transformation rules built into the standardized processing template, and generate standard-compliant individual reservoir DEM data and its metadata.

[0018] The product packaging module is used to perform product packaging on the DEM data of the individual reservoir based on the preset water resources survey product catalog and product packaging rules, and generate standardized spatial data products.

[0019] Thirdly, embodiments of this application provide an electronic device, including:

[0020] Central processing unit, memory, input / output interfaces, and power supply;

[0021] The memory is either a short-term storage memory or a persistent storage memory;

[0022] The central processing unit is configured to communicate with the memory and execute instructions in the memory to perform the aforementioned automated processing method for multi-source spatial data from water resource surveys.

[0023] Fourthly, embodiments of this application provide a computer-readable storage medium including instructions that, when executed on a computer, cause the computer to perform the aforementioned method for automated processing of multi-source spatial data from a water resources survey.

[0024] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to execute the aforementioned method for automated processing of multi-source spatial data from a water resources survey.

[0025] As can be seen from the above technical solutions, the embodiments of this application have the following advantages: This application creates a standardized processing template by dragging and dropping converter components and configuring component parameters and connecting components based on a visual data stream processing platform. The water resources survey data standards and product specifications are pre-placed in the standardized processing template, connecting the entire process of data access, automated quality inspection, automatic attribute association and fusion, rule-driven attribute normalization and geometric transformation, and productized packaging and output. This achieves automated processing with no or very little code, eliminating the need for manual switching between multiple software platforms. It avoids tedious and repetitive operations such as manually judging the map sheets involved in the reservoir, manually creating catalogs, projecting and cropping map sheets one by one, and manually filling in metadata tables, thereby significantly improving the efficiency and quality of multi-source spatial data processing in water resources surveys.

[0026] Correspondingly, the multi-source spatial data automated processing system, electronic equipment, computer-readable storage medium, and computer program product containing instructions provided in this application for water resources surveys also have the aforementioned technical effects. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating an automated processing method for multi-source spatial data from a water resources survey, as disclosed in an embodiment of this application.

[0028] Figure 2 This is a schematic diagram of the parameter configuration interface of a fusion converter disclosed in an embodiment of this application;

[0029] Figure 3 This is a schematic diagram of a single DEM image, the result after rasterization to polygonization, and the result after aggregation, as disclosed in an embodiment of this application.

[0030] Figure 4 This is a schematic diagram of a partial metadata field of a map sheet reservoir DEM data file after being automatically attached by an attribute auto-association engine, as disclosed in an embodiment of this application;

[0031] Figure 5 This is a schematic diagram of the operating interface of a system for generating a 1:10000 scale reservoir DEM, as disclosed in an embodiment of this application.

[0032] Figure 6 This is a schematic diagram of a 1:10000 scale multi-level directory disclosed in an embodiment of this application;

[0033] Figure 7 This is a schematic diagram of multiple individual reservoirs cut out from a 1:10000 map sheet, as disclosed in an embodiment of this application.

[0034] Figure 8 This is a schematic diagram of multiple Shapefile records (local fields) within a 1:10000 map sheet that correspond one-to-one with the DEM of a single reservoir, as disclosed in an embodiment of this application.

[0035] Figure 9 This is a schematic diagram of the operating interface of a system for generating a single reservoir DEM, as disclosed in an embodiment of this application.

[0036] Figure 10 This is a schematic diagram of a multi-unit reservoir DEM folder and its next-level files disclosed in an embodiment of this application;

[0037] Figure 11 This is a schematic diagram of the structure of an automated processing system for multi-source spatial data of water resource surveys disclosed in an embodiment of this application;

[0038] Figure 12 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of this application. Detailed Implementation

[0039] This application provides an automated processing method for multi-source spatial data from water resource surveys, an automated processing system for multi-source spatial data from water resource surveys, an electronic device, a computer-readable storage medium, and a computer program product containing instructions, which are used to significantly improve the efficiency and quality of multi-source spatial data processing in water resource surveys.

[0040] Please see Figure 1 , Figure 1 This is a flowchart illustrating an automated processing method for multi-source spatial data from a water resources survey, as disclosed in an embodiment of this application. The method is based on a visual data stream processing platform and creates a standardized processing template by dragging and dropping converter components, configuring component parameters, and connecting components. The standardized processing template contains pre-defined water resources survey data standards and product specifications, enabling fully automated processing. The method includes the following steps executed automatically in sequence:

[0041] It is understood that the embodiments of this application are based on a visual data stream processing platform. Standardized processing templates are created by dragging and dropping converter components, configuring component parameters, and connecting components. Water resource survey data standards and product specifications are pre-placed in the templates, connecting the entire process from data access, automated quality inspection, automatic attribute association and fusion, rule-driven attribute normalization and geometric transformation, to productized packaging and output, thus achieving automated processing. Besides the FME (Feature Manipulation Engine) platform, the visual data stream processing platform can also be other platforms that support visual workflow orchestration and spatial data processing, such as ArcGIS; specific limitations are not specified here. This application uses implementation on the FME platform as a preferred example to clearly demonstrate the complete implementation process and significant advantages of the automated processing method of this application on a visual, workflow-based data integration platform.

[0042] 101. After obtaining multi-source heterogeneous water resources survey data, access the water resources survey data. The water resources survey data should at least include the initial reservoir DEM data and null data tables.

[0043] In one optional implementation, multi-source heterogeneous water resources survey data refers to water resources survey-related data from different departments, different periods, and different acquisition methods, with differences in data format, coordinate system, and field definitions. This includes, but is not limited to, reservoir DEM image data, GIS vector data, and Excel or CSV structured table data. Initial reservoir DEM data refers to Digital Elevation Model (DEM) data that records the topographic elevation of the reservoir area based on the contour lines and elevation points of the original topographic map. An empty metadata table refers to a metadata template table pre-generated according to water resources survey standards, with a complete field structure but unfilled attribute values.

[0044] 102. Based on pre-defined water resources survey data standards and rules, perform automated quality checks on water resources survey data.

[0045] In one optional implementation, the pre-set water resources survey data standard rules refer to a standardized set of rules embedded in the visualization data stream processing platform, including spatial topology check rules, logical consistency check rules, attribute integrity check rules, data integrity check rules, and / or attribute value range normalization rules. Automated quality inspection refers to automatically performing spatial topology consistency checks, logical consistency checks, attribute integrity checks, data integrity checks, and elevation anomaly handling on the accessed data based on these rules, using spatial filter converters, feature merging converters, test converters, hole extraction converters, and raster band attribute extraction converters. If any check fails, the process is paused and a correction prompt is displayed.

[0046] 103. Using the attribute auto-association engine, based on the preset unique identifier of the reservoir, the water resource survey data that has passed the quality inspection is mapped and linked to generate a fused dataset.

[0047] In one optional implementation, the automatic attribute association engine refers to a data fusion mechanism based on precise keyword matching combined with spatial range relationship verification. This mechanism identifies the correspondence of the same reservoir in data from different sources and automatically links scattered attribute information to a unified data structure. The preset unique reservoir identifier is a pre-defined coded field used to uniquely identify the reservoir. This field may exist with different names in different data sources (e.g., mapno from the provincial surveying and mapping bureau, Gcode from the municipal water resources bureau, skbh from the environmental monitoring institute). The automatic attribute association engine identifies the same reservoir through field mapping. Mapping and linking attributes of water resources survey data that has passed quality inspection to generate a fused dataset involves matching structured tabular data from different sources using the unique reservoir identifier as the key field. After confirming the same reservoir by combining spatial range relationship verification, multi-source attributes such as image attributes, production method attributes, and survey line spacing attributes are automatically linked to the metadata field, forming a unified dataset containing complete spatial and attribute information.

[0048] 104. Based on the transformation rules built into the standardized processing template, perform attribute normalization and geometric transformation on the fused dataset to generate standard-compliant individual reservoir DEM data and its metadata.

[0049] In one optional implementation, the conversion rules built into the standardized processing template refer to the automated processing logic encoded in the conditional expression converter, which transforms attribute value domain clauses into conditional judgment assignment rules. These rules include coordinate system unification rules, product name standardization rules, date format unification rules, null value filling rules, and projection zone number calculation rules. This implementation can utilize these conversion rules to standardize attribute fields in the fused dataset, convert the format of date fields, and calculate projection zone numbers based on latitude and longitude ranges for coordinate unification. The final output is a single reservoir DEM data file with a unified coordinate system, complete attributes, and a standardized format, along with its corresponding metadata file.

[0050] 105. Based on the preset water resources survey product catalog and product packaging rules, perform product packaging on the DEM data of individual reservoirs to generate standardized spatial data products.

[0051] In one optional implementation, the water resources survey product catalog refers to a predefined hierarchical folder structure based on water resources survey data standards, used to standardize the storage and management of various standardized spatial data products. Product packaging rules refer to the processing rules that perform operations such as map cropping, raster calculation, vector-to-plane conversion, and area calculation on individual reservoir DEM data according to the preset water resources survey product catalog specifications. These rules include, but are not limited to, standard map cropping rules and normal water level inundation range rules, used to generate multi-level, multi-type standardized spatial data products that conform to water resources survey standards. Product packaging refers to the process of performing standardized processing operations (including but not limited to map cropping, raster calculation, vector-to-plane conversion, and area calculation) on individual reservoir DEM data based on the preset water resources survey product catalog and product packaging rules, generating standardized spatial data products with a unified format, complete metadata, and a standardized catalog structure. Standardized spatial data products refer to spatial data results generated after product packaging in accordance with water resources survey data standards and product specifications, which have a unified format, complete metadata, and standardized directory structure. These include, but are not limited to, map sheet reservoir DEM products and their map sheet reservoir DEM metadata, thematic spatial data products, etc. Thematic spatial data products include, but are not limited to, normal water level inundation range products, watershed boundary products, and inundation area land use products. Among them, the normal water level inundation range products include normal water level inundation range reservoir DEM data, inundation range vector surface, and reservoir information table with added normal water level inundation area attributes.

[0052] In this way, by creating standardized processing templates based on a visual data stream processing platform, and by dragging and dropping converter components and configuring component parameters and connecting components, water resources survey data standards and product specifications are pre-set in the standardized processing templates. This connects the entire process of data access, automated quality inspection, automatic attribute association and fusion, rule-driven attribute normalization and geometric transformation, and productized packaging and output, achieving automated processing with little or no code. It eliminates the need for manual switching between multiple software platforms and avoids tedious and repetitive operations such as manually judging the map sheets involved in the reservoir, manually creating catalogs, projecting and cropping map sheets one by one, and manually filling in metadata tables. This significantly improves the efficiency and quality of multi-source spatial data processing in water resources surveys.

[0053] In one optional implementation, based on a preset water resources survey product catalog and product packaging rules, the DEM data of individual reservoirs is productized and packaged to generate standardized spatial data products, including at least one of the following: Based on a preset water resources survey product catalog and standard map division rules, the DEM data of individual reservoirs is divided and cropped, and metadata creation operations are performed. Productization and packaging are then performed to generate map-based reservoir DEM products and their metadata, which are standardized spatial data products; Based on a preset water resources survey product catalog and normal water level inundation range rules, the DEM data of individual reservoirs is rasterized, vector-to-plane conversion, area calculation, and DEM cropping operations are performed. Productization and packaging are then performed to generate DEM data of reservoirs within the normal water level inundation range, inundation range vector surfaces, and a reservoir information table with added normal water level inundation area attributes. These are standardized spatial data products.

[0054] Specifically, it can be based on a pre-set water resources survey product catalog and standard map division rules (national standard topography). Figure 1The system employs a 10,000-map segmentation rule to segment and crop individual reservoir DEM data. For each data set, it creates map sheet DEM metadata containing coordinate system, resolution, and elevation datum information. After product packaging, it generates map sheet reservoir DEM products and their metadata, serving as standardized spatial data products. This ensures that individual reservoir DEM data conforms to national standard map sheet specifications, generating map sheet products with complete metadata, facilitating subsequent unified management and sharing. Based on a preset water resources survey product catalog and normal water level inundation range rules, using the normal water level as the elevation threshold, it performs raster calculations on individual reservoir DEM data to extract inundation areas. The raster results are then transformed into vector planes, and the inundation area is calculated. Simultaneously, the DEM data is cropped according to the inundation range, and after product packaging, it generates reservoir DEM data for the normal water level inundation range, an inundation range vector surface, and a reservoir information table with added normal water level inundation area attributes, serving as a standardized spatial data product. In this way, by automatically extracting the inundation range based on the normal water level and generating DEM data, vector surfaces, and attribute information tables, a standardized spatial data product for inundation zone analysis can be formed.

[0055] In one optional implementation, multi-source heterogeneous water resources survey data is acquired. This water resources survey data includes at least initial reservoir DEM data and a null metadata table. The process includes: determining the contour lines of the original topographic map of the reservoir area; if the contour lines are dense, constructing a triangular mesh using an irregular triangular mesh creation tool, and converting it to initial reservoir DEM data using a triangular mesh to raster tool; if the contour lines are sparse, generating initial reservoir DEM data using a terrain to raster tool, and creating a null metadata table corresponding to the initial reservoir DEM data according to water resources survey data standards. Furthermore, the water resources survey data is accessed, including: reading the water resources survey data into a visualization data stream processing platform using a reader corresponding to the water resources survey data format. Further, after accessing the water resources survey data, the method also includes: converting the initial reservoir DEM data into vector metadata surfaces using a raster to polygon converter, and fusing the vector metadata surfaces according to the reservoir's unique identifier using a fusion converter to generate a reservoir extent surface.

[0056] Specifically, contour lines are closed curves connecting points of the same elevation on a topographic map, used to represent the undulating terrain. Dense contour lines mean a large number of contour lines per unit area with small average spacing, indicating a steep slope and drastic elevation changes. Sparse contour lines mean a small number of contour lines per unit area with large average spacing, indicating a gentle slope and gradual elevation changes. More specifically, initial reservoir DEM data can be generated on a GIS platform based on the contour lines and elevation points of the original topographic map of the reservoir area: when the original topographic map is large-scale (e.g., 1:2000 topographic map with dense contour lines), the Create TIN (Create Irregular Triangular Network) tool is used to construct a triangular network, and then the TIN to Raster tool is used to convert it into initial reservoir DEM data; when the original topographic map is small-scale (e.g., 1:10000 topographic map with sparse contour lines), the Topo to Raster tool is used to generate the initial reservoir DEM data. In this embodiment, the initial DEMs are mostly generated from 1:2000 scale topographic maps, with a few generated from 1:10000 scale topographic maps or obtained through field measurements. The resolution is generally 2 meters or higher, and the final resolution needs to be resampled to 10 meters. According to the water resources survey data standards, a metadata template and empty metadata tables corresponding to the submitted DEM data are created for subsequent attribute information attachment and filling. After acquiring multi-source heterogeneous water resources survey data, the water resources survey data can be accessed: in a visualization data stream processing platform (such as FME, Feature Manipulation Engine), depending on the different formats of the water resources survey data, a corresponding reader is used to read in one or more files, including but not limited to img image data files, GIS vector data files, Excel or CSV structured table files, etc. After the initial batch import of the reservoir DEM, the initial reservoir DEM data can be converted into vector metadata polygons using a Raster to Polygon Converter. Then, a Dissolver can be used to merge the vector metadata polygons according to the reservoir's unique identifier to generate the reservoir extent surface. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of the parameter configuration interface of a fusion converter disclosed in an embodiment of this application. For details regarding the aggregation of multiple components after fusion based on reservoir codes, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram of a single DEM image, the result after rasterization to polygonization, and the result after aggregation, as disclosed in an embodiment of this application; if a range surface already exists, this automatic generation step is skipped.

[0057] In this way, the DEM construction method is selected based on the contour line density to ensure that initial reservoir DEM data that meets the accuracy requirements can be generated from original topographic maps of different accuracies. At the same time, the visualization data stream processing platform automatically reads multi-source heterogeneous data and generates reservoir extent surfaces, completing data access and extent surface generation, so as to provide standardized basic data for subsequent quality inspection, attribute association and product packaging.

[0058] In one optional implementation, based on pre-defined water resources survey data standard rules, an automated quality check is performed on the water resources survey data, including at least one of the following: using a spatial filter converter to detect whether the vector metadata surface of the water resources survey data spatially overlaps with the corresponding reservoir DEM, to determine whether the spatial location of the water resources survey data is consistent; using an element association and merging converter to match structured tabular data from different sources in the water resources survey data according to the reservoir's unique identifier, and using a test converter to test whether the key field values ​​after matching are consistent, to determine the logical consistency of the water resources survey data; using a test converter to detect whether the necessary attribute fields of the water resources survey data, such as production method, normal water level, and / or reservoir name, are missing, to determine whether the attribute information of the water resources survey data is complete; using a cavity extraction converter to detect whether there are cavity anomalies in the initial reservoir DEM data of the water resources survey data, and using a raster band attribute extraction converter and a test converter to determine whether the pixel value is missing, to determine whether the initial reservoir DEM data of the water resources survey data is complete; using a raster band no-data value setting converter to uniformly mark the no-data values ​​of the initial reservoir DEM data in the water resources survey data, to determine whether there are anomalies in the elevation information of the water resources survey data.

[0059] Specifically, as an example, automated quality checks can include at least one of the following: using the `Filter equals Candidate` and `Filterintersects Candidate` parameters of the SpatialFilter converter to detect whether the vector metadata polygons spatially overlap with the corresponding reservoir DEM. The initial vector metadata polygons are automatically generated from the reservoir DEM through raster-to-polygon conversion. Under normal circumstances, the two should spatially overlap and be logically consistent. However, after multiple processing steps such as mosaicking with the background image and re-cropping, errors may cause the two to not spatially overlap or be logically inconsistent. Therefore, a spatial topology check is required during re-accession to determine whether the spatial location of the water resources survey data after accession is consistent. The Feature Merger is used to merge structured tabular data from different sources in the integrated water resources survey data according to the reservoir's unique identifier (e.g., the serial number and unique identifier in the XLS format reservoir list provided by the Provincial Environmental Monitoring Institute, and the serial number and unique identifier in the XLS format reservoir mapping method list of this department). Then, the Tester is used to test whether the values ​​of key fields after merging are consistent to determine whether the codes for the same reservoir match, thereby confirming the logical consistency of the integrated water resources survey data. The Tester is also used to check whether necessary attribute fields have null values ​​or are missing (using the "Attribute has a value" condition), including whether the production method attribute in the production method list, the normal water level in the reservoir attribute table, and the reservoir name are missing, to determine whether the necessary attribute information of the integrated water resources survey data is complete. The DonutHoleExtractor is used to detect void anomalies in the initial reservoir DEM data. The RasterBandPropertyExtractor and Tester are used to determine if there are no data points in the pixel values, thus confirming the completeness of the initial reservoir DEM data in the integrated water resources survey data. The RasterBandNodataSetter is used to standardize the no-data values ​​in the initial reservoir DEM data to a preset standard value (e.g., -9999) to determine if there are any anomalies in the elevation information of the integrated water resources survey data. If any of the above checks fails, the process is paused and a modification prompt is displayed. The process is re-executed after the data is corrected.

[0060] In this way, by combining various converters such as spatial filtering, element merging, testing, hole extraction, raster band attribute extraction, and data-free value standardization, the entire process of automated quality inspection of spatial location consistency, logical consistency, attribute integrity, DEM data integrity, and elevation information standardization of multi-source heterogeneous water resources survey data after access can be achieved. This replaces inefficient manual visual inspection, systematically implements water resources survey data standards, and eliminates potential problems in spatial topology, logical consistency, and attribute integrity of the final dataset.

[0061] In one optional implementation, an automatic attribute association engine is used to map and link attributes of water resource survey data that has passed quality inspection, based on a preset unique reservoir identifier, to generate a fused dataset. This includes: converting the initial reservoir DEM data from different projection zones to geographic coordinates in the water resource survey data that has passed quality inspection; obtaining the bounding rectangle of the converted reservoir DEM data and calculating its latitude and longitude range; and making cross-zone judgments based on the latitude and longitude range; using the unique reservoir identifier in the water resource survey data that has passed quality inspection as a key field, performing precise keyword matching on the multi-source attribute data in the water resource survey data that has passed quality inspection, and verifying the matching results based on spatial range relationships to obtain verified attribute data; and automatically linking the verified attribute data to the metadata fields in the water resource survey data that has passed quality inspection to generate the fused dataset.

[0062] Specifically, the core of the attribute automatic association engine is based on precise keyword matching (usually matching reservoir numbers, such as mapno matching Gcode), supplemented by spatial relationship verification (using a spatial relationship converter to determine whether reservoirs with the same number or name are spatially connected; if they are not spatially connected, they may not actually be the same reservoir). More specifically, the initial reservoir DEM data from the water resources survey data that has passed quality checks can first be converted from planar coordinates of different projection zones to geographic coordinates (latitude and longitude). Then, the bounding rectangle of the converted reservoir DEM data is obtained, its lower left and upper right corner latitude and longitude coordinates are calculated, and it is converted to standard degree minute and second format. Next, based on the minimum and maximum longitude values ​​of the outer rectangle of the reservoir DEM data, the projection zone number is calculated. For example, in FME, it is calculated using the formula "@round((@Substring(@Value(LonExt),0,3)+@Substring(@Value(LonExt),3,2) / 60+@Substring(@Value(LonExt),5,2) / 3600) / 3)". This formula means "convert the longitude string in degree, minute, and second format to a decimal degree number, then calculate and round it according to the 3° zone division rule to obtain the Gauss-Kruger projection zone number of the longitude, which is used to determine whether the reservoir DEM data crosses projection zones". If the zone numbers are inconsistent, it is marked as a cross-zone reservoir DEM. Next, using the unique reservoir identifier from the water resources survey data that passed the quality inspection as the key field, precise keyword matching was performed on the multi-source attribute data. The matching results were then validated based on spatial relationships (using a spatial relationship converter to determine if reservoirs with the same number or name are spatially connected), resulting in validated attribute data. Finally, the validated attribute data (including image attributes such as row count, column count, top-left corner coordinates, production method attributes, and survey line spacing attributes) was automatically appended to the metadata fields of the water resources survey data that passed the quality inspection, generating a fused dataset. Please refer to [link / reference] for details. Figure 4 , Figure 4 This is a schematic diagram of a partial metadata field of a map sheet reservoir DEM data file after it has been automatically attached by an attribute auto-association engine, as disclosed in an embodiment of this application. Figure 4 The local metadata fields include automatically populated attribute information such as grid size (gridCell), grid order (gridOrder), number of rows (gridRowNum), number of columns (gridColNum), top-left X coordinate (leftUpX), top-left Y coordinate (leftUpY), and decimal places (deciNum).

[0063] In this way, by combining precise keyword matching with spatial range relationship verification, accurate association and automatic linking of multi-source heterogeneous attribute data can be achieved. This effectively integrates and structures the scattered production information, processing history and other metadata with the spatial data ontology, solving the problems of scattered and missing metadata, complicated filling, difficulty in data traceability and lack of basis for product credibility assessment in existing technologies. It provides complete information support for long-term data management, reuse and standardized product output.

[0064] In one optional implementation, the fused dataset is subjected to attribute normalization and geometric transformation according to the transformation rules built into the standardization processing template to generate standard-compliant individual reservoir DEM data and its metadata. This includes: using a conditional expression converter to transform the clauses of the attribute value domain in the standardization processing template into assignment rules containing conditional judgments; performing normalized assignment on the attribute fields in the fused dataset based on the assignment rules to obtain a normalized fused dataset; performing format conversion on the date field in the normalized fused dataset to obtain a date-formatted fused dataset; calculating the projection zone number based on the latitude and longitude range of the date-formatted fused dataset; and projecting the reservoir DEM data and vector data in the formatted fused dataset to the target coordinate system based on the projection zone number, performing geometric transformation and coordinate unification processing to generate standard-compliant individual reservoir DEM data and its metadata.

[0065] Specifically, the clauses regarding attribute value ranges in the standardized processing template refer to the specific provisions in the standard regarding the allowed value range, format, and assignment rules for each attribute field, such as the standardized value requirements for fields like coordinate system name, product name, date format, and production method corresponding to the production method. Assignment rules that include conditional judgments refer to rules that assign standardized values ​​based on the current value or characteristics of the attribute field through conditional expressions, such as "if the coordinate system name begins with CGCS2000, then assign the value '2000 National Geodetic Coordinate System' to the coorSystem field." Calculating the projection zone number based on the latitude and longitude range of the fused dataset after format conversion involves: extracting the minimum or maximum longitude value of the bounding rectangle of the reservoir DEM data in the fused dataset; dividing the decimal longitude value by 3 according to the Gauss-Kruger projection 3° zone division rule; and rounding the result to the nearest integer to obtain the projection zone number. For example, if the maximum longitude of the bounding rectangle of a reservoir DEM data is 112.5042°, then the projection zone number is calculated as round(112.5042 / 3) = round(37.5014) = 38, meaning the reservoir DEM data is located in the 38th projection zone. The calculated projection zone number can be used to determine the central meridian parameter of the target coordinate system. This allows for the unified projection of the reservoir DEM data and vector data in the fused dataset onto this target coordinate system, achieving geometric transformation and coordinate unification of multi-source data, and generating individual reservoir DEM data and its metadata that conform to water resources survey data standards.

[0066] In this implementation, the transformation rules built into the standardized processing template are directly encoded in the parameter settings of each FME functional converter, achieving standardized processing through conditional expressions, functions, and fixed parameters. For example, using the Conditional Expression module of AttributeManager (attribute management converter) to assign standardized values ​​to attributes essentially embeds business rules directly into the data transformation process. As an example: for assigning coordinate system and projection information, if the coordinate system name begins with CGCS2000, the coorSystem field is assigned the value "2000 National Geodetic Coordinate System" and the mapProj field is assigned the value "Gauss-Kruger Projection". For assigning product names, the prodName field is set to "10-meter Grid Digital Elevation Model" according to the specification. For assigning values ​​to the map corner point range, the LonExt and LatExt (longitude and latitude range of map corner points) fields are set to the DDDMMSS-DDDMMMSS style according to the specification. For handling null values, null values ​​for required fields such as dataInspec (data inspection date) and distDate (distribution date) are set to 99990909 according to the specification. For mapping production method to production mode, when the production mode is historical topographic map vectorization, the DEM production method is the TopotoRaster method; when the production mode is 2000 topographic map vectorization, the DEM production method is the linear interpolation method. For mapping data source type to field of view, when the data source type is multibeam bathymetry, the field of view is 120°; when it is singlebeam bathymetry, the field of view is -9999. For assigning values ​​to product version, the product version field value is a normalized production date in a style similar to "2025.03". Secondly, the date fields such as production date, data source timeliness, and product version date in the normalized and assigned fused dataset can be formatted to YYYY / MM / DD format, resulting in a format-converted fused dataset. For example, if the drawing method is vectorized from 2000 drawings, the production date and data source timeliness fields to be written to the Shapefile can be formatted as 2025 / 03 / 05 using the formula %Y%m%d; the production date and data source timeliness fields to be written to the xls format can be formatted as 2025 / 03 / 05 using the formula %Y / %m / %d. Furthermore, the projection zone number can be calculated based on the latitude and longitude range of the format-converted fused dataset. Based on the projection zone number, the reservoir DEM data and vector data in the format-converted fused dataset can be projected to the target coordinate system for geometric transformation and coordinate unification. Specifically, after translating and removing the zone number from the DEM images of reservoirs in different projection zones, they are uniformly projected onto a plane coordinate system or geographic coordinate system based on the central meridian of the projection zone, generating standard-compliant individual reservoir DEM data and its metadata.

[0067] In this way, the attribute value domain clauses and geometric transformation rules in the standardized processing template are directly encoded into the conditional expressions and parameter configurations of the visual data stream processing platform converter. This enables the standardized assignment of attributes, the unification of date formats, and the automated processing of coordinate system-projection for the fused dataset. It ensures that the generated individual reservoir DEM data and its metadata strictly comply with the water resources survey data standards, solving the problems of inconsistent manual processing standards, complicated metadata filling process, non-standard filling, and error-proneness in the existing technology, and improving the degree of data standardization and product quality consistency.

[0068] In one optional implementation, based on a preset water resources survey product catalog and standard map sheet rules, the DEM data of individual reservoirs is truncated and metadata is created. Product packaging is then performed to generate map sheet reservoir DEM products and their metadata. This includes: spatially overlaying the individual reservoir DEM data with standard map sheet vector data based on geographic coordinates to determine the standard map sheets involved in the individual reservoir DEM data; for each determined standard map sheet, creating a corresponding standard hierarchical folder structure in the output directory; projecting the vector data of the standard map sheets onto the same plane coordinate system as the individual reservoir DEM data; and performing an outward buffering operation on the projected map sheet vector surface. The standard map sheet vector surfaces after expansion and buffering are stored in the corresponding folder. Using the standard map sheet vector surfaces after expansion and buffering, the DEM data of individual reservoirs falling within their range are batch-cropped to obtain map sheet reservoir DEM data files. The map sheet reservoir DEM data files are stored in the corresponding folder. Files that are blank images after cropping are removed. For the map sheet reservoir DEM data files, the data conversion model of the visualization data stream processing platform is used, with the map sheet reservoir DEM data files as the requester and the target map sheet index files as the provider, to dynamically create standard map sheet vector features in batches and automatically fill attributes, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data files.

[0069] Specifically, based on geographic coordinates, the geographic extent of the individual reservoir's DEM data can be spatially intersected with the standard map sheet vector data. A spatial filter is then used to determine which standard map sheet boundaries the individual reservoir's DEM data crosses or falls within, thus obtaining all the standard map sheets involved in the individual reservoir's DEM data. Then, for each determined standard map sheet, the corresponding multi-level folder structure can be created in the output directory using the FME function (Feature Manipulation Engine) based on the map number. For details, please refer to [link to relevant documentation]. Figure 5 and Figure 6 , Figure 5 This is a schematic diagram of the operating interface of a system for generating a 1:10000 scale reservoir DEM, as disclosed in an embodiment of this application. Figure 6 This is a schematic diagram of a 1:10000 scale multi-level map sheet directory disclosed in an embodiment of this application. Next, coordinate projection transformation is used to place the vector data of the standard map sheet and the DEM data of individual reservoirs in the same plane coordinate system. Then, the boundary of the map sheet vector plane is extended outward by a certain distance (e.g., 100 meters) to ensure that the clipping range covers the map sheet boundary as completely as possible. The extended standard map sheet vector data (standard map sheet vector plane) is then stored in the corresponding folder. Next, using the extended and buffered map sheet vector plane as the clipping boundary, batch clipping is performed on the DEM data of individual reservoirs within this boundary range to generate map sheet reservoir DEM data files. These map sheet reservoir DEM data files are then stored in the corresponding folder. Files that are blank images after clipping are discarded. For details, please refer to [link to relevant documentation]. Figure 7 , Figure 7 This application discloses a schematic diagram of multiple individual reservoir DEMs cropped from a 1:10000 map sheet. Finally, using a data transformation model from a visualization data stream processing platform (e.g., the data transformation model specific to the FME platform), the cropped map sheet reservoir DEM data file is used as the requester, and the target map sheet index file is used as the supplier. Standard map sheet vector features are dynamically and batch-created, and attributes are automatically filled in, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file.

[0070] In this way, the standard map sheet range is automatically determined through spatial overlay analysis. Combined with coordinate projection unification, outward expansion buffering, and batch clipping, the DEM data of individual reservoirs is automatically processed into map sheets after being expanded according to the national standard map sheet. Then, the data conversion model of the visualization data stream processing platform is used to dynamically create map sheet vector elements in batches and automatically fill attributes, realizing a one-to-one accurate association between the map sheet reservoir DEM metadata and the clipped DEM data. This completes the standardized product packaging of reservoir DEM data, ensuring that the output product meets the requirements for water resources survey submission, and facilitates unified management, sharing and exchange, and seamless splicing with other standard map sheet data.

[0071] In one optional implementation, the data transformation model of the visualization data stream processing platform is used, with the map sheet reservoir DEM data file as the requester and the target map sheet index file as the provider, to dynamically and batch create standard map sheet vector features and automatically fill in attributes, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file. This includes: using the data transformation model of the visualization data stream processing platform, with the map sheet reservoir DEM data file as the requester and the target map sheet index file as the provider, and configuring the visualization data stream engine to perform data transformation based on each element provided by the provider. Data recording involves dynamically creating corresponding standard map sheet vector elements in batches on the requesting side, automatically creating map sheet attribute records, and automatically attaching attribute information from the map sheet reservoir DEM data files to the map sheet attribute records through an automatic attribute association engine. This generates map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data files. The automatically attached attribute information includes the top-left corner coordinates, number of rows, number of columns, production method of structured tabular data from multi-source heterogeneous water resources survey data, current status of the main data source, production date, and interpolation method attributes of the cropped map sheet reservoir DEM data.

[0072] Specifically, the metadata of a map sheet reservoir DEM refers to data describing the basic characteristics and background information of the map sheet reservoir DEM data file, such as its spatial range, coordinate system, and production method. It is used to explain and locate the content, source, and quality status of the map sheet reservoir DEM data. This implementation method uses a unique data transformation model of the FME platform (Feature Manipulation Engine) to achieve accurate, efficient, and one-to-one automatic association between the map sheet reservoir DEM metadata and the clipped DEM data for each map sheet reservoir DEM data file. During the linking process, the system uses the clipped map sheet reservoir DEM data file as the "Requestor" and the target map sheet index file as the "Supplier." Through configuration, the FME engine can dynamically and in batches create corresponding standard map sheet vector features in the Requestor based on each data record of the Supplier. This process eliminates the need for complex loop scripts, automating the entire process from data clipping to metadata record creation and attribute population. It automatically creates map sheet attribute records in the corresponding map sheet's index file (such as a Shapefile or graphic file). Subsequently, through an automatic attribute association engine, it automatically links the attribute information of the clipped map sheet reservoir DEM (including top-left corner coordinates, number of rows, number of columns, production method of structured tabular data from multi-source heterogeneous water resources survey data, current status of the main data source, production date, and interpolation method, etc.) to the map sheet attribute records, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file. For details, please refer to [link to relevant documentation]. Figure 8 , Figure 8 This is a schematic diagram of multiple Shapefile records (local fields) within a 1:10000 map sheet that correspond one-to-one with the DEM of a single reservoir, as disclosed in an embodiment of this application.

[0073] In this way, by utilizing the data transformation model of the visual data stream processing platform, dynamic batch creation and automatic attribute attachment of map sheet reservoir DEM metadata can be achieved. The entire process from data clipping to metadata generation can be automated without writing complex scripts, ensuring that the map sheet reservoir DEM metadata and the clipped DEM data are accurately, efficiently, and in a one-to-one correspondence, significantly improving the efficiency and accuracy of product packaging.

[0074] In one optional implementation, based on a preset water resources survey product catalog and normal water level inundation range rules, raster calculation, vector-to-plane conversion, area calculation, and DEM clipping operations are performed on the DEM data of individual reservoirs. Product packaging is then performed to generate DEM data of reservoirs with normal water level inundation ranges, inundation range vector surfaces, and a reservoir information table with added normal water level inundation area attributes. This includes: reading the normal water level attributes of reservoirs from structured tabular data from different sources in the water resources survey data; and using a raster calculator to sort the individual reservoir DEM data by elevation values ​​less than or equal to... The area at the normal water level is defined as the normal water level inundation range raster surface. The inundation range raster surface is converted into an inundation range vector surface. The area of ​​the inundation range vector surface is calculated. The vector surface is used to cut the individual reservoir DEM data to obtain the reservoir DEM data of the normal water level inundation range. A corresponding standard hierarchical folder structure is created in the output directory, with the reservoir code as the file name. The reservoir DEM data of the normal water level inundation range, the inundation range vector surface, and the reservoir information table with the added normal water level inundation area attribute are batch stored into the corresponding folder to generate a standardized spatial data product.

[0075] Specifically, the normal water level attribute refers to the highest water level elevation that a reservoir is allowed to store under normal operating conditions, a key parameter used to define the designed water storage range of the reservoir. A raster calculator is a tool that performs conditional judgments and mathematical operations on the pixel values ​​of raster data, used to extract raster areas that meet specific conditions according to preset rules. A reservoir information table is structured tabular data that records basic reservoir attribute information (including reservoir code, name, normal water level, normal storage capacity, etc.). In this embodiment of the application, it specifically refers to the reservoir information table after the addition of the normal water level inundation area attribute.

[0076] As an example, the process of generating a normal water level inundation range product can be as follows: Read the normal water level attribute of the loaded reservoir from structured tabular data from different sources of water resources survey data after access. Using a raster expression evaluator, the area in the individual reservoir DEM data whose elevation value is less than or equal to the normal water level attribute is determined as the inundation range raster surface. The calculation formula is: if(A[0]<= normal water level,1,0), that is, when the DEM cell elevation value is less than or equal to the normal water level, it is assigned a value of 1 (inundated area), otherwise it is assigned a value of 0 (non-inundated area). Convert the inundation range raster surface into an inundation range vector surface (using a raster to polygon coercer), and use an area calculator to calculate the area of ​​the vector surface. Use a clipper to clip the individual reservoir DEM data with the vector surface as the clipping boundary to obtain the normal water level inundation range reservoir DEM data. Create a corresponding standard hierarchical folder structure in the output directory, using the reservoir code as the filename. Batch save the DEM data of reservoirs within the normal water level inundation range, the inundation range vector surface, and the reservoir information table with the added normal water level inundation area attribute into the corresponding folders to generate a normal water level inundation range product (one type of standardized spatial data product in this application). The system can also automatically output final data products in various formats according to the standard and automatically organize them into a multi-level directory conforming to water resources survey standards, achieving "one-click" product packaging and output. Please refer to [link / reference] for details. Figure 9 and Figure 10 , Figure 9 This is a schematic diagram of the operating interface of a system for generating a single reservoir DEM, as disclosed in an embodiment of this application. Figure 10 This is a schematic diagram of a multi-unit reservoir DEM folder and its next-level files disclosed in an embodiment of this application.

[0077] In this way, the flooded area can be automatically extracted based on the normal water level and DEM data, vector surface and attribute information table can be generated, realizing the automated packaging and output of flooded area products. This provides ready-to-use standardized spatial data products for reservoir capacity calculation and land use analysis of flooded areas, which can significantly improve the efficiency and accuracy of water resources survey data productization.

[0078] To facilitate understanding of the embodiments of this application, the present application will be compared and described below with the prior art:

[0079] I. Existing technical solutions:

[0080] Existing automated processing methods for multi-source spatial data in water resource surveys mainly rely on manual switching between multiple software platforms such as ArcGIS, Excel, and FME. In ArcGIS, reservoir DEM (Digital Elevation Model) image data is imported and coordinates are manually transformed. In Excel, attribute tables containing normal water levels are organized and codes are manually checked. In FME, format conversion is manually configured. Then, the data is returned to ArcGIS for spatial overlay, attribute linking, and fusion of DEM and boundary vectors. Subsequently, map sheets are manually determined, DEMs are cropped one by one, and metadata is manually filled in to generate map sheet reservoir DEM products. Finally, water level attributes are manually extracted, raster calculations are performed, and vector conversions are performed to generate thematic products of inundation range.

[0081] II. Deficiencies of existing technologies:

[0082] 1. Difficulty in fusing multi-source heterogeneous data, requiring extensive manual intervention and resulting in low efficiency: The data formats are diverse, the coordinate systems and projections are inconsistent, and the attribute field definitions vary. It is necessary to frequently switch between multiple software platforms for manual conversion, projection, and attachment, which is prone to errors during large-scale processing.

[0083] 2. Data quality inspection methods are limited, standards are inconsistently applied, and the quality of results varies: There is a lack of automated capabilities to check the inherent logical consistency of multi-source data. It is impossible to automatically batch check whether the coding of the same reservoir is consistent across different source tables, or to quickly detect abnormal holes in the DEM or check the spatial matching between vector surfaces and DEM images. Quality inspection relies on manual visual inspection and experience-based judgment, making it difficult to systematically implement data standards, resulting in potential problems with attribute completeness, logical consistency, and spatial topological relationships in the results.

[0084] 3. Fragmented production processes and insufficient automation for standardized products: Producing standard reservoir DEM products requires manual identification of relevant map sheets, manual creation of catalogs, projection conversion and cropping of each map sheet, and manual filling of metadata tables. The process is tedious, repetitive, and difficult to ensure uniformity. Thematic products such as derived inundation ranges also require manual extraction of water level attributes, raster calculation, vector conversion, and area calculation, making it impossible to achieve rule-based batch automatic derivation.

[0085] 4. Weak metadata management, making data traceability and product reuse difficult: Data production information, processing history, attribute sources and other metadata are scattered or missing, and are not effectively and automatically linked to the spatial data ontology and stored in a structured manner, which makes data traceability difficult, lacks a basis for product credibility assessment, and is not conducive to long-term management and reuse.

[0086] III. Technical Solution of this Application:

[0087] This application provides an automated processing method for multi-source spatial data from water resource surveys. Based on a visual data stream processing platform, it creates a standardized processing template by dragging and dropping converter components and configuring component parameters and connections between components. The standardized processing template includes pre-set water resource survey data standards and product specifications, enabling fully automated processing. The method includes the following steps executed automatically in sequence: After acquiring multi-source heterogeneous water resource survey data, it integrates the water resource survey data, which at least includes initial reservoir DEM data and a null metadata table. Based on pre-set water resource survey data standard rules, it performs automated quality checks on the water resource survey data. Through an automatic attribute association engine, based on a preset unique reservoir identifier, it maps and attaches attributes to the quality-checked water resource survey data to generate a fused dataset. According to the conversion rules built into the standardized processing template, it performs attribute normalization and geometric transformation on the fused dataset to generate standard-compliant single-reservoir DEM data and its metadata. Based on a preset water resource survey product catalog and product encapsulation rules, it performs product encapsulation on the single-reservoir DEM data to generate standardized spatial data products.

[0088] IV. Technical effects of this application:

[0089] 1. Full-process automated processing, significantly reducing manual intervention: It connects the entire process from data access, quality inspection, attribute association, normalization and transformation to product packaging. Through a visual data flow processing platform, it can automatically generate standardized spatial data products with one or two clicks, greatly improving data processing efficiency and consistency.

[0090] 2. Automatic fusion of multi-source data to improve accuracy and completeness: Through the automatic attribute association engine, the system realizes automatic matching and attribute attachment of multi-source heterogeneous data based on the unique identifier of the reservoir, supplemented by spatial range relationship verification to ensure the accuracy of attribute association.

[0091] 3. Standardized products can be output flexibly to meet multi-level needs: It supports the automatic generation of map sheet reservoir DEM products and their metadata based on standard map sheet rules, and also supports the automatic encapsulation of thematic products based on normal water level inundation range rules (such as normal water level inundation range DEM data, vector surfaces and reservoir information tables), so as to realize the rapid generation of multiple types of standardized spatial data products.

[0092] 4. Multi-dimensional automated quality inspection to ensure data quality: Through pre-set water resource survey data standard rules, the system automatically performs checks on spatial topological consistency, logical consistency, attribute integrity, DEM data integrity, and elevation anomaly, systematically implementing data standards and providing a reliable foundation for subsequent processing.

[0093] 5. Low-code graphical development, reducing technical barriers and maintenance costs: Utilizing the FME graphical programming approach, the complex data loading and transformation process is visualized through an intuitive "building block" method, eliminating the need for extensive coding. Water resource survey data standards and product specifications are embedded as rules. Components such as conditional expression converters and automatic attribute association engines automatically perform attribute normalization, metadata creation, and product encapsulation. When standards are updated, only rule configuration needs to be adjusted, rather than modifying the underlying code, significantly reducing iteration and maintenance costs.

[0094] 6. Automatic metadata linking and structured storage to support data traceability and reuse: The automatic attribute association engine automatically links metadata such as production information and processing history to the spatial data ontology and stores it in a structured manner, so as to achieve data traceability, assess product credibility, and facilitate long-term management and reuse.

[0095] In summary, this application solves the prominent problems in the prior art, such as low automation, difficulty in implementing standardization, limited production efficiency, difficulty in ensuring quality consistency, and weak metadata management, by using pre-set standardized processing templates and rule-driven approaches. It achieves automation and standardization of multi-source spatial data processing for water resources surveys.

[0096] For further details, please refer to Figure 11 One embodiment of the automated processing system for multi-source spatial data from water resource surveys in this application includes:

[0097] The access module is used to acquire multi-source heterogeneous water resources survey data and then access the water resources survey data, which at least includes initial reservoir DEM data and a null data table.

[0098] The quality inspection module is used to perform automated quality inspection on the water resources survey data based on preset water resources survey data standard rules;

[0099] The attribute mapping and linking module is used to automatically associate attributes with the reservoir based on the preset unique identifier code of the reservoir, and perform attribute mapping and linking on the water resource survey data that has passed the quality inspection to generate a fused dataset.

[0100] The attribute normalization and geometric transformation module is used to perform attribute normalization and geometric transformation on the fused dataset according to the transformation rules built into the standardized processing template, and generate standard-compliant individual reservoir DEM data and its metadata.

[0101] The product packaging module is used to perform product packaging on the DEM data of the individual reservoir based on the preset water resources survey product catalog and product packaging rules, and generate standardized spatial data products.

[0102] In one alternative implementation, the productized packaging module can be used for:

[0103] Based on the preset water resources survey product catalog and standard map segmentation rules, the individual reservoir DEM data is segmented and cropped and metadata is created. Product packaging is then performed to generate map-based reservoir DEM products and their metadata. The map-based reservoir DEM products and their metadata are the standardized spatial data products.

[0104] Based on the preset water resources survey product catalog and normal water level inundation range rules, the DEM data of the individual reservoirs are subjected to raster calculation, vector-to-plane conversion, area calculation, and DEM clipping operations. Product packaging is then performed to generate reservoir DEM data with normal water level inundation range, inundation range vector surface, and reservoir information table with added normal water level inundation area attribute. The reservoir DEM data with normal water level inundation range, inundation range vector surface, and reservoir information table with added normal water level inundation area attribute constitute the standardized spatial data product.

[0105] In one alternative implementation, the productized packaging module can be used for:

[0106] Based on geographic coordinates, the DEM data of the individual reservoir is spatially overlaid with the standard map sheet vector data to determine the standard map sheet involved in the DEM data of the individual reservoir.

[0107] For each defined standard map sheet, a corresponding standard hierarchical folder structure is created in the output directory. The vector data of the standard map sheet is projected to the same plane coordinate system as the DEM data of the individual reservoir. The vector surface of the projected map sheet is then buffered and the buffered standard map sheet vector surface is stored in the corresponding folder.

[0108] Using the standard map sheet vector surface after expansion and buffering, the DEM data of the individual reservoirs falling within its range are batch-cropped to obtain map sheet reservoir DEM data files. The map sheet reservoir DEM data files are stored in the corresponding folder, and files that are blank images after cropping are removed.

[0109] For the map sheet reservoir DEM data file, the data transformation model of the visualization data stream processing platform is used to dynamically and batch create standard map sheet vector elements and automatically fill attributes, with the map sheet reservoir DEM data file as the requester and the target map sheet index file as the provider, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file.

[0110] In one alternative implementation, the productized packaging module can be used for:

[0111] Using the data transformation model of the visualization data stream processing platform, with the DEM data file of the reservoir as the requester and the target map sheet index file as the provider, the visualization data stream engine is configured to dynamically and batch create corresponding standard map sheet vector elements in the requester based on each data record of the provider, and automatically create map sheet attribute records.

[0112] The attribute auto-association engine automatically links the attribute information of the map sheet reservoir DEM data file to the map sheet attribute record, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file. The automatically linked attribute information includes the top-left corner coordinates, number of rows, number of columns, production method of the structured table data from the multi-source heterogeneous water resources survey data, current status of the main data source, production date, and interpolation method attribute of the cropped map sheet reservoir DEM data.

[0113] In one alternative implementation, the productized packaging module can be used for:

[0114] The normal water level attributes of the reservoir are read from structured tabular data from different sources in the water resources survey data;

[0115] Using a grid calculator, the area in the DEM data of the individual reservoir whose elevation value is less than or equal to the normal water level is determined as the grid surface of the normal water level inundation range;

[0116] The inundation range grid surface is converted into an inundation range vector surface, the area of ​​the inundation range vector surface is calculated, and the individual reservoir DEM data is cropped using the vector surface to obtain the reservoir DEM data of the normal water level inundation range.

[0117] Create a corresponding standard hierarchical folder structure in the output directory, using the reservoir code as the file name, and batch store the reservoir DEM data of the normal water level inundation range, the inundation range vector surface, and the reservoir information table with the added normal water level inundation area attribute into the corresponding folder to generate the standardized spatial data product.

[0118] In one optional implementation, the access module can be used to:

[0119] Determine the contour lines of the original topographic map of the reservoir area. If the contour lines are dense, construct a triangular network using the tool to create an irregular triangular network, and convert it into initial reservoir DEM data using the triangular network to raster tool. If the contour lines are sparse, generate initial reservoir DEM data using the terrain to raster tool.

[0120] Based on the water resources survey data standards, create a null metadata table corresponding to the initial reservoir DEM data;

[0121] In the visualization data stream processing platform, the water resources survey data is read in using a reader corresponding to the water resources survey data format. Then, the initial reservoir DEM data is converted into vector metadata surfaces using a raster-to-surface converter. Finally, the vector metadata surfaces are fused using a fusion converter according to the unique identifier of the reservoir to generate the reservoir extent surface.

[0122] In one alternative implementation, the quality inspection module can be used for:

[0123] A spatial filter converter is used to detect whether the vector metadata surface of the water resources survey data and the corresponding reservoir DEM spatially overlap, in order to determine whether the spatial location of the water resources survey data is consistent.

[0124] The element association and merging converter is used to match the structured tabular data from different sources in the water resources survey data according to the unique identifier of the reservoir, and the test converter is used to test whether the key field values ​​after matching are consistent, so as to determine the logical consistency of the water resources survey data.

[0125] The test converter is used to detect whether the necessary attribute fields of the water resources survey data, such as production method, normal water level and / or reservoir name, are missing, in order to determine whether the attribute information of the water resources survey data is complete.

[0126] The void extraction converter is used to detect whether there are void anomalies in the initial reservoir DEM data of the water resources survey data, and the raster band attribute extraction converter and test converter are used to determine whether the pixel value is missing, so as to determine whether the initial reservoir DEM data of the water resources survey data is complete.

[0127] A grid band no-data-value setting converter is used to uniformly mark the no-data-values ​​in the initial reservoir DEM data of the water resources survey data in order to determine whether there are any anomalies in the elevation information of the water resources survey data.

[0128] In one alternative implementation, the attribute mapping and hooking module can be used for:

[0129] The initial reservoir DEM data in the water resources survey data that has passed the quality inspection is converted from planar coordinates of different projection zones to geographic coordinates. The bounding rectangle of the converted reservoir DEM data is obtained and the latitude and longitude range of the bounding rectangle is calculated. Cross-zone judgment is made based on the latitude and longitude range.

[0130] Using the unique identifier of the reservoir in the water resources survey data that has passed the quality inspection as the key field, keyword precision matching is performed on the multi-source attribute data in the water resources survey data that has passed the quality inspection, and the matching results are verified in combination with the spatial range relationship to obtain the verified attribute data.

[0131] The verified attribute data is automatically appended to the metadata fields of the water resources survey data that has passed the quality check, thereby generating the fused dataset.

[0132] In one alternative implementation, the attribute normalization and geometric transformation module can be used for:

[0133] Using a conditional expression converter, the clauses of the attribute value domain in the standardized processing template are transformed into assignment rules containing conditional judgments. Based on the assignment rules, the attribute fields in the fused dataset are normalized and assigned values ​​to obtain the normalized fused dataset.

[0134] The date field in the normalized and assigned fused dataset is format-converted to obtain the date-format-converted fused dataset. The projection zone number is calculated based on the latitude and longitude range of the date-format-converted fused dataset.

[0135] Based on the projection zone number, the reservoir DEM data and vector data in the fused dataset after format conversion are projected to the target coordinate system, and geometric transformation and coordinate unification processing are performed to generate the standard-compliant individual reservoir DEM data and its metadata.

[0136] For further details, please refer to Figure 12 One embodiment of the electronic device in this application includes:

[0137] Central processing unit 1201, memory 1204, input / output interface 1203 and power supply 1202;

[0138] Memory 1204 is either a short-term storage memory or a persistent storage memory;

[0139] The central processing unit 1201 is configured to communicate with the memory 1204 and execute instructions in the memory 1204 to perform the aforementioned operations. Figure 1 The method in the illustrated embodiment.

[0140] Furthermore, embodiments of this application also provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to perform the aforementioned... Figure 1 The method in the illustrated embodiment.

[0141] Furthermore, embodiments of this application also provide a computer program product containing instructions, which, when run on a computer, causes the computer to perform the aforementioned... Figure 1 The method in the illustrated embodiment.

[0142] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0143] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0147] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. An automated processing method for multi-source spatial data from water resource surveys, characterized in that, The method is based on a visual data stream processing platform. It creates a standardized processing template by dragging and dropping converter components, configuring component parameters, and connecting components. The standardized processing template contains pre-set water resource survey data standards and product specifications, and performs fully automated processing, including the following steps executed automatically in sequence: After acquiring multi-source heterogeneous water resources survey data, the water resources survey data is accessed. The water resources survey data includes at least initial reservoir DEM data and null data tables. Based on pre-defined water resources survey data standards and rules, the water resources survey data is subjected to automated quality checks. Using an automatic attribute association engine, based on a preset unique identifier for the reservoir, water resource survey data that has passed quality inspection is mapped and linked to generate a fused dataset. Based on the transformation rules built into the standardized processing template, the fused dataset is subjected to attribute normalization and geometric transformation to generate standard-compliant individual reservoir DEM data and its metadata. Based on the preset water resources survey product catalog and product packaging rules, the DEM data of the individual reservoirs are productized and packaged to generate standardized spatial data products.

2. The method according to claim 1, characterized in that, Based on a preset water resources survey product catalog and product packaging rules, the DEM data of individual reservoirs is productized and packaged to generate standardized spatial data products, including at least one of the following: Based on the preset water resources survey product catalog and standard map segmentation rules, the individual reservoir DEM data is segmented and cropped and metadata is created. Product packaging is then performed to generate map-based reservoir DEM products and their metadata. The map-based reservoir DEM products and their metadata are the standardized spatial data products. Based on the preset water resources survey product catalog and normal water level inundation range rules, the DEM data of the individual reservoirs are subjected to raster calculation, vector-to-plane conversion, area calculation, and DEM clipping operations. Product packaging is then performed to generate reservoir DEM data with normal water level inundation range, inundation range vector surface, and reservoir information table with added normal water level inundation area attribute. The reservoir DEM data with normal water level inundation range, inundation range vector surface, and reservoir information table with added normal water level inundation area attribute constitute the standardized spatial data product.

3. The method according to claim 2, characterized in that, Based on a preset water resources survey product catalog and standard map segmentation rules, the individual reservoir DEM data undergoes map segmentation and metadata creation operations, and product packaging is performed to generate map-based reservoir DEM products and their corresponding map-based reservoir DEM metadata, including: Based on geographic coordinates, the DEM data of the individual reservoir is spatially overlaid with the standard map sheet vector data to determine the standard map sheet involved in the DEM data of the individual reservoir. For each defined standard map sheet, a corresponding standard hierarchical folder structure is created in the output directory. The vector data of the standard map sheet is projected to the same plane coordinate system as the DEM data of the individual reservoir. The vector surface of the projected map sheet is then buffered and the buffered standard map sheet vector surface is stored in the corresponding folder. Using the standard map sheet vector surface after expansion and buffering, the DEM data of the individual reservoirs falling within its range are batch-cropped to obtain map sheet reservoir DEM data files. The map sheet reservoir DEM data files are stored in the corresponding folder, and files that are blank images after cropping are removed. For the map sheet reservoir DEM data file, the data transformation model of the visualization data stream processing platform is used to dynamically and batch create standard map sheet vector elements and automatically fill attributes, with the map sheet reservoir DEM data file as the requester and the target map sheet index file as the provider, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file.

4. The method according to claim 3, characterized in that, The data transformation model utilizing the visualization data stream processing platform, with the map sheet reservoir DEM data file as the requester and the target map sheet index file as the provider, dynamically batch creates standard map sheet vector features and automatically fills in attributes, generating map sheet reservoir DEM metadata corresponding one-to-one with the map sheet reservoir DEM data file, including: Using the data transformation model of the visualization data stream processing platform, with the DEM data file of the reservoir as the requester and the target map sheet index file as the provider, the visualization data stream engine is configured to dynamically and batch create corresponding standard map sheet vector elements in the requester based on each data record of the provider, and automatically create map sheet attribute records. The attribute auto-association engine automatically links the attribute information of the map sheet reservoir DEM data file to the map sheet attribute record, generating map sheet reservoir DEM metadata that corresponds one-to-one with the map sheet reservoir DEM data file. The automatically linked attribute information includes the top-left corner coordinates, number of rows, number of columns, production method of the structured table data from the multi-source heterogeneous water resources survey data, current status of the main data source, production date, and interpolation method attribute of the cropped map sheet reservoir DEM data.

5. The method according to claim 2, characterized in that, Based on a preset water resources survey product catalog and normal water level inundation range rules, the individual reservoir DEM data undergoes raster calculation, vector-to-surface conversion, area calculation, and DEM cropping operations. Product packaging is then performed to generate reservoir DEM data for normal water level inundation ranges, inundation range vector surfaces, and a reservoir information table with added normal water level inundation area attributes, including: The normal water level attributes of the reservoir are read from structured tabular data from different sources in the water resources survey data; Using a grid calculator, the area in the DEM data of the individual reservoir whose elevation value is less than or equal to the normal water level is determined as the grid surface of the normal water level inundation range; The inundation range grid surface is converted into an inundation range vector surface, the area of ​​the inundation range vector surface is calculated, and the individual reservoir DEM data is cropped using the vector surface to obtain the reservoir DEM data of the normal water level inundation range. Create a corresponding standard hierarchical folder structure in the output directory, using the reservoir code as the file name, and batch store the reservoir DEM data of the normal water level inundation range, the inundation range vector surface, and the reservoir information table with the added normal water level inundation area attribute into the corresponding folder to generate the standardized spatial data product.

6. The method according to claim 1, characterized in that, The acquisition of multi-source heterogeneous water resource survey data includes at least initial reservoir DEM data and a null data table, including: Determine the contour lines of the original topographic map of the reservoir area. If the contour lines are dense, construct a triangular network using the tool to create an irregular triangular network, and convert it into initial reservoir DEM data using the triangular network to raster tool. If the contour lines are sparse, generate initial reservoir DEM data using the terrain to raster tool. Based on the water resources survey data standards, create a null metadata table corresponding to the initial reservoir DEM data; The access to the water resources survey data includes: In the visualization data stream processing platform, the water resources survey data is read in using a reader that corresponds to the format of the water resources survey data. After accessing the water resources survey data, the method further includes: The initial reservoir DEM data is converted into vector metadata surfaces using a raster-to-surface converter. The vector metadata surfaces are then fused using a fusion converter based on the reservoir's unique identifier to generate the reservoir extent surface.

7. The method according to claim 1, characterized in that, The automated quality check of the water resource survey data, based on pre-defined water resource survey data standard rules, includes at least one of the following: A spatial filter converter is used to detect whether the vector metadata surface of the water resources survey data and the corresponding reservoir DEM spatially overlap, in order to determine whether the spatial location of the water resources survey data is consistent. The element association and merging converter is used to match the structured tabular data from different sources in the water resources survey data according to the unique identifier of the reservoir, and the test converter is used to test whether the key field values ​​after matching are consistent, so as to determine the logical consistency of the water resources survey data. The test converter is used to detect whether the necessary attribute fields of the water resources survey data, such as production method, normal water level and / or reservoir name, are missing, in order to determine whether the attribute information of the water resources survey data is complete. The void extraction converter is used to detect whether there are void anomalies in the initial reservoir DEM data of the water resources survey data, and the raster band attribute extraction converter and test converter are used to determine whether the pixel value is missing, so as to determine whether the initial reservoir DEM data of the water resources survey data is complete. A grid band no-data-value setting converter is used to uniformly mark the no-data-values ​​in the initial reservoir DEM data of the water resources survey data in order to determine whether there are any anomalies in the elevation information of the water resources survey data.

8. The method according to claim 1, characterized in that, The automatic attribute association engine, based on a preset unique identifier for the reservoir, maps and links attributes to water resource survey data that has passed quality inspection, generating a fused dataset, including: The initial reservoir DEM data in the water resources survey data that has passed the quality inspection is converted from planar coordinates of different projection zones to geographic coordinates. The bounding rectangle of the converted reservoir DEM data is obtained and the latitude and longitude range of the bounding rectangle is calculated. Cross-zone judgment is made based on the latitude and longitude range. Using the unique identifier of the reservoir in the water resources survey data that has passed the quality inspection as the key field, keyword precision matching is performed on the multi-source attribute data in the water resources survey data that has passed the quality inspection, and the matching results are verified in combination with the spatial range relationship to obtain the verified attribute data. The verified attribute data is automatically appended to the metadata fields of the water resources survey data that has passed the quality check, thereby generating the fused dataset.

9. The method according to claim 1, characterized in that, The process involves performing attribute normalization and geometric transformation on the fused dataset according to the built-in transformation rules of the standardized processing template to generate standard-compliant individual reservoir DEM data and its metadata, including: Using a conditional expression converter, the clauses of the attribute value domain in the standardized processing template are transformed into assignment rules containing conditional judgments. Based on the assignment rules, the attribute fields in the fused dataset are normalized and assigned values ​​to obtain the normalized fused dataset. The date field in the normalized and assigned fused dataset is format-converted to obtain the date-format-converted fused dataset. The projection zone number is calculated based on the latitude and longitude range of the date-format-converted fused dataset. Based on the projection zone number, the reservoir DEM data and vector data in the fused dataset after format conversion are projected to the target coordinate system, and geometric transformation and coordinate unification processing are performed to generate the standard-compliant individual reservoir DEM data and its metadata.

10. An automated processing system for multi-source spatial data from water resource surveys, characterized in that, include: The access module is used to acquire multi-source heterogeneous water resources survey data and then access the water resources survey data, which at least includes initial reservoir DEM data and a null data table. The quality inspection module is used to perform automated quality inspection on the water resources survey data based on preset water resources survey data standard rules; The attribute mapping and linking module is used to automatically associate attributes with the reservoir based on the preset unique identifier code of the reservoir, and perform attribute mapping and linking on the water resource survey data that has passed the quality inspection to generate a fused dataset. The attribute normalization and geometric transformation module is used to perform attribute normalization and geometric transformation on the fused dataset according to the transformation rules built into the standardized processing template, and generate standard-compliant individual reservoir DEM data and its metadata. The product packaging module is used to perform product packaging on the DEM data of the individual reservoir based on the preset water resources survey product catalog and product packaging rules, and generate standardized spatial data products.

Images