Natural resource asset checking result data storage method and device, terminal and medium
Through a multi-source data conversion engine, intelligent coordinate correction and semantic mapping toolbox, combined with Python and SQL technology, the fragmentation and lack of standards of natural resource asset inventory results are solved, efficient and standardized data storage and management are achieved, and data application efficiency is improved.
Patent Information
- Application Number
- CN202510504117.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-22
- Publication Date
- 2025-08-05
- Estimated Expiration
- 2045-04-22
AI Technical Summary
The data on the natural resource asset inventory results have problems such as data fragmentation, missing standards, lagging dynamic updates, and ineffective application performance, resulting in a trust crisis after data is put into the database and cannot meet the needs of refinement.
Through a multi-source data conversion engine, intelligent coordinate system correction, and semantic mapping toolbox, data format conversion, unified coordinate system and automatic data classification are realized, and data specification rules and logical verification matrix are built in combination with Python technology and SQL technology, and distributed object storage and geo-weighted principal component analysis are used for data storage.
The standardization of data on the inventory results of natural resource assets of the whole people has been realized, which has improved the efficiency and scientificity of data storage, reduced errors caused by artificial quality inspection, and improved data application scenarios and management efficiency.
Smart Images

Figure CN120429291A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, terminal and medium for storing data of natural resource asset inventory results. Background Art
[0002] The natural resource asset inventory is a fundamental and systematic project based on the unified registration of natural resource ownership. Through systematic investigation, monitoring, and accounting, it aims to comprehensively identify the core attributes of various natural resource assets, including quantity, quality, ownership, distribution, and value, and establish a dynamic update mechanism. Its essence is to transform natural resources from "physical resources" into quantifiable, manageable, and tradable "asset data," providing a scientific basis for natural resource management, ecological protection and restoration, and asset property rights reform.
[0003] The uploading and storing of data from natural resource asset inventory results is a core step in building a "one map" management system for natural resources. However, in actual implementation, it often faces multiple systemic challenges, such as data fragmentation, lack of standards, delayed dynamic updates, and low application efficiency. On the one hand, the data sources of natural resource asset inventory results are heterogeneous, with mixed formats of multi-source data, including remote sensing images, vector data, and tabular data. There are also problems such as inconsistent coordinate systems and attribute fields, resulting in low data fusion efficiency. On the other hand, there are defects in data quality, including typical topological errors, attribute logic contradictions, and other quality issues. This leads to a trust crisis in the application of data after it is stored in the database, and it cannot meet the needs of refinement. In addition, the data storage and management model is relatively backward, resulting in a lack of efficiency in data application. At the same time, the lack of a dynamic data update mechanism has led to the "zombieization" of data.
[0004] To effectively, standardized, and unified digitally manage inventory data, provide a foundational data source for natural resource asset management, facilitate the construction of a natural resource management system, and enhance digital governance capabilities, the accurate, efficient, standardized, and regular establishment of a natural resource asset inventory database has become an inevitable requirement for the current digital management of natural resources. Therefore, it is necessary to propose a method for uploading and storing all natural resource asset inventory data, unify the data storage process, manage the data in a unified manner, and visualize it through digital means. This will facilitate direct use by management departments, improve efficiency, expand data application scenarios, and support digital development.
[0005] This technology can effectively, quickly, accurately and standardizedly realize the standardized mapping and storage of data on the results of the inventory of natural resource assets owned by the whole people by constructing a standardized method and system for mapping and storing data on the results of the inventory of natural resource assets owned by the whole people, providing technical support for the establishment of an asset inventory database that meets the needs of natural resource management, and greatly improving the efficiency and scientificity of mapping and storing data on the results of the inventory of natural resource assets owned by the whole people. Summary of the Invention
[0006] In view of the shortcomings of the existing technology, the present invention proposes a method for storing data of natural resource asset inventory results, aiming to improve the digital management capabilities of natural resource asset inventory results data.
[0007] In a first aspect, an embodiment of the present application provides a method for storing data of natural resource asset inventory results, including:
[0008] Obtain resource asset inventory results data and basic element data;
[0009] Processing the resource asset inventory result data and the basic element data to obtain a standard asset inventory result data set;
[0010] Conduct full-chain quality inspection on the asset inventory results standard data set to obtain a quality inspection results standard data set;
[0011] The standard data set of quality inspection results is stored in a hierarchical manner to obtain an asset inventory results database.
[0012] Optionally, data processing is performed on the resource asset inventory result data and the basic element data to obtain an asset inventory result standard data set, including:
[0013] Performing data conversion, coordinate correction, and data classification processing on the resource asset inventory result data to obtain pre-processing result data;
[0014] The pre-processing result data and the basic element data are integrated to obtain a standard data set of asset inventory results.
[0015] Optionally, data conversion, coordinate correction, and data classification processing are performed on the resource asset inventory result data to obtain pre-processed result data, including:
[0016] Performing multi-source data format conversion on the resource asset inventory result data to obtain converted inventory result data;
[0017] The coordinates of the converted inventory result data are corrected by error compensation through the residual neural network model to obtain the corrected inventory result data;
[0018] The correction and inventory results data are classified to obtain pre-processed results data and a natural resource asset ontology library is constructed through dynamic knowledge graph technology.
[0019] Optionally, performing multi-source data format conversion on the resource asset inventory result data to obtain converted inventory result data includes: performing multi-source data format conversion on the resource inventory result data in a conventional format to obtain conventional converted inventory result data;
[0020] The format conversion model is used to convert the multi-source data format of resource inventory results data in non-standard formats to obtain non-standard converted inventory results data. The format conversion model is a deep learning model built using the Transformer architecture and is trained on historical non-standard format resource inventory results data.
[0021] Merge the conventional conversion inventory result data and the non-standard conversion inventory result data to obtain the conversion inventory result data;
[0022] Optionally, performing error compensation correction on the coordinates of the converted inventory result data through a residual neural network model to obtain corrected inventory result data includes:
[0023] Determine the original coordinate system type parameters, projection parameters and elevation benchmark parameters based on the conversion inventory results data;
[0024] The original coordinate system type parameters, projection mode parameters, and elevation benchmark parameters are input into a preset residual neural network model, and the obtained surveying and mapping control point result data are integrated into the residual neural network model to obtain corrected inventory result data; wherein the residual neural network model is used to perform error compensation correction on the coordinates of the converted inventory result data, and the residual neural network model is constructed by fitting the conversion model parameters by the least squares method and using Kalman filtering to achieve dynamic correction of the residual to obtain a coordinate error compensation correction model of a double closed-loop feedback supplementary algorithm;
[0025] Optionally, data classification is performed on the correction and inventory result data to obtain pre-processing result data, including:
[0026] Build a natural resource asset ontology database through dynamic knowledge graph technology;
[0027] Semantic features of the correction and inventory results data are extracted according to the natural resource asset ontology library through a named entity recognition model to obtain pre-processed results data after data classification. The named entity recognition model is a BiLSTM+CRF model.
[0028] Optionally, the pre-processing result data and the basic element data are integrated to obtain a standard data set of asset inventory results, including:
[0029] Obtain the inventory time range, inventory space range, and preset type classification;
[0030] Encode the pre-processing result data within the inventory time range and the inventory space range to obtain the spatial element identification code and type data identification code;
[0031] Divide the basic element data according to the inventory time range and inventory space range to obtain the time and space demand data;
[0032] The pre-processing result data is sorted according to the preset type classification, spatial element identification code, type data identification code, and time and space demand data to obtain a standard data set of asset inventory results.
[0033] Optionally, perform full-chain quality inspection on the asset inventory results standard dataset to obtain a quality inspection results standard dataset, including:
[0034] Use SQL technology to design quality inspection rule engine, build data standardization rules and logic check matrix;
[0035] Construct a data quality inspection database for asset inventory results based on data standardization rules and logic check matrix;
[0036] The data in the asset inventory result standard data set is called through the Python script in the asset inventory result data quality inspection library, and the data of the asset inventory result standard data set that has passed the quality inspection is extracted to obtain the quality inspection result standard data set.
[0037] Optionally, the standard data set of quality inspection results is stored in a hierarchical manner to obtain an asset inventory results database, including:
[0038] Establish an asset inventory results database including data storage layer, data index layer, quality assessment layer and comprehensive analysis layer;
[0039] Identify and classify the data in the quality inspection results standard data set according to the spatial element identification code and the type data identification code, and store the data in the quality inspection results standard data set in the data storage layer using distributed object storage technology;
[0040] Acquire the attribute table data corresponding to the data in the quality inspection result standard data set, and store the attribute table data corresponding to the data in the quality inspection result standard data set in the data index layer;
[0041] Performing quality assessment on the data in the quality inspection results standard data set using a data quality scoring model to obtain quality inspection report data, and storing the quality inspection report data in a quality assessment layer;
[0042] If a comprehensive analysis instruction is received, analysis report data is obtained through geographically weighted principal component analysis based on the information in the comprehensive analysis instruction, and the analysis report data is stored in the comprehensive analysis layer.
[0043] In a second aspect, an embodiment of the present application provides a device for storing data of natural resource asset inventory results, including:
[0044] Data acquisition module, used to obtain resource asset inventory results data and basic element data;
[0045] A data processing module, configured to process the resource asset inventory result data and the basic element data to obtain a standard data set of asset inventory results;
[0046] The data quality inspection module is used to perform full-chain quality inspection on the standard data set of asset inventory results to obtain a standard data set of quality inspection results;
[0047] The data storage module is used to store the standard data set of quality inspection results in a hierarchical manner to obtain the asset inventory results database.
[0048] In a third aspect, an embodiment of the present application provides a terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for storing data of natural resource asset inventory results as described in any one of the first aspects above is implemented.
[0049] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method for storing data of natural resource asset inventory results as described in any one of the first aspects above.
[0050] In a fifth aspect, an embodiment of the present application provides a computer program product. When the computer program product is run on a terminal device, the terminal device executes the method for storing data of natural resource asset inventory results described in any one of the first aspects above.
[0051] In the embodiment of this application, resource asset inventory results data and basic element data are obtained; data processing is performed on the resource asset inventory results data and the basic element data to obtain a standard asset inventory results data set; full-chain quality inspection is performed on the standard asset inventory results data set to obtain a standard quality inspection results data set; and the standard quality inspection results data set is stored in a hierarchical manner to obtain an asset inventory results database. This improves the digital governance capabilities of natural resource asset inventory results data.
[0052] This application completes data format conversion, unified coordinate system, and automatic data classification by establishing a multi-source data conversion engine, intelligent coordinate system correction, and semantic mapping toolbox, and realizes the automated preprocessing of data on the inventory results of various categories of natural resource assets owned by the whole people, solving the problem of heterogeneous multi-source data, forming a unified data foundation, and providing a standard data processing paradigm for stored data.
[0053] This application uses Python technology to call SQL technology to construct data standardization rules and logical verification matrix for the results of the national natural resource asset inventory, and constructs a standard asset inventory results data quality inspection library, which realizes the automated inspection of the national natural resource asset inventory results data, improves data quality inspection efficiency, and reduces data errors caused by manual quality inspection participation.
[0054] This application uses Python technology to adopt distributed object storage to complete data storage, and the data index layer adopts R-Tree spatial index. In the comprehensive analysis layer, local constraint learning is introduced to reduce the dimension of the data to improve the data response rate. A four-layer architecture system is used to complete the asset inventory results data mapping and storage. It greatly improves the efficiency and standardization of data storage, saves memory after data storage, increases the scientificity and reliability of the database, and improves the response efficiency after data storage, providing a new template for the mapping and storage of data on the results of the national natural resource asset inventory. BRIEF DESCRIPTION OF THE DRAWINGS
[0055] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.
[0056] Figure 1 This is a flow chart of a method for storing data on natural resource asset inventory results provided in one embodiment of the present application;
[0057] Figure 2 This is a flow chart of the second embodiment of the method for storing data on natural resource asset inventory results provided by this application;
[0058] Figure 3 This is a schematic diagram of the geographical unit data of the method for storing the natural resource asset inventory results data provided in this application;
[0059] Figure 4 This is a schematic diagram of the surface coverage data of the method for storing the results of the natural resource asset inventory provided in this application;
[0060] Figure 5This is a schematic diagram of the natural resource asset inventory results data after data classification of the natural resource asset inventory results data storage method provided in this application;
[0061] Figure 6 This is a schematic diagram of the structure of the device for storing data of natural resource asset inventory results provided in an embodiment of the present application;
[0062] Figure 7 It is a structural diagram of the terminal device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0063] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.
[0064] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.
[0065] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
[0066] The method for storing natural resource asset inventory results data in this embodiment can be executed by a natural resource asset inventory results data storage device. The natural resource asset inventory results data storage device acquires natural resource asset inventory results; integrates and processes the natural resource asset inventory results to obtain integrated results data; and uses the RETE algorithm to match the integrated results data with rules in a quality inspection rule database constructed based on the results data quality inspection rules to obtain data quality inspection results.
[0067] Figure 1 The following is a schematic flow chart of the natural resource asset inventory results data storage provided by the embodiment of the present application. As an example and not a limitation, this method can be applied to the above-mentioned natural resource asset inventory results data storage device, or it can be a method for users or operators to operate and judge on the natural resource asset inventory results data storage device. Figure 1 As shown, the method may include:
[0068] S10, obtaining resource asset inventory results data and basic element data;
[0069] In order to improve the digital management capabilities of natural resource asset inventory results data, the natural resource asset inventory results data storage device obtains natural resource asset inventory results data and basic element data.
[0070] Among them, the resource asset inventory results data includes resource category data and resource asset data corresponding to the resource category data; the resource category data includes: public land resource assets, public forest resource assets, public grassland resource assets, public wetland resource assets, mineral resource assets, public water resource assets, and marine resource assets; the resource asset data includes resource physical quantity data, asset value data, and usage rights information; among them, each resource category data contains the corresponding resource physical quantity data, asset value data, and usage rights information; the basic element data includes: geographic unit data, surface cover data, and three-zone and three-line data.
[0071] S20, processing the resource asset inventory result data and the basic element data to obtain a standard asset inventory result data set;
[0072] After obtaining the natural resource asset inventory result data and basic element data, the natural resource asset inventory result data storage device processes the resource asset inventory result data and the basic element data to obtain a natural resource asset inventory result standard data set.
[0073] Further, refer to Figure 2 , Figure 2 This is a flow chart of the second embodiment of the method for storing data of natural resource asset inventory results of the present invention. Figure 2 In the embodiment shown, data processing is performed on the resource asset inventory result data and the basic element data to obtain a standard asset inventory result data set, specifically including:
[0074] S21, performing data conversion, coordinate correction, and data classification processing on the resource asset inventory result data to obtain pre-processed result data;
[0075] After obtaining the natural resource asset inventory result data and basic element data, the natural resource asset inventory result data storage device performs data conversion, coordinate correction and data classification processing on the resource asset inventory result data to obtain pre-processed result data;
[0076] To address the heterogeneous nature of multi-source data, establish a unified data foundation, and provide a standardized data source for the final database, the acquired multi-source data from the public natural resource asset inventory will undergo automated pre-processing, primarily including multi-source data format conversion, coordinate system correction, and data classification.
[0077] As an implementation method, the resource asset inventory result data is subjected to data conversion, coordinate correction and data classification processing to obtain pre-processed result data, which may include: converting the resource asset inventory result data into a multi-source data format to obtain converted inventory result data; performing error compensation correction on the coordinates of the converted inventory result data through a residual neural network model to obtain corrected inventory result data; performing data classification on the corrected inventory result data to obtain pre-processed result data; and constructing a natural resource asset ontology library through dynamic knowledge graph technology.
[0078] As an implementation method, performing multi-source data format conversion on the resource asset inventory result data to obtain converted inventory result data may include: performing multi-source data format conversion on resource inventory result data in conventional format to obtain conventional converted inventory result data; performing multi-source data format conversion on resource inventory result data in non-standard format using a format conversion model to obtain non-standard converted inventory result data, wherein the format conversion model is a deep learning model established through a Transformer architecture and is obtained by training historical non-standard format resource inventory result data; merging the conventional converted inventory result data and the non-standard converted inventory result data to obtain converted inventory result data;
[0079] First, we established a multi-source data conversion engine, integrating over 100 format conversion methods to form a workflow for automated multi-format data conversion, such as converting Excel to GeoJSON data. This solves over 90% of multi-source data conversion challenges. To address the remaining data conversion challenges, we introduced the Transformer architecture to establish a deep learning-based format conversion model. This model automatically identifies field mappings in non-standard formats through historical data training, solving the remaining conversion challenges. By integrating existing methods to form a conversion chain and building a deep learning-based format conversion model, we achieve fully automated processing of multi-source data formats.
[0080] As an implementation method, the coordinates of the converted inventory result data are error compensated and corrected through a residual neural network model to obtain corrected inventory result data, which may include: determining the original coordinate system type parameters, projection method parameters and elevation reference parameters in the converted inventory result data; inputting the original coordinate system type parameters, projection method parameters and elevation reference parameters into a preset residual neural network model, and fusing the acquired surveying and mapping control point result data into the residual neural network model to obtain corrected inventory result data; wherein, the residual neural network model is used to error compensate and correct the coordinates of the converted inventory result data, and the residual neural network model is a coordinate error compensation correction model of a double closed-loop feedback supplementary algorithm constructed by fitting the conversion model parameters by the least squares method and realizing dynamic correction of the residuals by using Kalman filtering.
[0081] Secondly, an intelligent coordinate system correction tool was built. By developing an error-compensating coordinate conversion system, this error-compensating coordinate conversion system, based on a coordinate offset prediction model using a residual neural network, achieved millimeter-level coordinate conversion accuracy. This unified mathematical foundation for the data obtained from the national natural resource asset inventory, such as the conventional "2000 National Geodetic Coordinate System" (CGCS2000) and the "1985 National Elevation Datum," was used. The coordinate offset prediction model based on the residual neural network requires three parameters as input: the original coordinate system type, projection method, and elevation datum. It also integrates surveying and mapping control point data (GNSS base station data) to complete the coordinates predicted by the conversion model. To address millimeter-level coordinate errors, a dual closed-loop feedback supplementary algorithm was designed for adjustment. The first loop (outer loop) uses the least squares method to fit the conversion model parameters, while the second loop (inner loop) uses a Kalman filter to dynamically correct the residuals.
[0082] As an implementation method, classifying the correction and inventory results data to obtain pre-processed results data may include: constructing a natural resource asset ontology library through dynamic knowledge graph technology; extracting semantic features from the correction and inventory results data based on the natural resource asset ontology library through a named entity recognition model to obtain pre-processed results data after data classification, wherein the named entity recognition model is a BiLSTM+CRF model. Before extracting semantic features from the correction and inventory results data based on the natural resource asset ontology library through the named entity recognition model to obtain pre-processed results data after data classification, the method may include: constructing a semantic similarity matrix based on the natural resource asset ontology library; detecting the correction and inventory results data based on the semantic similarity matrix, and if a field naming rule of a new data source is detected, expanding the nodes of the natural resource asset ontology library; and if a field naming rule of a new data source is not detected, not expanding the nodes of the natural resource asset ontology library.
[0083] Further constructing semantic knowledge mapping, we introduced dynamic knowledge graph technology into the semantic mapping toolbox and established an ontology for the natural resource asset domain (including attribute semantic rule trees for resource types such as land, minerals, forests, grasslands, and wetlands). We first used the BiLSTM+CRF (a classic named entity recognition) model to extract semantic features from multi-source data fields. We then constructed a semantic similarity matrix based on the ontology (formula: Sim = α*A+β*B+γ*C, where A represents structural similarity; B represents semantic relevance; and C represents resource domain weight; the model coefficients α, β, and γ are dynamically adjusted through machine learning). We also developed an adaptive mapping engine that automatically expands the ontology nodes when field naming rules for new data sources are detected, ultimately achieving intelligent mapping. By constructing the ontology and employing the BiLSTM+CRF model to automatically distinguish data of the same resource type, we ultimately obtained asset inventory results data by resource type.
[0084] The multi-source data conversion engine, intelligent coordinate conversion system and semantic mapping tools can be used to truly realize the automated segmentation and classification pre-processing of the data obtained from the inventory of all natural resource assets owned by the whole people.
[0085] This application completes data format conversion, unified coordinate system, and automatic data classification by establishing a multi-source data conversion engine, intelligent coordinate system correction, and semantic mapping toolbox, and realizes the automated preprocessing of data on the inventory results of various categories of natural resource assets owned by the whole people, solving the problem of heterogeneous multi-source data, forming a unified data foundation, and providing a standard data processing paradigm for stored data.
[0086] S22, integrating the pre-processing result data and the basic element data to obtain a standard data set of asset inventory results.
[0087] After obtaining the pre-processing result data, the natural resource asset inventory result data storage device integrates the pre-processing result data and the basic element data to obtain an asset inventory result standard data set.
[0088] That is, the pre-processed data sets of the inventory results of various categories of natural resources assets owned by the whole people are obtained and integrated to generate a standard data set of the inventory results of natural resources assets owned by the whole people.
[0089] As an implementation method, the pre-processing result data and the basic element data are integrated to obtain a standard data set of asset inventory results, including: obtaining the inventory time range, inventory space range and preset type classification; encoding the pre-processing result data within the inventory time range and the inventory space range to obtain a spatial element identification code and a type data identification code; segmenting the basic element data according to the inventory time range and the inventory space range to obtain time and space demand data; organizing the pre-processing result data according to the preset type classification, spatial element identification code, type data identification code, and time and space demand data to obtain a standard data set of asset inventory results.
[0090] First, establish a unified inventory time range and space range. According to the time requirements for storage, clean up the inventory data of natural resource assets owned by the whole people that meet the time nodes; then, according to the requirements of the inventory space range, organize the inventory data of various categories of resource assets within the spatial range; then encode each piece of spatial data of natural resource assets owned by the whole people, determine the unique identification code of the spatial element (recorded as kjwybsm), and encode the table data, document data and image data corresponding to the spatial data. The unique codes of the elements are recorded as kjwybsm+bg, kjwybsm+wd, and kjwybsm+tx respectively. At the same time, the basic element data are segmented according to the time and space requirements to form geographic unit data, surface cover data and "three zones and three lines" data that meet the time and space requirements. Among them, geographic unit data such as Figure 3 As shown; the main data of land cover are as follows Figure 4 shown.
[0091] Subsequently, the data of the inventory of natural resources assets owned by all people are integrated according to the time and spatial scope of the inventory, such as Figure 5 As shown in the figure, a standard dataset for natural resource asset inventory results is generated by integrating and classifying 10 categories: state-owned agricultural land, state-owned construction land, state-owned unused land, state-owned construction land with undetermined user rights, minerals, state-owned forests, state-owned grasslands, state-owned wetlands, state-owned water resources, and marine resources. Each resource asset inventory dataset consists of data on the physical quantity of resources, asset value, and usage rights.
[0092] S30, perform full-chain quality inspection on the asset inventory results standard dataset to obtain a quality inspection results standard dataset;
[0093] After obtaining the standard data set of the asset inventory results, the natural resource asset inventory results data storage device performs full-chain quality inspection on the standard data set of the asset inventory results to obtain a standard data set of quality inspection results.
[0094] As an implementation method, a full-chain quality inspection is performed on the standard data set of asset inventory results to obtain the standard data set of quality inspection results, which can include: using SQL technology to design a quality inspection rule engine, and constructing data specification rules and a logical check matrix; constructing an asset inventory results data quality inspection library according to the data specification rules and the logical check matrix; calling the data in the standard data set of asset inventory results through a Python script in the quality inspection library of the asset inventory results data, and extracting the data of the standard data set of asset inventory results after quality inspection for data integrity, logical consistency, coordinate system consistency, attribute standardization, value range correctness, spatial reference, and time validity, to obtain the standard data set of quality inspection results.
[0095] As another implementation method, after extracting the data of the asset inventory results standard data set after quality inspection for data integrity, logical consistency, coordinate system consistency, attribute standardization, value range correctness, spatial reference, and time validity to obtain the quality inspection results standard data set, it can include: using a risk weight sampling algorithm to determine a dynamic sampling ratio based on data quality history records; sending the data in the quality inspection results standard data set to the quality inspection terminal according to the dynamic sampling ratio; and receiving the manual review record returned by the quality inspection terminal based on the data in the quality inspection results standard data set sent.
[0096] The standard data set of the inventory results of all natural resources assets owned by the whole people to be stored will be constructed, and the full-chain quality inspection of the data to be stored will be completed by combining automated inspection and manual review.
[0097] Automated quality inspection uses Python technology to develop batch inspection scripts, which conduct batch inspections by resource type, focusing on data integrity, logical consistency, coordinate system consistency, attribute standardization, value range correctness, spatial reference, time validity, and so on. SQL technology is used to design a quality inspection rule engine, construct data specification rules and a logical verification matrix for the results of the national natural resource asset inventory. A standard asset inventory results data quality inspection library is established, and the data quality inspection library is called through Python scripts to complete the automated inspection of the national natural resource asset inventory results data. If the automated inspection fails, the process returns to execute S21 and S22 for data processing, re-preprocessing and data integration, until all the results data to be stored pass the automated inspection.
[0098] After completing the automated data check, manual review is used to complete the data review before storage, so as to achieve full-chain quality inspection of the data before storage. Manual review is carried out by random sampling. Python is also used to build a random sampling script for the results of natural resource assets owned by all people. A risk-weighted sampling algorithm is used to dynamically adjust the sampling ratio (such as 5% for minerals and 3% for forests) based on the data quality history of the resource type (such as the high error rate of mineral data). According to this rule, the sampling ratio of each resource type within the inventory scope is allocated, and then automatic sampling is carried out to improve the efficiency of quality inspection. The sampled data is handed over to professional data quality inspection staff for quality inspection one by one.
[0099] The data quality inspection process (including automated results and manual review records) is stored on the chain to ensure that the quality inspection process is traceable and cannot be tampered with, meeting the needs of natural resource data auditing.
[0100] This application uses Python technology to call SQL technology to construct data standardization rules and logical verification matrix for the results of the national natural resource asset inventory, and constructs a standard asset inventory results data quality inspection library, which realizes the automated inspection of the national natural resource asset inventory results data, improves data quality inspection efficiency, and reduces data errors caused by manual quality inspection participation.
[0101] S40, the quality inspection results standard data set is stored in a hierarchical manner to obtain an asset inventory results database.
[0102] After obtaining the standard data set of quality inspection results, spatial element identification code, and type data identification code, the natural resource asset inventory result data storage device stores the standard data set of quality inspection results in a hierarchical manner to obtain an asset inventory result database.
[0103] As an implementation method, the standard data set of quality inspection results is stored in a layered manner to obtain an asset inventory results database, which may include: establishing an asset inventory results database including a data storage layer, a data index layer, a quality assessment layer and a comprehensive analysis layer; performing data identification and classification on the data in the standard data set of quality inspection results according to the spatial element identification code and the type data identification code, and using distributed object storage technology to store the data in the standard data set of quality inspection results in the data storage layer; obtaining the attribute table data corresponding to the data in the standard data set of quality inspection results, and storing the attribute table data corresponding to the data in the standard data set of quality inspection results in the data index layer; performing quality assessment on the data in the standard data set of quality inspection results through a data quality scoring model to obtain quality inspection report data, and storing the quality inspection report data in the quality assessment layer, where the data quality scoring model is a multi-dimensional quality assessment indicator system, including quality scores for data integrity, accuracy, timeliness, consistency and traceability; if a comprehensive analysis instruction is received, obtaining analysis report data through geographically weighted principal component analysis based on the information in the comprehensive analysis instruction, and storing the analysis report data in the comprehensive analysis layer.
[0104] The datasets that have passed quality inspection and are currently being stored in layers. The data from the national natural resource asset inventory are diverse, diverse, and large in volume. Therefore, a layered approach is used to store the data. This layered approach consists of a four-tiered architecture: data storage, data indexing, quality assessment, and comprehensive analysis.
[0105] The data storage layer serves as the core data foundation for the database after data is stored. It uses unique identifiers to identify and classify data, employing Python technology and distributed object storage. This ultimately creates a distributed spatial database capable of supporting petabyte-level data storage, addressing the challenge of massive data storage. To address data updates, data is stored in a tiered system: hot and cold data (the latest inventory results or updated portions) is stored in an in-memory database to achieve millimeter-level responsiveness, while cold data (historical versions of inventory results) is stored in a distributed columnar storage system with compression to reduce petabyte-level storage costs. Furthermore, data is decoupled through physical and logical stratification, categorizing data by result type into spatial data, tabular data, document data, and image data. Tabular data, document data, and image data are connected to spatial data using mapping rules and stored as supplementary information for spatial data. Spatial data is physically separated into a base layer (including geographic unit data, land cover data, and "three zones and three lines" data) and a thematic layer (data on the inventory results of all natural resource assets, stored by resource category). The GeoMesa framework is used to uniformly encode spatial data and attribute data into a space-time cube, realizing spatial-attribute integrated storage and improving the efficiency of subsequent related queries.
[0106] The data indexing layer creates a composite spatial index, utilizing a combination of R-Tree and Geohash technology to build a two-layer spatial index structure. This indexes the inventory results data after storage. User-adaptive indexes can be customized based on the needs of natural resource management or indexed based on resource directories, enabling rapid response to terabyte-scale data queries. R-Tree is a self-balancing tree-like data structure used to store spatial objects with multidimensional coordinates. It uses the minimum bounding box (MBR) as the basis for spatial indexing, organizing data in a hierarchical tree structure to ensure that irrelevant objects can be quickly filtered out during queries. Through R-Tree, all data on the inventory results of all natural resource assets owned by the whole people is stored based on its bounding rectangle. Intermediate nodes aggregate the MBRs of lower-level nodes to form higher-dimensional spatial index regions. This structure achieves efficient retrieval through hierarchical nesting of bounding boxes. Queries only require comparing bounding boxes to locate target data, eliminating the need to search underlying records one by one. This significantly reduces the computational complexity of querying data on the inventory results of all natural resource assets owned by the whole people. In addition, based on the spatial indexing using R-Tree technology, a resource type weight factor is introduced to prioritize indexing high-frequency query resources, such as construction land, which can reduce the retrieval time of MBR overlapping areas and further improve indexing efficiency.
[0107] The quality assessment layer builds a multi-dimensional quality assessment indicator system, including data integrity, accuracy, timeliness, consistency, and traceability, and thus designs a data quality scoring model. The formula is as follows:
[0108] Q=a1*W+a2*Z+a3*S+a4*Y+a5*K
[0109] Where a1, a2, a3, a4, and a5 are the coefficients of the data quality scoring model. The model is adaptively and dynamically adjusted through machine learning to reflect the data quality priorities of different resource types. W represents data integrity; Z represents data accuracy; S represents data timeliness; Y represents data consistency; and K represents data traceability. Based on the data quality scoring model, a data quality evaluation matrix for the results of the national natural resource asset inventory is established, and a data quality evaluation report is generated. Furthermore, the quality inspection process (including automated results and manual review records) is stored to ensure that the data quality inspection process is traceable and cannot be tampered with, meeting the requirements of natural resource data audits and establishing data blockchain for evidence storage and traceability.
[0110] The comprehensive analysis layer conducts a comprehensive analysis of the data from the national natural resource asset inventory, which has been stored in the database. This layer provides guidance for its application. The national natural resource asset inventory data encompasses multiple resource categories and is high in dimensionality. To improve the efficiency of comprehensive data analysis, dimensionality reduction is necessary for this multi-resource data. However, existing dimensionality reduction methods fail to account for the spatial heterogeneity (spatial autocorrelation) of natural resource asset inventory data. Therefore, Geographically Weighted Principal Component Analysis (GWPCA) is employed here to reduce the dimensionality of the spatial data. This reduces the impact of data dimensionality (multiple resource categories) and ensures that resource data from adjacent regions maintain clustering in a low-dimensional space. This improves the stability and reliability of the database, thereby enhancing the accuracy and efficiency of comprehensive big data analysis. Geographically Weighted Principal Component Analysis (GWPCA) differs from standard Principal Component Analysis (PCA) in that a geographic weight matrix is incorporated into the calculation of the covariance matrix. PCA generates new variables through linear combinations of the original variables, significantly reducing the number of variables included in the model and achieving data dimensionality reduction while preserving the majority of information. GWPCA uses kernel weighting and geographic weighting to find localized principal components at the target location. At the target location, the neighboring observations are weighted using a distance decay weighting function, and then standard principal component analysis is applied locally to its specific weighted data subset. The specific principles are as follows:
[0111] 1. Geographically weighted variance-covariance matrix at the target location: Σ(u,v)=X T W(u,v)X, where W(u,v) is the diagonal matrix of geographic weights generated by the selected kernel weighting function, and X represents the asset inventory results data matrix at the target area.
[0112] 2. GWPCA of a certain position i in space is calculated as follows: LVL T |(u i ,vi )=∑(u i ,v i ) where L is the matrix of n×m dimensional eigenvectors and V is the diagonal matrix of eigenvalues.
[0113] 3. The score matrix of the same position i: T(u i ,v i )=XL(u i ,v i );
[0114] 4. Divide each local eigenvalue by tr(V(u i ,v i )), tr(·) is the abbreviation of trace, which means the trace of the matrix. The trace of the matrix refers to the sum of the main diagonal elements of a square matrix, which can find the localized version of the proportion of each component to the total variance in the original data.
[0115] The window size of this localized GWPCA application is controlled by the kernel bandwidth, i.e. the geographically weighted window size. Small bandwidths make the spatial variation of the results faster, while large bandwidths produce results that are increasingly close to global principal component analysis.
[0116] As another embodiment, after the standard data set of quality inspection results is stored in a hierarchical manner to obtain an asset inventory results database, the method may include: receiving a results data update instruction, and updating the asset inventory results database according to the results data update instruction. The results data update instruction includes a data modification instruction and a data deletion instruction. When a data modification instruction is received, the query data corresponding to the data modification instruction is found through the data index layer, and the correct data in the data modification instruction replaces the data at the corresponding position in the asset inventory results database, and inherits the unique identification code of the original data. When a data deletion instruction is received, the query data corresponding to the data modification instruction is found through the data index layer, and the data at the corresponding position in the asset inventory results database in the data deletion instruction is deleted.
[0117] That is, according to the requirements of the national natural resource asset inventory task, the corresponding asset inventory results data will be updated and stored in the database according to the time task node, and the modification and deletion of the stored data will also be included. The asset inventory results data are updated and stored in the database according to steps S21, S22, and S30, and the updated data of the natural resource asset inventory results of the whole people at the new time node are finally completed. At the same time, the stored asset inventory results data are divided into blocks and stored according to the annual time node. The modification and deletion of the stored data first finds the questionable data through the data index layer, and then replaces the correct data into the database (the data to be deleted is assigned a null value), inherits the unique identification code of the original data, and completes the modification and deletion of the questionable data.
[0118] In summary, the following steps involve obtaining resource asset inventory results data and basic element data; processing these data to obtain a standard dataset for asset inventory results; conducting full-chain quality inspection on the standard dataset to obtain a standard dataset for quality inspection results; and storing the standard dataset for quality inspection results in a hierarchical manner to obtain an asset inventory results database. This improves the digital governance capabilities of natural resource asset inventory results data.
[0119] This application uses Python technology to adopt distributed object storage to complete data storage, and the data index layer adopts R-Tree spatial index. In the comprehensive analysis layer, local constraint learning is introduced to reduce the dimension of the data to improve the data response rate. A four-layer architecture system is used to complete the asset inventory results data mapping and storage. It greatly improves the efficiency and standardization of data storage, saves memory after data storage, increases the scientificity and reliability of the database, and improves the response efficiency after data storage, providing a new template for the mapping and storage of data on the results of the national natural resource asset inventory.
[0120] In line with the above, please see Figure 6 , Figure 6 The present invention provides a schematic diagram of the structure of a device for storing data of natural resource asset inventory results. Figure 6 As shown, the device includes:
[0121] Data acquisition module 601, used to obtain resource asset inventory results data and basic element data;
[0122] The data processing module 602 is used to process the resource asset inventory result data and the basic element data to obtain a standard data set of asset inventory results;
[0123] The data quality inspection module 603 is used to perform full-chain quality inspection on the asset inventory result standard data set to obtain a quality inspection result standard data set;
[0124] The data storage module 604 is used to store the quality inspection results standard data set in a hierarchical manner to obtain an asset inventory results database.
[0125] An embodiment of the present application also provides a terminal device, which includes: at least one processor, a memory, and a computer program stored in the memory and capable of running on the at least one processor. When the processor executes the computer program, the steps in the embodiment of the method for storing data of natural resource asset inventory results are implemented.
[0126] An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program enables the computer to execute part or all of the steps of any method for storing data of natural resource asset inventory results as recorded in the above method embodiments.
[0127] An embodiment of the present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and the computer program enables a computer to execute part or all of the steps of any method for storing data of natural resource asset inventory results as recorded in the above method embodiments.
[0128] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, a computer-readable storage medium cannot be an electric carrier signal or a telecommunication signal.
[0129] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.
[0130] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0131] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0132] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
Claims
1. A method for storing data on natural resource asset inventory results, characterized in that: include: Obtain resource asset inventory results data and basic element data; Processing the resource asset inventory result data and the basic element data to obtain a standard asset inventory result data set; Conduct full-chain quality inspection on the asset inventory results standard data set to obtain a quality inspection results standard data set; The standard data set of quality inspection results is stored in a hierarchical manner to obtain an asset inventory results database.
2. The method for storing data of natural resource asset inventory results according to claim 1 is characterized in that: The resource asset inventory result data and the basic element data are processed to obtain a standard data set of asset inventory results, including: Performing data conversion, coordinate correction, and data classification processing on the resource asset inventory result data to obtain pre-processing result data; The pre-processing result data and the basic element data are integrated to obtain a standard data set of asset inventory results.
3. The method for storing data of natural resource asset inventory results according to claim 2 is characterized in that: The resource asset inventory result data is subjected to data conversion, coordinate correction and data classification processing to obtain pre-processing result data, including: Performing multi-source data format conversion on the resource asset inventory result data to obtain converted inventory result data; The coordinates of the converted inventory result data are corrected by error compensation through the residual neural network model to obtain the corrected inventory result data; The correction and inventory results data are classified to obtain pre-processed results data and a natural resource asset ontology library is constructed through dynamic knowledge graph technology.
4. The method for storing data of natural resource asset inventory results according to claim 3 is characterized in that: Performing multi-source data format conversion on the resource asset inventory result data to obtain converted inventory result data, including: performing multi-source data format conversion on the resource inventory result data in a conventional format to obtain conventional converted inventory result data; The format conversion model is used to convert the multi-source data format of resource inventory results data in non-standard formats to obtain non-standard converted inventory results data. The format conversion model is a deep learning model built using the Transformer architecture and is trained on historical non-standard format resource inventory results data. Merge the conventional conversion inventory result data and the non-standard conversion inventory result data to obtain the conversion inventory result data; Alternatively, the coordinates of the converted inventory result data are subjected to error compensation correction through a residual neural network model to obtain corrected inventory result data, including: Determine the original coordinate system type parameters, projection parameters and elevation benchmark parameters based on the conversion inventory results data; The original coordinate system type parameters, projection mode parameters, and elevation benchmark parameters are input into a preset residual neural network model, and the obtained surveying and mapping control point result data are integrated into the residual neural network model to obtain corrected inventory result data; wherein the residual neural network model is used to perform error compensation correction on the coordinates of the converted inventory result data, and the residual neural network model is constructed by fitting the conversion model parameters by the least squares method and using Kalman filtering to achieve dynamic correction of the residual to obtain a coordinate error compensation correction model of a double closed-loop feedback supplementary algorithm; Alternatively, data classification is performed on the correction and inventory result data to obtain pre-processing result data, including: Build a natural resource asset ontology database through dynamic knowledge graph technology; Semantic features of the correction and inventory results data are extracted according to the natural resource asset ontology library through a named entity recognition model to obtain pre-processed results data after data classification. The named entity recognition model is a BiLSTM+CRF model.
5. The method for storing data of natural resource asset inventory results according to any one of claims 1 to 4, characterized in that: The pre-processing result data and the basic element data are integrated to obtain a standard data set of asset inventory results, including: Obtain the inventory time range, inventory space range, and preset type classification; Encode the pre-processing result data within the inventory time range and the inventory space range to obtain the spatial element identification code and type data identification code; Divide the basic element data according to the inventory time range and inventory space range to obtain the time and space demand data; The pre-processing result data is sorted according to the preset type classification, spatial element identification code, type data identification code, and time and space demand data to obtain a standard data set of asset inventory results.
6. The method for storing data of natural resource asset inventory results according to claim 5 is characterized in that: Perform full-chain quality inspection on the asset inventory results standard dataset to obtain the quality inspection results standard dataset, including: Use SQL technology to design quality inspection rule engine, build data standardization rules and logic check matrix; Construct a data quality inspection database for asset inventory results based on data standardization rules and logic check matrix; The data in the asset inventory result standard data set is called through the Python script in the asset inventory result data quality inspection library, and the data of the asset inventory result standard data set that has passed the quality inspection is extracted to obtain the quality inspection result standard data set.
7. The method for storing data of natural resource asset inventory results according to claim 5 is characterized in that: The standard data set of quality inspection results is stored in a hierarchical manner to obtain the asset inventory results database, including: Establish an asset inventory results database including data storage layer, data index layer, quality assessment layer and comprehensive analysis layer; Identify and classify the data in the quality inspection results standard data set according to the spatial element identification code and the type data identification code, and store the data in the quality inspection results standard data set in the data storage layer using distributed object storage technology; Acquire the attribute table data corresponding to the data in the quality inspection result standard data set, and store the attribute table data corresponding to the data in the quality inspection result standard data set in the data index layer; Performing quality assessment on the data in the quality inspection results standard data set using a data quality scoring model to obtain quality inspection report data, and storing the quality inspection report data in a quality assessment layer; If a comprehensive analysis instruction is received, analysis report data is obtained through geographically weighted principal component analysis based on the information in the comprehensive analysis instruction, and the analysis report data is stored in the comprehensive analysis layer.
8. A device for storing data of natural resource asset inventory results, characterized in that: include: Data acquisition module, used to obtain resource asset inventory results data and basic element data; A data processing module, configured to process the resource asset inventory result data and the basic element data to obtain a standard data set of asset inventory results; The data quality inspection module is used to perform full-chain quality inspection on the standard data set of asset inventory results to obtain a standard data set of quality inspection results; The data storage module is used to store the standard data set of quality inspection results in a hierarchical manner to obtain the asset inventory results database.
9. A terminal device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, it implements the method for storing data of natural resource asset inventory results as described in any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for storing data of natural resource asset inventory results as described in any one of claims 1 to 7 is implemented.
Citation Information
Patent Citations
Encoding method and apparatus for low density parity-check code
CN101340192A
Data management method and system for land conservation and intensive utilization
CN119597823A
Computer Controlled Video Game Incorporating Constraints
US20110230258A1
Recording medium for an optical data memory, especially a DVD+ / -r
WO2004097821A1
Cited By
Natural resource right confirmation registration management system based on big data
CN120912409A