Cultivated land basic data standardization management method

Abstract

The invention discloses a cultivated land basic data standardization management method. The method comprises the steps of collecting original data needing to be managed, conducting pre-check according to a pre-check strategy, finding out a data difference problem, and making a data analysis and processing strategy in combination with the data difference problem and cultivated land achievement data specifications. Based on a visual plug-in program, according to the strategy, a component script is called to process the data difference problem step by step. And storing the processed data into a cultivated land data result database according to cultivated land result data specifications. According to the method, the visual plug-in program and the componentized script library are constructed, a traditional data governance process depending on artificial experience is converted into a configurable and repeatable automatic processing assembly line, repeated manual operation is greatly reduced, continuous dependence on high-skill professionals is reduced, and the efficiency is improved. The data management efficiency and the automation level are improved, and the labor cost and the time period are remarkably reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of arable land basic data standardization, and in particular to an arable land basic data standardization treatment method. BACKGROUND

[0003] Arable land is the basis of food production, and strictly protecting the arable land red line can ensure the stability and sustainability of food production, thereby ensuring national food security. In order to clarify the rural land contract management right, improve the utilization efficiency of arable land, stabilize and perfect the rural land contract relationship, protect the legitimate rights and interests of farmers, and promote the stability of the rural society and the development of urban and rural integration.

[0004] Standard and accurate and effective arable land basic surveying and mapping data results are not only the basis for protecting the rural land contract management right, but also the indispensable data basis for the development of digital agriculture and smart agriculture.

[0005] The existing method for forming arable land basic data results is: first, professional basic geographic surveying and mapping personnel carry professional tools to conduct field surveying and mapping, and then professional GIS tools are used for indoor processing to form the results. This technical method relies on professional surveying and mapping tool software and GIS tools on the one hand, and relies on professional basic geographic surveying and mapping and GIS technical personnel on the other hand. Moreover, the indoor and outdoor work is mainly manual, which is high in cost and long in cycle.

[0006] Second, based on high-definition satellite remote sensing images and image interpretation technology, the changes of the current arable land relative to the previous period are identified to form change polygons, and then the change polygons are reviewed by manual indoor work to form change monitoring polygons. Finally, the change monitoring polygons are measured and reviewed according to the method one. This method makes full use of satellite remote sensing images and image interpretation technology, greatly reduces the surveying and mapping range, and thus greatly reduces the cost and shortens the cycle. However, the biggest bottleneck of this technology is to increase the workload of basic data treatment. The main reasons are: first, the old basic surveying and mapping results are inconsistent with the new surveying and mapping results in terms of specification requirements; second, the more institutions involved in surveying and mapping, the more inconsistent and larger deviations in the obtained surveying and mapping results, which increases the data treatment difficulty and workload.

[0007] The present application is a configurable automatic data treatment method based on the second existing method for forming arable land basic data results, which uses the advantages of Python language and data analysis technology in combination with professional GIS tools to realize efficient arable land basic data treatment and form standard, accurate and effective arable land basic surveying and mapping data results. SUMMARY

[0008] In view of the above problems, the purpose of the present application is to provide a farmland basic data standardization management method, which solves the problem of large data management workload caused by inconsistent basic surveying and mapping results and large deviation of multi-agency surveying and mapping results in existing farmland basic data management.

[0009] The first aspect is a farmland basic data standardization management method, comprising the following steps:

[0010] S1, collecting original data needing management;

[0011] S2, pre-inspecting the collected original data according to a pre-inspection strategy, and finding out data difference problems;

[0012] S3, formulating a data analysis and processing strategy according to the data difference problems and farmland result data specifications;

[0013] S4, based on a visual plug-in program, calling a component script to process the data difference problems in steps according to the data analysis and processing strategy;

[0014] S5, storing the processed data into a farmland data result database according to the farmland result data specifications.

[0015] In an embodiment of the present application, the original data includes three-adjustment farmland data, farmland change monitoring polygons and farmland supplementary precise surveying data.

[0016] In an embodiment of the present application, the pre-inspection process in S2 includes:

[0017] S21, observing the collected original data, and formulating a pre-inspection strategy according to experience;

[0018] S22, executing the pre-inspection strategy, obtaining a pre-inspection analysis result, and finding out data difference problems;

[0019] S23, manually verifying and confirming the found data difference problems;

[0020] S24, formulating a data analysis and processing strategy according to the verified and confirmed data difference problems.

[0021] In an embodiment of the present application, the component script obtaining step in S4 includes:

[0022] S41, based on ArcGIS Desktop or ArcGIS Pro, performing component encapsulation of tools in commonly used ArcTool box through an internal ArcPy package;

[0023] S42, realizing a visual plug-in program based on Python, and forming a component library by classifying management of various components;

[0024] S43, based on the visualization plug-in, the component is integrated and called, and is configured into a Python component script for process processing.

[0025] In one embodiment of the application, the component type includes:

[0026] Space class: coordinate system conversion, topology analysis and processing, overlap analysis and processing, merging and erasing;

[0027] Attribute class: uniqueness analysis and processing, null value analysis and processing, special character analysis and processing, date and time format processing, field name case conversion, field type conversion, field order adjustment, batch field deletion, batch field addition, graphic fitting assignment, graphic area assignment, area unit conversion, numerical precision processing, administrative boundary classification assignment and birth date assignment.

[0028] In one embodiment of the application, in S4, when the component script is called to process the data difference problem in steps,

[0029] Select part of the data for pilot operation, calibrate the component parameters based on the pilot operation results, reconstruct the Python component script, and process the batch data based on the reconstructed Python component script.

[0030] In one embodiment of the application, in S4, when the component script is called to process the data difference problem in steps, the execution log is recorded.

[0031] The beneficial effects of the application are:

[0032] 1. The method of the application converts the traditional data governance process relying on manual experience into a configurable and repeatable automated processing pipeline by constructing a visual plug-in program and a componentized script library, which greatly reduces repetitive manual operations and realizes 24-hour uninterrupted processing of batch data. Application examples show that for the governance of 3 million mu of cultivated land data, manual governance requires 12.5 days and the quality score is 79.15, while using the method of the application, it only takes 3 days to complete, the quality score is improved to 86.90, the efficiency is improved by more than 75%, and the continuous dependence on high-skilled professionals is reduced, the data governance efficiency and automation level are improved, and the manual cost and time cycle are significantly reduced.

[0033] 2. The method of the present application identifies data difference problems through pre-check strategy, formulates processing strategy based on standardized specifications, and finally executes by preset component script. The system ensures the consistency of each processing step through parameterized configuration, thereby ensuring the standardization and accuracy of the output data. The system supports pilot operation and parameter calibration mechanism, further optimizing the processing accuracy. The log recording function ensures the traceability of the processing process, providing a basis for quality inspection and avoiding the influence of personnel experience, technical level and operation habit in traditional manual processing.

[0034] 3. The component library design of the present application allows continuous accumulation and enrichment of processing functions, and can flexibly cope with various data sources and new specification requirements. Through visual plug-ins, non-programming professionals can also quickly adjust or combine component scripts to adapt to different governance needs. This architecture not only reduces the difficulty of system maintenance and update, but also leaves room for integrating more advanced analysis algorithms in the future.

[0035] 4. The method of the present application reduces manual intervention and ensures data consistency and accuracy. Manual processing often requires the simultaneous involvement of multiple people, and even after technical specification training, different personnel experience, technical level, and operation habits can cause result bias. Through condition setting and parameter configuration, the automatic governance program can achieve the consistency of the results of the unified processing process, and the automatic governance program of the present application can effectively avoid such problems. BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 The principle flowchart of the standardized governance method of the present application;

[0037] Figure 2 The flowchart of the standardized governance method of the present application;

[0038] Figure 3 The structural schematic diagram of the electronic device of the present application. DETAILED DESCRIPTION

[0039] The embodiments of the present application will be described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar symbols represent the same or similar elements or elements with the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as a limitation of the present application.

[0040] When existing cultivated land basic data results are formed, the basic data governance workload is large, the old basic surveying and mapping results are inconsistent with the new surveying and mapping results, and the participating surveying and mapping institutions are many. The more inconsistent places where surveying and mapping results are obtained, the greater the deviation, which increases the data governance difficulty and workload.

[0041] In view of the above problems, the application discloses a farmland basic data standardization management method, Figure 1 For the principle flowchart of the method, the method system architecture relies on component packaging tools and plug-in application implementation, and comprises:

[0042] The underlying tool adopts professional GIS software (such as ArcGIS Desktop / ArcGIS Pro), and rich spatial processing tools (ArcTool box) built in the GIS software.

[0043] The ArcTool box component is realized based on the packaging layer, common and scattered GIS processing tools are packaged into single-function and independent components by using ArcPy (the Python library of ArcGIS), and each component can be configured (such as input file, output path, processing rule and the like).

[0044] The component types comprise:

[0045] The spatial class comprises coordinate system conversion, topological analysis and processing, overlap analysis and processing, merging and erasing and the like;

[0046] The attribute class comprises uniqueness analysis and processing, null value analysis and processing, special character analysis and processing, date and time format processing, field name case conversion, field type conversion, field order adjustment, batch field deletion, batch field addition, graphic fitting assignment, graphic area assignment, area unit conversion, numerical precision processing, administrative boundary grading assignment and birth date assignment and the like.

[0047] Through continuous accumulation of the rich component library, more comprehensive and efficient support can be provided for the farmland basic data standardization management. With the increase of the component quantity and the refinement of the function, the system processing complex data problem capability will be greatly improved. Continuous updating and optimization of the component library can ensure that the system keeps up with the latest industry technology and standards. For newly appeared farmland data types and processing requirements, the corresponding components are added in time, so that the system always maintains leading applicability and functionality.

[0048] The plug-in application and flow processing are realized based on the application layer, and a visual plug-in program is developed. Users can drag or compile the packaged components into a complete Python component script on the visual plug-in program interface, for data processing flow. For example, each functional component such as “coordinate system conversion”, “data merging”, “topology check” and “field calculation” is packaged into an independent Python component script function.

[0049] Embodiment 1:

[0050] Based on the above processing architecture, the embodiment discloses a cultivated land basic data standardization management method, as shown in the following steps: Figure 2

[0051] S1, gather original data that needs to be managed.

[0052] The original data includes three-adjusted cultivated land data, cultivated land change monitoring polygons, and cultivated land supplementary precision measurement data, etc. The three-adjusted cultivated land data adopts the latest three-adjusted cultivated land data results, the cultivated land change monitoring polygons are data obtained through continuous dynamic monitoring, which can accurately reflect the change of cultivated land in different time periods. The cultivated land supplementary precision measurement data is detailed data obtained through high-precision measurement means for newly added cultivated land or cultivated land after reclamation.

[0053] The system will perform preliminary format verification and integrity check on these original data to ensure that the data meets the basic requirements for subsequent processing. For data that does not meet the format requirements, the system will issue a prompt message to guide the operator to perform data conversion or correction. After completing the preliminary check, the system will store the original data according to the data type and source for classification, providing convenience for subsequent data processing and analysis.

[0054] S2, pre-inspect the collected original data according to the pre-inspection strategy, and find out the data difference problems.

[0055] The analysis of the quality of the collected original data can be based on the experience of the operator to develop a pre-inspection strategy, which includes checking whether the original data has missing values, empty fields, etc.; checking whether the original data of different sources and different types is consistent in related indicators and definitions; checking whether the format, coding, etc. of the data conforms to the unified standard, such as date format, coding rules, etc. After developing these pre-inspection strategies, the system will conduct a comprehensive and detailed pre-inspection of the collected original data according to the strategies, and quickly find out the difference problems between the data through automatic program.

[0056] Then, according to the data difference problems found, manual verification is carried out to ensure the authenticity and accuracy of the problems. Since the automatic pre-inspection may have some misjudgments, manual verification can make more accurate judgments combined with professional knowledge and actual situation. In the process of manual verification, the staff will carefully check the specific situation of the data and analyze the possible reasons for the difference. For some problems that are difficult to judge directly, the source of the data will be further traced, and the data provider will be communicated to obtain more background information.

[0057] S3, according to the data difference problems and the cultivated land result data specification, develop data analysis and processing strategies.

[0058] ​According to the data difference problem confirmed by manual verification, develop data analysis and processing strategy, which is the basis of Python component script. It guides the writing and implementation of Python component script.

[0059] S4, based on the visualization plug-in, according to the data analysis and processing strategy, call component script to process data difference problem step by step.

[0060] According to the analysis and processing strategy formed, Python component script is formed on the system visualization plug-in interface. Based on Python component script, data flow processing is carried out. In the process of data flow processing, Python component script will automatically operate the data according to the established logic and rules.

[0061] For example, for the problem of missing data, the data analysis and processing strategy can focus on evaluating the impact of missing data on the overall data. If the proportion of missing data is small and has little effect on key indicators, the analysis and processing strategy can delete the records with missing data. If the proportion of missing data is large, interpolation methods such as mean interpolation, median interpolation or interpolation methods based on machine learning algorithms can be used for filling.

[0062] For example, for the problem that different sources and different types of data are inconsistent in related indicators and definitions, the data analysis and processing strategy needs to deeply study the meaning and calculation method of indicators of each data source, and find out the difference source. The analysis and processing strategy can develop unified indicator definition and calculation rules, and convert the data of each data source according to the new rules. In Python component script, custom functions can be written to convert and unify different component data.

[0063] For example, for the problem that data format and coding do not conform to the unified standard, the data analysis and processing strategy identifies the data format and coding type that does not conform to the standard. The analysis and processing strategy uses regular expression to match and correct the data format, and uses coding conversion function to convert the data coding to unified standard.

[0064] S5, the processed data is stored in the cultivated land data product database according to the cultivated land product data specification.

[0065] According to the latest cultivated land product data specification requirements, establish the final product data specification, fully investigate the platform or system business demand of data product utilization, plan to expand attributes, and establish the final result database.

[0066] According to the cultivated land achievement data specification, guide the formation data analysis and processing strategy, form the Python component script, carry out data flow processing to the data based on the Python component script Automatic operation, use the data interface in the system, store the processed data into the cultivated land data achievement database.

[0067] Further, the system is also provided with a log recording module, which records in detail each data processing operation, including operation time, operator, operation content and other information, and the system monitors and analyzes and preprocesses the log record, and when necessary, the process can be interrupted for manual intervention and processing, and then the interrupted step is executed again until the final expected success is achieved.

[0068] Application example:

[0069] XX city involves more than 300 million mu of cultivated land basic surveying and mapping achievement management of more than 10 teams, selects one team cultivated land surveying and mapping achievement through 2 people group data management time 12.5 days to complete data management, the achievement is analyzed and evaluated by quality inspection 79.15 points.

[0070] The same group of people, according to the previous data analysis and processing, develop an automatic data analysis strategy and carry out automatic management, and through 3 times of manual calibration, the remaining team data achievement management is completed in 3 days, and the achievement is analyzed and evaluated by quality inspection 86.90 points.

[0071] As can be seen from the application example, the method of the present application has obvious advantages in cultivated land basic surveying and mapping achievement management. The method of the present application can effectively reduce manual intervention, ensure the consistency and accuracy of data, and has high application value and popularization significance in the field of cultivated land basic data standardization management, can provide efficient and reliable solution for large-scale cultivated land basic surveying and mapping achievement management, and promote the development of cultivated land data management work to a more scientific and standardized direction.

[0072] Finally, it should be pointed out that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing examples, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for standardizing and managing basic farmland data, characterized in that, include: S1. Gather the raw data that needs to be managed; S2. Perform pre-inspection on the collected raw data according to the pre-inspection strategy to identify data discrepancies; S3. Develop data analysis and processing strategies based on data discrepancies and farmland achievement data standards; S4. Based on the visualization-based plug-in program, according to the data analysis and processing strategy, the component script is called to process the data difference problem step by step; S5. The processed data is stored in the cultivated land data results database in accordance with the cultivated land results data specifications.

2. The method according to claim 1, characterized in that, The raw data includes: the third national land survey data, farmland change monitoring patches, and farmland supplementary detailed survey data.

3. The method according to claim 1, characterized in that, The pre-detection process in S2 includes: S21. Observe the collected raw data and formulate a pre-inspection strategy based on experience; S22. Implement the pre-inspection strategy, obtain the pre-inspection analysis results, and identify data discrepancies. S23. For any discrepancies found in the data, manual verification and confirmation shall be conducted. S24. Based on the verified data discrepancies, formulate data analysis and processing strategies.

4. The method according to claim 1, characterized in that, The component script acquisition step in S4 includes: S41. Based on ArcGIS Desktop or ArcGIS Pro, use the built-in ArcPy package to encapsulate commonly used tools in the ArcToolbox by component. S42. Implement a visual plugin program based on Python to classify and manage various components to form a component library; S43. Based on the visual plugin program, the components are integrated and called, and the configuration is transformed into Python component scripts for process processing.

5. The method according to claim 4, characterized in that, The component types include: Spatial functions: coordinate system transformation, topology analysis and processing, overlap analysis and processing, merging and erasing; Attribute-related functions include: uniqueness analysis and processing, null value analysis and processing, special character analysis and processing, date and time format processing, field name case conversion, field type conversion, field order adjustment, batch deletion of fields, batch addition of fields, graphic overlay assignment, graphic area assignment, area unit conversion, number precision processing, administrative boundary level assignment, and birth date assignment.

6. The method according to claim 1, characterized in that, When calling the component script in S4 to process the data discrepancy issue step by step, A pilot run was conducted using selected data. Based on the pilot run results, the component parameters were calibrated, the Python component script was reconstructed, and batch data processing was performed based on the reconstructed Python component script.

7. The method according to claim 1, characterized in that, When the component script is called in S4 to handle the data difference problem step by step, an execution log is recorded.

Images