Management method, system and equipment for experimental test data and medium
By establishing the association between primitives and table structures in WebGIS and PostgreSQL, the correlation problem between experimental test data and original data is solved, efficient data management and query are achieved, and data utilization efficiency and security are improved.
Patent Information
- Application Number
- CN202411179126.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-27
- Publication Date
- 2024-11-26
AI Technical Summary
In the prior art, the experimental test data cannot be quickly viewed after visualization, resulting in inefficient management.
Establish element and attribute information in the geographic information system WebGIS, use interpolation algorithm to generate grid data, and generate trend charts through contour tracing technology. Create table structures in the object-related database PostgreSQL and insert foreign keys to implement experimental testing The association between data, trend charts, primitives and research projects allows to view each other's correlation by clicking.
It realizes the interrelationship between experimental test data and trend charts, primitives and research projects, improves the efficiency of data management and visual query capabilities, reduces the workload of researchers in sorting and comparing analysis, saves labor costs and improves data storage security and utilization efficiency.
Smart Images

Figure CN119025608A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data management, and in particular to a management method, system, equipment and medium for experimental test data. Background Art
[0002] The project scope of the Mineral Resources Survey Center covers multiple fields such as geology, ecological restoration and natural resources, and over the years, a large amount of experimental test data of different types such as rock, soil, water, etc. These experimental test data are extremely large and complex, and there is a lack of effective management of historical experimental data.
[0003] In the prior art, method 1 stores and manages various experimental test data through form-based management. Method 2 is based on the integrated management of geographic information system GIS graphic data and thematic database and attribute data, making the experimental test data visual and able to efficiently retrieve, browse, store and process data.
[0004] However, the data management method in the prior art manages the experimental test data in a decentralized manner, and after the experimental test data is visualized, it is impossible to quickly view the visualized experimental data and the original experimental test data. Summary of the invention
[0005] The embodiments of the present invention provide a method, system, device and medium for managing experimental test data, which can solve the problem in the prior art that it is impossible to quickly view the visualized experimental data and the original experimental test data by clicking.
[0006] The embodiment of the present invention provides a data management method, comprising the following steps: in a geographic information system WebGIS, different types of primitives and attribute information corresponding to the primitives are established according to different sampling points in a research project, wherein the attribute information is experimental test data of the sampling points; a layer is created according to different types of primitives and a range is delineated on the created layer, an interpolation algorithm is used to generate grid data based on the attribute information corresponding to the primitives within the range, and a contour line trend map of the range is generated from the grid data through a contour line tracking technology; in an object-relational database PostgreSQL, table structures for storing data are established for experimental test data, trend maps, primitives and research projects respectively, and foreign keys are inserted into the table structures corresponding to the experimental test data, contour line trend maps, primitives and research projects, wherein the foreign keys are used to associate each table structure with the remaining table structures, and the table structures can be mutually viewed by clicking, so that the experimental test data are respectively associated with the trend map, primitives and research projects.
[0007] Furthermore, different types of primitives and their corresponding attribute information are established according to different sampling points in the research project, specifically including: establishing three types of primitives, namely points, lines and surfaces, and their corresponding attribute information, and converting all types of primitives into files in a unified format.
[0008] Furthermore, the research project is unstructured data including a project name, number and description.
[0009] Furthermore, the establishment of different types of graphics elements and attribute information corresponding to the graphics elements according to different sampling points in the research project also includes: when querying the experimental test data through the geographic information system WebGIS, the graphics elements corresponding to the experimental test data can be highlighted and located.
[0010] The present invention provides a management system for experimental test data, including:
[0011] The data acquisition module is used to establish different types of primitives and attribute information corresponding to the primitives according to different sampling points in the research project in the geographic information system WebGIS, and the attribute information is the experimental test data of the sampling points; the trend chart generation module is used to create layers according to different types of primitives and circle the range on the created layer, use the interpolation algorithm to generate grid data based on the attribute information corresponding to the primitives in the range, and use the contour line tracking technology to generate the contour line trend chart of the range from the grid data; the image data association module is used to establish table structures for storing data for experimental test data, trend charts, primitives and research projects in the object-relational database PostgreSQL, and insert foreign keys in the table structures corresponding to the experimental test data, contour line trend charts, primitives and research projects, and the foreign keys are used to associate each table structure with the remaining table structures, and can view each other by clicking between the table structures, so that the experimental test data are respectively associated with the trend chart, primitives and research projects.
[0012] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the above-mentioned method for managing experimental test data when executing the computer program.
[0013] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for managing experimental test data is implemented.
[0014] The embodiments of the present invention provide a method, system, device and medium for managing experimental test data. Compared with the prior art, the beneficial effects thereof are as follows:
[0015] Create layers according to different types of graphics and define ranges on the created layers. Use interpolation algorithms to generate grid data from the attribute information corresponding to the graphics within the range. Generate contour trend maps of the range from the grid data through contour tracing technology. In the object-relational database PostgreSQL, establish table structures for storing data for experimental test data, trend maps, graphics, and research projects, and insert foreign keys into the table structures corresponding to the experimental test data, contour trend maps, graphics, and research projects. The foreign keys are used to associate each table structure with the remaining table structures, and the table structures can be viewed by clicking on each other, so that the experimental test data are respectively associated with the trend maps, graphics, and research projects.
[0016] Among them, the contour trend chart is created using the graphics elements of the sampling points in the research project and the attribute information corresponding to the graphics elements, and the experimental test data are respectively associated with the contour trend chart, graphics elements and research projects, so that they can be viewed from each other by clicking, and the mutual viewing of experimental test data and visualization graphics is realized when the experimental test data is visualized. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 A business architecture diagram of a method for managing experimental test data provided by an embodiment of the present invention;
[0018] Figure 2 A GIS image display and data linkage effect diagram of an experimental test data management method provided by an embodiment of the present invention, wherein a is a form-based display method of the graph-related data, and b is a statistical graph display method of the graph-related data;
[0019] Figure 3 A functional module diagram of a method for managing experimental test data provided by an embodiment of the present invention;
[0020] Figure 4 A relationship diagram between research project data and experimental test data provided for an embodiment of the present invention. DETAILED DESCRIPTION
[0021] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.
[0022] See also Figure 1 to Figure 4 , an embodiment of the present invention provides a method for managing experimental test data, comprising the following steps:
[0023] Step 1: In the geographic information system WebGIS, different types of primitives and their corresponding attribute information are established according to different sampling points in the research project. The attribute information is the experimental test data of the sampling points.
[0024] Step 2: Create layers based on different types of graphics and define ranges on the created layers. Use interpolation algorithms to generate grid data based on the attribute information corresponding to the graphics within the range. Generate a contour trend map of the range using contour line tracing technology for the grid data, and store the trend map as a graphic file.
[0025] Step 3: In the object-relational database PostgreSQL, table structures for storing data are established for experimental test data, trend charts, graphics elements and research projects respectively, and foreign keys are inserted in the table structures corresponding to the experimental test data, contour trend charts, graphics elements and research projects. The foreign keys are used to associate each table structure with the remaining table structures, and the table structures can be viewed by clicking on each other, so that the experimental test data are respectively associated with the trend charts, graphics elements and research projects.
[0026] This paper mainly adopts H5, automatic interpolation algorithm and other technologies, designs 9 functional modules such as hierarchical management, advanced query, fuzzy search and GIS graphic display, and realizes the full business chain management and application of experimental test data. The main contents include the following aspects:
[0027] 1. GIS graphic display: Through automatic color matching, the three types of graphic elements of points, lines and surfaces stored in the database are displayed in a diversified manner, so that the experimental test data has the ability to be expressed visually. The specific operation is: converting points, lines and surface graphics of different formats into shapefile files that can be recognized by GIS, so that their formats are standardized and vectorized.
[0028] 2. Hierarchical management: classify and manage the experimental test data, unify the data format and coordinate projection, and standardize the data in the database. The specific classification method is: classify the experimental test data according to different research projects, such as mining areas, and generate a first-level menu of "research project" and a second-level menu of "mining area". When the user clicks on the first-level menu "research project" or the second-level menu "mining area" to interact, different levels of data and graphics are displayed respectively.
[0029] 3. Layer control: Summarize the maps at different levels to realize the hierarchical management function, and support the up and down movement, display / hide of layers, etc., to achieve a flexible display effect. Specifically, GIS layers are stacked up and down, and their order is controlled by the layers. Layers of different levels can be moved up and down by control buttons to change their stacking relationship. Among them, the check box on the interface can control the overall display / hide of the elements in the layer, and dragging the transparency control bar left and right on the interface can change the transparency of the elements in the layer.
[0030] 4. Advanced query: realize multi-element overlapping data query and multi-level fast query. The specific expression is: the experimental test data contained in the research project is visualized in the form of WebGIS and can be queried for experimental test data. The user opens the visualization interface through the browser, issues a query request according to certain conditions, obtains the map object and its internal graphic attributes through JavaScript functions, quickly highlights the target object, and further displays the data table, graphics and other detailed information contained in the target object.
[0031] 5. Fuzzy search: Through fuzzy input elements, data of different categories can be quickly located and highlighted to achieve efficient retrieval.
[0032] The specific performance is as follows: after receiving the fuzzy search request sent by the user, the fuzzy query is performed according to the search keyword, and the match is determined by comparing the similarity between the keyword and the target data. The similarity is measured by the edit distance, and the edit distance between the two strings is further calculated recursively. After that, the target data list is traversed, the edit distance between each data and the keyword is calculated, and the data with the edit distance matching is added to the result list. After obtaining the results, the location is highlighted in WebGIS, and the detailed information such as data tables and graphics contained in the fuzzy search results are further displayed.
[0033] 6. Automatic mapping: Users can draw the mapping range (boundary) on the GIS page to select the experimental test data involved in the mapping. Through the interpolation algorithm, the contour trend map with boundaries is automatically generated. The graph can be stored in the data table and associated with the relevant data. The specific operation is: use the mapping range drawn by the user as the boundary (the boundary must be closed), and obtain the experimental data concentration or content within the mining area, within the mapping range, and a secondary layer (equivalent to a single mineral species), interpolate using ordinary kriging, grid the experimental data within the boundary, and then perform contour line tracking to generate related contour maps, and store the contour maps as graphic files related to the data.
[0034] 7. Data drilling: Bind the attribute information corresponding to each experimental test data. After clicking, you can intuitively display the key data such as the area, mineral type and resource volume of the associated graphics element. You can also further drill into other tool components contained in the data.
[0035] 8. Linkage between graphs and numbers: Display experimental test data in a graphically visualized way on GIS. When clicking on a GIS element, the relevant attribute information of the element can be displayed. Associate the research project with the experimental test data. When the user clicks on an element, the form data corresponding to the element will appear at the bottom of the interface, realizing linkage management, linkage query, and linkage statistical analysis between data and graphics, and realizing the two-way application of "checking numbers with graphs" and "checking graphs with numbers".
[0036] 9. Statistical analysis: Conduct multi-dimensional statistical analysis on the experimental test data stored in the database, and make the data more intuitive through rich statistical chart presentation.
[0037] The main technologies include the following aspects:
[0038] 1. Unified technical architecture and UI design specifications
[0039] The PaaS infrastructure is built based on the mainstream Docker+Kubernetes, and the microservice framework is built with open source products such as Spring to achieve standardized encapsulation of public system services and business services, and support mainstream development languages such as Java and C / C++. The system application is mainly based on the B / S architecture, and the client supports operating systems Windows7 and above, and supports browsers such as Google, Edge and 360. Based on the mainstream front-end framework (Ant Design+VUE), Bootstrap's grid mode is selected as the basis for page layout, supporting responsive layout (suitable for multi-terminal applications).
[0040] 2. WebGIS technology
[0041] The client sends a request to the server according to the HTTP specification; the server performs the requested GIS operation and sends a response to the client through HTTP; the response sent to the client supports multiple formats, such as HTML, binary images, XML or JSON, with wider access, platform independence, low cost, simple operation, and balanced and efficient computing load.
[0042] 3. Map tile technology
[0043] Map data is usually large in size and requires sufficient bandwidth and data rendering capabilities. Rendering the map into an image on the server side and displaying the map image on the browser side can improve graphics loading efficiency and user experience.
[0044] According to certain mathematical rules, the map is layered according to multiple scales and cut into tile matrices. The vector layer in shapefile format is encoded through the VectorTile specification. According to the projection coordinates and tile organization system, the one-to-one correspondence between the geographic area range and the tile number is achieved. According to the MBTiles specification, the generated tile data is stored in a SQLite database to form a tile data set, which enables the convenient use, management and migration of map tiles.
[0045] 4. Automatic mapping technology
[0046] After the user manually delineates the mapping area on the GIS interface through interactive tools, the contour trend map of the area is automatically generated through various interpolation algorithms such as ordinary Kriging. The graphics can be saved and automatically associated with related data for easy user access.
[0047] 5. S3 storage technology
[0048] Based on the MicroService Architecture and Object Storage, the classification management and global sharing services for unstructured data, namely maps, are implemented, and service calls for application scenarios such as map upload, download, indexing, retrieval and preview are supported. At the same time, the object storage is integrated with the geological cloud platform data lake, and the data stored in the lake is shared as a data source to the big data computing platform for analysis and calculation.
[0049] There are four important aspects of the linkage management of experimental test data:
[0050] (1) Unify the standards for storing experimental test data
[0051] A set of standards for the storage of experimental test data is formed, following the principles of unified format, unified coordinate projection, and standardized management.
[0052] (2) Image and data linkage
[0053] Strongly associate experimental test data with research data to achieve linked query, summary statistics, automatic mapping, and decision support for project data.
[0054] (3) Experimental test data evolution traceability and storage
[0055] Establish an experimental test history database and graphics library to facilitate the subsequent expansion of system functions, achieve effective management of experimental data assets, and enhance their utilization value.
[0056] (4) This project uses the PostgreSQL object-relational database. Through foreign key associations, one-to-many and many-to-many relationships are established between the research project and the tables of experimental test data, trend charts, and graphics elements, so that related data can be viewed by each other.
[0057] Research projects are unstructured data. The database table defines basic information such as project name, number, description, etc. It is the main data of the project. Its relationship with structured data such as experimental test data is as follows: Figure 4 .
[0058] The objects of the present invention are:
[0059] Build a unified GIS comprehensive application platform to support experimental test data management and spatial graphic data management, realize intelligent push and automatic mapping of experimental test data, assist researchers in decision-making, and provide strong support for a new round of strategic actions for mineral exploration breakthroughs.
[0060] The embodiment of the present invention provides a management system for experimental test data, including: a data acquisition module, which is used to establish different types of primitives and attribute information corresponding to the primitives according to different sampling points in a research project in a geographic information system WebGIS, wherein the attribute information is the experimental test data of the sampling points. A trend chart generation module, which is used to create a layer according to different types of primitives and delimit a range on the created layer, generate grid data using an interpolation algorithm for the attribute information corresponding to the primitives within the range, and generate a contour trend chart of the range using the contour line tracking technology. A graph data association module, which is used to establish a table structure for storing data for experimental test data, trend charts, primitives, and research projects in an object-relational database PostgreSQL, and insert foreign keys in the table structures corresponding to the experimental test data, contour trend charts, primitives, and research projects, wherein the foreign keys are used to associate each table structure with the remaining table structures, and can be viewed mutually by clicking between the table structures, so that the experimental test data are respectively associated with the trend charts, primitives, and research projects.
[0061] An embodiment of the present invention provides a computer device, comprising: a memory and a processor; the memory stores a computer program, and the processor implements the steps of a method for managing experimental test data when executing the computer program.
[0062] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a method for managing experimental test data are implemented.
[0063] A specific embodiment is as follows:
[0064] The experimental test data of a mineral resource survey center were classified and managed, achieving efficient query and utilization, which can meet the requirements of experimental test data management and decision-making analysis of researchers.
[0065] like Figure 2 As shown, this is the automatic mapping GIS graphic display and data linkage (image-number linkage) effect of geochemical exploration data of some projects involved in a mineral resources survey center in 2024. The visualized image-number display effect improves the query efficiency and can better provide data support for researchers.
[0066] Finally, the standardization and storage of experimental test data for 197 parameters in four categories, including geochemical exploration, electron probe, laser ablation, and optical thin section, were completed in the analysis and testing laboratory of a mineral resources survey center. This reduced the workload of researchers in collating experimental data and conducting comparative analysis, saved about 80% of labor costs, and greatly improved data storage security and utilization efficiency.
[0067] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
Claims
1. A method for managing experimental test data, It is characterized in that The following steps are involved: In the geographic information system WebGIS, different types of primitives and attribute information corresponding to the primitives are established according to different sampling points in the research project. The attribute information is the experimental test data of the sampling points. Create layers according to different types of graphics and define ranges on the created layers. Use interpolation algorithms to generate grid data for the attribute information corresponding to the graphics within the range. Use contour line tracking technology to generate contour line trend maps for the range. In the object-relational database PostgreSQL, table structures for storing data are established for experimental test data, contour trend graphs, graphics elements and research projects respectively, and foreign keys are inserted in the table structures corresponding to the experimental test data, contour trend graphs, graphics elements and research projects. The foreign keys are used to associate each table structure with the remaining table structures, and the table structures can be viewed by clicking on each other, so that the experimental test data are respectively associated with the contour trend graphs, graphics elements and research projects.
2. A method for managing experimental test data as claimed in claim 1, It is characterized in that The method of establishing different types of primitives and their corresponding attribute information according to different sampling points in the research project specifically includes: establishing three types of primitives, namely points, lines and surfaces, and their corresponding attribute information, and converting all types of primitives into files in a unified format.
3. A method for managing experimental test data as claimed in claim 1, It is characterized in that The research project is unstructured data including project name, number and description.
4. A method for managing experimental test data as claimed in claim 1, It is characterized in that The method of establishing different types of graphic elements and attribute information corresponding to the graphic elements according to different sampling points in the research project also includes: when querying experimental test data through the geographic information system WebGIS, the graphic elements corresponding to the experimental test data can be highlighted and located.
5. A management system for experimental test data, It is characterized in that include: A data acquisition module is used to establish different types of primitives and attribute information corresponding to the primitives in the geographic information system WebGIS according to different sampling points in the research project, wherein the attribute information is the experimental test data of the sampling points; The trend chart generation module is used to create layers according to different types of graphics elements and circle the range on the created layers, use the interpolation algorithm to generate grid data for the attribute information corresponding to the graphics elements in the range, and use the contour line tracking technology to generate the contour line trend chart of the range from the grid data; The graph-data association module is used to establish table structures for storing data in the object-relational database PostgreSQL for experimental test data, trend graphs, graph elements and research projects, and insert foreign keys in the table structures corresponding to the experimental test data, contour trend graphs, graph elements and research projects. The foreign keys are used to associate each table structure with the remaining table structures, and the table structures can be viewed by clicking on each other, so that the experimental test data are respectively associated with the trend graphs, graph elements and research projects.
6. A computer device, include: Memory and processor; The memory stores a computer program, wherein the processor implements a method for managing experimental test data according to any one of claims 1 to 4 when executing the computer program.
7. A computer-readable storage medium storing a computer program, It is characterized in that When the computer program is executed by a processor, a method for managing experimental test data according to any one of claims 1 to 4 is implemented.
