Geological data acquisition, storage and analysis method

Abstract

The invention relates to the technical field of geological exploration, in particular to a geological data acquisition, storage and analysis method. The geological data acquisition, storage and analysis method comprises the following steps: step S100, determining an acquisition purpose and content; step S200, determining appropriate tools and equipment based on the purpose and the content, and formulating an acquisition plan and a flow; and S300, on-site investigation and investigation are carried out based on the process, the collection range and content are determined, problems possibly encountered in the site are identified and solved, and it is ensured that the collected data meet actual requirements. The method has the advantages of being high in data precision and high in mass data obtaining and processing efficiency, and solves the problems that existing geological data is affected by human and environmental factors, the geological data precision is not high easily, the geological mass data is difficult to obtain and process, and the geological data processing efficiency is high. And the data acquisition and processing efficiency is low.

Description

Technical Field

[0001] The present invention relates to the field of geological prospecting technology, and in particular to a method for collecting, storing and analyzing geological data. Background Art

[0002] Geological data collection is the process of obtaining geological information through various technical means. This includes field measurements, drilling sampling, remote sensing detection and other methods. The collected data includes information on geological structure, mineral resource distribution, environmental impact assessment and other aspects. These data are of great significance to geological research, engineering construction and disaster prevention. The storage system is used to save the collected geological data to ensure the integrity and accessibility of the data. The storage system can be a database system that classifies, organizes and preserves massive geological data for a long time, supporting efficient data retrieval and management. Data analysis is the process of processing and interpreting the collected geological data. This includes statistical analysis of data, model building and prediction, etc., to reveal the laws and trends of geological phenomena. The analysis results can be used in resource assessment, environmental impact assessment and disaster risk prediction.

[0003] Geological analysis refers to the systematic observation, experimentation and research of rocks, minerals, geological structures, etc. on the surface or underground of the earth. As an important part of geology, geological analysis reveals information on the evolution of the earth, the distribution of geological resources, and the risk of geological disasters through the analysis of geological processes, geological events and geological phenomena, providing an important scientific basis for resource exploration and environmental protection. As the objective basis of geological analysis, geological data needs to be collected and analyzed to objectively understand geological structures, geological processes, geological events, etc., and avoid subjective assumptions and misjudgments.

[0004] At present, the existing geological data are affected by human and environmental factors, which may lead to low accuracy of geological data. In addition, it is difficult to obtain and process massive geological data, resulting in low efficiency of data acquisition and processing. Therefore, we propose a method for geological data collection, storage and analysis. Summary of the invention

[0005] The purpose of the present invention is to provide a method for collecting, storing and analyzing geological data, which has the advantages of high data accuracy and high efficiency in acquiring and processing massive data, and solves the problem that existing geological data are affected by human and environmental factors, which easily leads to low accuracy of geological data, and the acquisition and processing of massive geological data are difficult, resulting in low efficiency in data acquisition and processing.

[0006] To achieve the above object, the present invention provides the following technical solution: a method for collecting, storing and analyzing geological data, the method comprising:

[0007] Step S100: clarify the purpose and content of collection;

[0008] Step S200: Based on the purpose and content, determine appropriate tools and equipment, and formulate a collection plan and process;

[0009] Step S300: Based on the process, conduct on-site investigation and survey, determine the scope and content of collection, and identify and solve problems that may be encountered on-site to ensure that the collected data meets actual needs;

[0010] Step S400: Based on the exploration and investigation, using MapGIS field geological data acquisition system to assist in geological data acquisition, and preprocessing the acquired geological data;

[0011] Step S500: Based on the preprocessed data, the geological data is stored and the data is analyzed;

[0012] Step S600: Based on the storage and analysis data, data encryption is performed and regular backup is performed, and employees are trained on data security awareness and network security is strengthened.

[0013] Preferably, the step S100 includes determining the accuracy and range of the data type to be collected, whether it is a point, line or surface data, to ensure that the collected data meets actual application requirements.

[0014] Preferably, the tools and equipment in step S200 can be selected from drones, satellite remote sensing, Beidou locators, cameras, laser scanners, drills, and samplers. Different tools and equipment have different accuracy and scope of application, and need to be selected according to actual needs. When formulating collection plans and processes, it is necessary to include information on collection time, location, personnel, tasks, and tools to ensure that the collection process is orderly and efficient.

[0015] Preferably, the preparation and precautions for on-site inspection and investigation in step S300 include the following aspects:

[0016] S301: Before conducting a field survey, it is necessary to understand the basic conditions of the survey area, including geographical location, climate conditions, topography, and population distribution. At the same time, prepare the necessary survey tools and equipment according to the specific task;

[0017] S302: Develop a detailed survey plan, including the survey time, location, tasks, and personnel arrangements. At the same time, develop appropriate safety measures to ensure the safety of survey personnel;

[0018] S303: When conducting on-site investigations, it is necessary to comply with local laws, regulations and cultural practices, and respect local customs and national culture;

[0019] S304: Before conducting on-site investigation, it is necessary to contact relevant departments and personnel to understand the local situation and matters that need attention in order to better carry out the investigation;

[0020] S305: All persons should be equipped with wide-brimmed sun hats, goggles, compasses, sunscreen and disinfectants, understand the water source conditions in the work area, be familiar with protective measures, have sufficient drinking water and drink it properly. When the temperature is above 38°C and there are no cooling facilities, work should be stopped. The best working time is early morning or evening.

[0021] Preferably, the method of preprocessing the collected geological data in step S400 includes the following aspects:

[0022] S401: Data cleaning and verification. After data collection is completed, the data needs to be cleaned and the authenticity and accuracy of the data are ensured through data verification and anomaly detection algorithms and technologies.

[0023] S402: Missing value processing, which can be done by interpolation, deletion or filling, so as to obtain complete data in subsequent analysis and avoid inaccurate data caused by incomplete data collection, storage errors or unstable data sources;

[0024] S403: Data consistency check: ensure the comparability and availability of data through standardization and integration methods to avoid the existence of inconsistent data;

[0025] S404: Outlier detection, using statistical analysis and visualization methods to detect outliers and outliers in the data, and using box plots, scatter plots, and histogram tools to find anomalies in the data and process or remove them;

[0026] S405: Logical error check, by checking the rationality of the date and whether the relationship between the values ​​is logical, the data is logically verified to check for logical errors and contradictions in the data;

[0027] S406: Data deduplication, by using the functions in the tool Kettle or pandas, identify and delete duplicate data to ensure the uniqueness of the data set.

[0028] Preferably, the data storage and analysis in step S500 should pay attention to the security and accessibility of the data during storage, and statistical methods and machine learning algorithm tools can be used for in-depth analysis during analysis.

[0029] Preferably, the key to improving the security of geological data storage in step S600 includes the following aspects:

[0030] S601: By using encryption technology, data can be converted into an unreadable format that can only be accessed through a specific decryption key, which can effectively prevent unauthorized persons from obtaining data;

[0031] S602: Backing up data to offline media or cloud storage services regularly can prevent data loss due to hard disk failure or human error;

[0032] S603: By setting up user authentication, access rights, and access auditing functions, you can limit access to sensitive data. Using strong passwords and multi-factor authentication can further improve account security.

[0033] S604: Using firewalls, intrusion detection and protection systems, and regularly conducting network vulnerability scans and penetration tests can effectively prevent external attacks;

[0034] S605: Security incidents caused by internal negligence can be greatly reduced by educating employees on basic concepts of data security, common threats, and how to handle sensitive information.

[0035] Preferably, the data cleaning and verification in S401 can be completed by writing scripts or using data processing tools.

[0036] Preferably, in step S400, multiple data sources are merged into one data source to increase the data volume, while paying attention to entity recognition, attribute redundancy and data value conflicts, and converting the data into an appropriate form to reduce the complexity of the data for more efficient processing and analysis.

[0037] Preferably, in step S400, the data volume is reduced and data irrelevant to the data analysis or data mining subject is deleted while maintaining the original appearance of the data.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. The present invention can reduce unnecessary costs by clarifying the purpose and content of collection, making the collection of corresponding geological data more targeted, and can reduce the equipment to be carried, which is conducive to the rapid familiarity of personnel with the equipment, ensuring the improvement of the accuracy of geological data collection, solving problems that may affect the accuracy of data collection through field surveys, ensuring the accuracy and accuracy of geological data collection, and adopting MapGIS field geological data collection system, which is a system specially designed for geological surveys, aiming to improve the efficiency and accuracy of field data collection. The system integrates multiple technologies, including GIS technology, modern database technology, visualization technology and computer network technology, and realizes Real-time collection, storage and sharing of field data. Through this system, survey data can be stored in real time and connected with professional mapping tools, assisting in the realization of the four-in-one information transformation of the whole process from "investigation-database construction-map compilation-modeling", making the collection of geological data simpler and more efficient, and being able to strictly clean and verify data before data collection to ensure data accuracy and consistency, thereby improving the reliability and effectiveness of data analysis, and greatly reducing the workload, so that the efficiency of data processing can be improved. At the same time, through data encryption and regular backup, as well as training employees on data security awareness and strengthening network security and other multiple methods, the security of geological data storage is improved.

[0040] 2. The present invention combines multiple data sources into one data source through data integration to increase the data volume. At the same time, attention should be paid to issues such as entity recognition, attribute redundancy and data value conflicts. The data is converted into an appropriate form through data transformation to reduce the complexity of the data for more efficient processing and analysis. At the same time, data reduction can streamline the data volume and delete data irrelevant to the data analysis or data mining subject while maintaining the original appearance of the data, thereby further reducing the data processing volume and significantly improving the preprocessing speed and efficiency of geological data. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the present invention;

[0042] Figure 2 Flow chart of data preprocessing of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Embodiment 1

[0045] See also Figure 1-Figure 2 As shown, the present invention provides a technical solution: a method for collecting, storing and analyzing geological data, the method comprising:

[0046] Step S100: clarify the purpose and content of collection;

[0047] Step S200: Determine appropriate tools and equipment based on purpose and content, and develop a collection plan and process;

[0048] Step S300: Based on the process, conduct on-site investigation and survey, determine the scope and content of collection, and identify and solve problems that may be encountered on-site to ensure that the collected data meets actual needs;

[0049] Step S400: Based on the exploration and investigation, the MapGIS field geological data collection system is used to assist in the collection of geological data, and the collected geological data is preprocessed;

[0050] Step S500: storing the geological data based on the preprocessed data and analyzing the data;

[0051] Step S600: Based on the storage and analysis data, data encryption is performed and regular backup is performed, while employees are trained on data security awareness and network security is strengthened.

[0052] Furthermore, step S100 includes determining the accuracy and range of the data type to be collected, whether it is a point, line or surface data, to ensure that the collected data meets the actual application requirements.

[0053] Furthermore, the tools and equipment in step S200 may be selected from drones, satellite remote sensing, Beidou locators, cameras, laser scanners, drills, and samplers. Different tools and equipment have different accuracy and scope of application, and need to be selected according to actual needs. When formulating collection plans and processes, it is necessary to include information on collection time, location, personnel, tasks, and tools to ensure that the collection process is orderly and efficient.

[0054] Furthermore, the preparation and precautions for the on-site inspection and investigation in step S300 include the following aspects:

[0055] S301: Before conducting a field survey, it is necessary to understand the basic conditions of the survey area, including geographical location, climate conditions, topography, and population distribution. At the same time, prepare the necessary survey tools and equipment according to the specific task;

[0056] S302: Develop a detailed survey plan, including the survey time, location, tasks, and personnel arrangements. At the same time, develop appropriate safety measures to ensure the safety of survey personnel;

[0057] S303: When conducting on-site investigations, it is necessary to comply with local laws, regulations and cultural practices, and respect local customs and national culture;

[0058] S304: Before conducting on-site investigation, it is necessary to contact relevant departments and personnel to understand the local situation and matters that need attention in order to better carry out the investigation;

[0059] S305: All persons should be equipped with wide-brimmed sun hats, goggles, compasses, sunscreen and disinfectants, understand the water source conditions in the work area, be familiar with protective measures, have sufficient drinking water and drink it properly. When the temperature is above 38°C and there are no cooling facilities, work should be stopped. The best working time is early morning or evening.

[0060] Furthermore, the method of preprocessing the collected geological data in step S400 includes the following aspects:

[0061] S401: Data cleaning and verification. After data collection is completed, the data needs to be cleaned and the authenticity and accuracy of the data are ensured through data verification and anomaly detection algorithms and technologies.

[0062] S402: Missing value processing, which can be done by interpolation, deletion or filling, so as to obtain complete data in subsequent analysis and avoid inaccurate data caused by incomplete data collection, storage errors or unstable data sources;

[0063] S403: Data consistency check: ensure the comparability and availability of data through standardization and integration methods to avoid the existence of inconsistent data;

[0064] S404: Outlier detection, using statistical analysis and visualization methods to detect outliers and outliers in the data, and using box plots, scatter plots, and histogram tools to find anomalies in the data and process or remove them;

[0065] S405: Logical error check, by checking the rationality of the date and whether the relationship between the values ​​is logical, the data is logically verified to check for logical errors and contradictions in the data;

[0066] S406: Data deduplication, by using the functions in the tool Kettle or pandas, identify and delete duplicate data to ensure the uniqueness of the data set.

[0067] Furthermore, in the data storage and analysis in step S500, attention should be paid to the security and accessibility of the data during storage, and statistical methods and machine learning algorithm tools can be used for in-depth analysis during analysis.

[0068] Furthermore, the key to improving the security of geological data storage in step S600 includes the following aspects:

[0069] S601: By using encryption technology, data can be converted into an unreadable format that can only be accessed through a specific decryption key, which can effectively prevent unauthorized persons from obtaining data;

[0070] S602: Backing up data to offline media or cloud storage services regularly can prevent data loss due to hard disk failure or human error;

[0071] S603: By setting up user authentication, access rights, and access auditing functions, you can limit access to sensitive data. Using strong passwords and multi-factor authentication can further improve account security.

[0072] S604: Using firewalls, intrusion detection and protection systems, and regularly conducting network vulnerability scans and penetration tests can effectively prevent external attacks;

[0073] S605: Security incidents caused by internal negligence can be greatly reduced by educating employees on basic concepts of data security, common threats, and how to handle sensitive information.

[0074] This technical solution: by clarifying the purpose and content of collection, it can reduce unnecessary costs and expenditures, make the collection of corresponding geological data more targeted, and reduce the equipment to be carried, which is conducive to the rapid familiarity of personnel with the equipment, ensuring the accuracy of geological data collection. Through field surveys, problems that may affect the accuracy of data collection are solved to ensure the accuracy and accuracy of geological data collection. The MapGIS field geological data collection system is used, which is a system specially designed for geological surveys to improve the efficiency and accuracy of field data collection. The system integrates a variety of technologies, including GIS technology, modern database technology, visualization technology and computer network technology to achieve The real-time collection, storage and sharing of field data are realized. Through this system, survey data can be stored in real time and connected with professional mapping tools, which assists in the realization of the four-in-one information transformation of the whole process from "survey-database construction-map compilation-modeling", making the collection of geological data simpler and more efficient, and can strictly clean and verify the data before data collection to ensure the accuracy and consistency of the data, thereby improving the reliability and effectiveness of data analysis, and greatly reducing the workload, so that the efficiency of data processing can be improved. At the same time, through data encryption and regular backup, as well as training employees on data security awareness and strengthening network security and other multiple methods, the security of geological data storage is improved.

[0075] Embodiment 2

[0076] Based on the first embodiment, the present invention is as follows Figure 1 As shown, the data cleaning and verification in S401 are disclosed and can be completed by writing scripts or using data processing tools. In step S400, multiple data sources are merged into one data source to increase the data volume. At the same time, attention should be paid to entity recognition, attribute redundancy and data value conflicts, and the data is converted into an appropriate form to reduce the complexity of the data for more efficient processing and analysis. In step S400, the data volume is streamlined while maintaining the original appearance of the data, and data that is not related to the data analysis or data mining subject is deleted.

[0077] This technical solution: merge multiple data sources into one data source through data integration to increase the data volume, and pay attention to issues such as entity recognition, attribute redundancy and data value conflicts. Convert the data into an appropriate form through data transformation to reduce the complexity of the data for more efficient processing and analysis. At the same time, data reduction can streamline the data volume and delete data that is not related to the data analysis or data mining subject while maintaining the original appearance of the data, thereby further reducing the data processing volume and significantly improving the preprocessing speed and efficiency of geological data.

[0078] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A method for collecting, storing and analyzing geological data, characterized in that: The method comprises: Step S100: clarify the purpose and content of collection; Step S200: Based on the purpose and content, determine appropriate tools and equipment, and formulate a collection plan and process; Step S300: Based on the process, conduct on-site investigation and survey, determine the scope and content of collection, and identify and solve problems that may be encountered on-site to ensure that the collected data meets actual needs; Step S400: Based on the exploration and investigation, using MapGIS field geological data acquisition system to assist in geological data acquisition, and preprocessing the acquired geological data; Step S500: Based on the preprocessed data, the geological data is stored and the data is analyzed; Step S600: Based on the storage and analysis data, data encryption is performed and regular backup is performed, and employees are trained on data security awareness and network security is strengthened.

2. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The step S100 includes determining the accuracy and range of the data type to be collected, whether it is a point, line or surface data, to ensure that the collected data meets the actual application requirements.

3. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The tools and equipment in step S200 may be selected from drones, satellite remote sensing, Beidou locators, cameras, laser scanners, drills, and samplers. Different tools and equipment have different accuracy and scope of application, and need to be selected according to actual needs. When formulating collection plans and processes, it is necessary to include information on collection time, location, personnel, tasks, and tools to ensure that the collection process is orderly and efficient.

4. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The preparation and precautions for the on-site inspection and investigation in step S300 include the following aspects: S301: Before conducting a field survey, it is necessary to understand the basic conditions of the survey area, including geographical location, climate conditions, topography, and population distribution. At the same time, prepare the necessary survey tools and equipment according to the specific task; S302: Develop a detailed survey plan, including the survey time, location, tasks, and personnel arrangements. At the same time, develop appropriate safety measures to ensure the safety of survey personnel; S303: When conducting on-site investigations, it is necessary to comply with local laws, regulations and cultural practices, and respect local customs and national culture; S304: Before conducting on-site investigation, it is necessary to contact relevant departments and personnel to understand the local situation and matters that need attention in order to better carry out the investigation; S305: All persons should be equipped with wide-brimmed sun hats, goggles, compasses, sunscreen and disinfectants, understand the water source conditions in the work area, be familiar with protective measures, have sufficient drinking water and drink it properly. When the temperature is above 38°C and there are no cooling facilities, work should be stopped. The best working time is early morning or evening.

5. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The method of preprocessing the collected geological data in step S400 includes the following aspects: S401: Data cleaning and verification. After data collection is completed, the data needs to be cleaned and the authenticity and accuracy of the data are ensured through data verification and anomaly detection algorithms and technologies. S402: Missing value processing, which can be done by interpolation, deletion or filling, so as to obtain complete data in subsequent analysis and avoid inaccurate data caused by incomplete data collection, storage errors or unstable data sources; S403: Data consistency check: ensure the comparability and availability of data through standardization and integration methods to avoid the existence of inconsistent data; S404: Outlier detection, using statistical analysis and visualization methods to detect outliers and outliers in the data, and using box plots, scatter plots, and histogram tools to find anomalies in the data and process or remove them; S405: Logical error check, by checking the rationality of the date and whether the relationship between the values ​​is logical, the data is logically verified to check for logical errors and contradictions in the data; S406: Data deduplication, by using the functions in the tool Kettle or pandas, identify and delete duplicate data to ensure the uniqueness of the data set.

6. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The data storage and analysis in step S500 should pay attention to the security and accessibility of the data during storage, and statistical methods and machine learning algorithm tools can be used for in-depth analysis during analysis.

7. A method for collecting, storing and analyzing geological data according to claim 1, characterized in that: The key to improving the security of geological data storage in step S600 includes the following aspects: S601: By using encryption technology, data can be converted into an unreadable format that can only be accessed through a specific decryption key, which can effectively prevent unauthorized persons from obtaining data; S602: Backing up data to offline media or cloud storage services regularly can prevent data loss due to hard disk failure or human error; S603: By setting up user authentication, access rights, and access auditing functions, you can limit access to sensitive data. Using strong passwords and multi-factor authentication can further improve account security. S604: Using firewalls, intrusion detection and protection systems, and regularly conducting network vulnerability scans and penetration tests can effectively prevent external attacks; S605: Security incidents caused by internal negligence can be greatly reduced by educating employees on basic concepts of data security, common threats, and how to handle sensitive information.

8. A method for collecting, storing and analyzing geological data according to claim 5, characterized in that: The data cleaning and verification in S401 can be completed by writing scripts or using data processing tools.

9. A method for collecting, storing and analyzing geological data according to claim 5, characterized in that: In step S400, multiple data sources are merged into one data source to increase the data volume, and at the same time, attention is paid to entity recognition, attribute redundancy and data value conflicts, and the data is converted into an appropriate form to reduce the complexity of the data for more efficient processing and analysis.

10. A method for collecting, storing and analyzing geological data according to claim 9, characterized in that: In step S400, the data volume is reduced while maintaining the original appearance of the data, and data irrelevant to the data analysis or data mining subject is deleted.

Images