Method and device for improving efficiency and intelligent level of mine industry

Abstract

The invention relates to the technical field of computer science, and particularly provides a method and device for improving the efficiency and the intelligent level of the mine industry, and the method comprises the following steps: S1, carrying out the calculation of resources, a storage system and a grid architecture at an infrastructure layer; s2, performing data acquisition, data cleaning, data labeling, data balancing and data enhancement on the data resource layer; s3, performing preliminary training, fine adjustment, cue word optimization, vector embedding, knowledge specification and retrieval enhancement generation of a large-scale corpus in the algorithm model layer; s4, setting an API interface and applying a user interface in the application service layer; and S5, performing data encryption, access control, audit tracking and test verification in the security, credibility and test layer. Compared with the prior art, an open and shared industry knowledge base can be constructed, and integrated development of industry-university-research is promoted.

Description

Technical Field

[0001] The present invention relates to the field of computer science and technology, and specifically provides a method and a device for improving the efficiency and intelligence level of the mining industry. Background Art

[0002] With the increasing awareness of environmental protection around the world and the pursuit of carbon emission reduction goals by various countries, the "dual carbon" strategy has become one of the important issues in many countries and regions.

[0003] As an important part of traditional energy, the coal industry is facing unprecedented transformation pressure. In order to better adapt to this trend of change, coal enterprises are seeking to achieve their own transformation and upgrading by introducing emerging information technology. Among them, the big model technology based on big data analysis and artificial intelligence has attracted much attention due to its powerful information processing capabilities and broad application prospects.

[0004] How to realize intelligent management and optimization of mining industry systems by integrating big models and big data processing technologies is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] The present invention aims at solving the above-mentioned deficiencies of the prior art and provides a method for improving the efficiency and intelligence level of the mining industry with strong practicality.

[0006] A further technical task of the present invention is to provide a device that is rationally designed, safe and applicable to improve the efficiency and intelligence level of the mining industry.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] A method for improving the efficiency and intelligence level of the mining industry has the following steps:

[0009] S1, computing resources, storage systems and grid architecture at the infrastructure layer;

[0010] S2, perform data collection, data cleaning, data labeling, data balancing and data enhancement at the data resource layer;

[0011] S3, the algorithm model layer performs preliminary training, fine-tuning, prompt word optimization, vector embedding, knowledge specification, and retrieval enhancement generation on a large corpus;

[0012] S4: Setting of API interface and application of user interface in the application service layer;

[0013] S5: Data encryption, access control, audit tracking and test verification are performed in the security, trust and testing layer.

[0014] Furthermore, in step S1, when calculating resources, hardware configuration and cloud services are performed, GPU and TPU accelerators are configured in the hardware configuration, each node is configured with at least 1TB of memory and high SSD storage, and Kubernetes is used in the cloud service for container orchestration to achieve efficient management and scheduling of resources;

[0015] In the storage system, distributed storage, object storage and cache layer are used. In the distributed storage, Hadoop HDFS or Ceph distributed storage system is deployed, and Erasure Coding technology is used to improve storage efficiency;

[0016] The object storage uses an object storage service to store unstructured data and automatically archives cold data through a lifecycle management strategy; the cache layer introduces a Redis memory database as a cache layer;

[0017] The network architecture uses InfiniBand network to connect various computing nodes, and uses SD-WAN technology to optimize wide area network connections.

[0018] Furthermore, in step S2, during data collection, the temperature, humidity, gas concentration and equipment operation status data in the mine are collected through IoT sensors, and the video surveillance data in the mine are collected using cameras and infrared cameras, including personnel activities, equipment status and environmental changes; the geological structure and ore deposit distribution data of the mine are collected through geological exploration equipment and stored in GIS format; the log data of various operations in the mine are recorded, including equipment operation records and personnel entry and exit records;

[0019] When cleaning data, use hash functions and Bloom filters to remove duplicate data, use rule engines and statistical methods to identify and correct errors in the data, use IQR methods to detect and correct outliers, use interpolation, regression models or deep learning methods to fill in missing values, and use anomaly detection algorithms to identify and correct outliers.

[0020] Furthermore, when labeling data, a professional labeling team is established to manually label images, videos, and text data, using existing pre-trained models for preliminary labeling, followed by manual review and correction;

[0021] When balancing data, oversampling is to use the Synthetic Minority Over-sampling Technique to generate minority class samples to balance the data set; undersampling is to randomly delete majority class samples to reduce the imbalance of the data set; mixed sampling is a method that combines oversampling and undersampling;

[0022] Data enhancement includes image enhancement, text enhancement and audio enhancement. The image enhancement increases the diversity of image data by rotation, scaling, cropping and flipping methods; the text enhancement expands text data by synonym replacement, sentence rewriting and antonym insertion methods; the audio enhancement enhances the robustness of audio data by adding noise, changing pitch and changing speed methods.

[0023] Furthermore, in step S3, the large model is preliminarily trained using a large-scale public corpus, using BERT and RoBERTa pre-trained models; pre-training is performed on text, image, and audio multimodal data, using CLIP and DALL-E multimodal pre-trained models to improve the generalization ability of the model;

[0024] Based on the pre-training, the model is fine-tuned using a specific data set for the coal industry. The pre-trained model is fine-tuned according to the specific tasks of the coal industry. The existing pre-trained model is used as the initial weight and fine-tuned with a small amount of labeled data.

[0025] When the prompt words are optimized, the model is guided to generate more accurate and useful results. For equipment fault diagnosis tasks, specific prompt templates are designed.

[0026] Furthermore, vector embedding includes text embedding, image embedding and multimodal embedding. The text embedding is to generate a vector representation of text using BERT and Word2Vec pre-trained models. The image embedding is to generate a vector representation of an image using ResNet and VGG convolutional neural networks for image classification and retrieval. The multimodal embedding is to combine the embedding vectors of text and image to generate a joint representation of multimodal data.

[0027] The knowledge specification is to introduce the knowledge and rules of industry experts, inject this knowledge into the model through the constraint optimization method, and correct it through the post-processing step after the model generates the results;

[0028] When the retrieval enhancement is generated, relevant information is retrieved from historical data, the auxiliary model generates a reply, full-text retrieval is performed using Elasticsearch, and sorting is performed in combination with the BM25 algorithm. The retrieval results and the output of the generated model are combined to generate the final answer through weighted fusion.

[0029] Furthermore, in step S4, a standardized API interface is provided to exchange data in JSON format, and semantic version management is adopted to make the API compatible and stable;

[0030] In the user interface, develop web applications, provide graphical interfaces, develop mobile applications, support iOS and Android platforms, integrate visualization tools, and generate custom dashboards and reports.

[0031] Furthermore, in step S5, when encrypting data, the TLS / SSL protocol is used to encrypt data transmission, the AES-256 encryption algorithm is used to encrypt stored data, and the KMS is used to generate, store and manage keys;

[0032] When performing access control, OAuth 2.0 and OpenID Connect protocols are used for user authentication, and a role-based access control mechanism is adopted to define different roles and permissions for fine-grained access control.

[0033] Furthermore, when auditing and tracking, all data access and operation logs are recorded, and ELK Stack is used for log management and analysis, and machine learning algorithms are used to detect abnormal behaviors in logs;

[0034] During test verification, Junit and pytest are used to test each module, and the entire system is integrated tested to ensure the collaboration between modules. Load testing and stress testing are performed to evaluate the performance bottlenecks of the system, and penetration testing and vulnerability scanning are performed to discover and fix potential security vulnerabilities.

[0035] A device for improving the efficiency and intelligence level of the mining industry, comprising: at least one memory and at least one processor;

[0036] The at least one memory is used to store a machine-readable program;

[0037] The at least one processor is used to call the machine-readable program to execute a method for improving the efficiency and intelligence level of the mining industry.

[0038] Compared with the prior art, the method and device for improving the efficiency and intelligence level of the mining industry of the present invention have the following outstanding beneficial effects:

[0039] This invention can not only significantly improve the safety, efficiency and sustainability of mining operations, but also promote the transformation of the coal industry into a technology-intensive one, build an open and shared industry knowledge base, and promote the integrated development of production, education, research and application. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Attached Figure 1It is a flow chart of a method to improve the efficiency and intelligence level of the mining industry. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the solution of the present invention, the present invention is further described in detail below in conjunction with specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] A best embodiment is given below:

[0044] like Figure 1 As shown, a method for improving the efficiency and intelligence level of the mining industry in this embodiment has the following steps:

[0045] S1, computing resources, storage systems and grid architecture at the infrastructure layer;

[0046] include:

[0047] (1) Computing resources;

[0048] Hardware configuration: Build a high-performance computing cluster equipped with accelerators such as GPU and TPU to support large-scale data processing and model training. Each node is equipped with at least 1TB of memory and high SSD storage.

[0049] Cloud services: Cloud service providers’ elastic computing resources, such as ECS (Elastic Compute Service) and GPU instances, ensure on-demand expansion. Use Kubernetes for container orchestration to achieve efficient resource management and scheduling.

[0050] (2) Storage system;

[0051] Distributed storage: Deploy distributed storage systems such as Hadoop HDFS or Ceph to ensure high availability and fault tolerance of data. Use Erasure Coding technology to improve storage efficiency;

[0052] Object storage: Use object storage services (such as OSS) to store unstructured data, and automatically archive cold data through lifecycle management strategies to reduce costs.

[0053] Cache layer: Introduce the Redis memory database as a cache layer to speed up the reading of frequently accessed data.

[0054] (3) Network architecture;

[0055] Network architecture: Ensure high-speed and stable data transmission. Use InfiniBand network to connect various computing nodes to ensure high-speed and stable data transmission. Use SD-WAN technology to optimize wide area network connections and improve the efficiency of remote data transmission.

[0056] S2, perform data collection, data cleaning, data labeling, data balancing and data enhancement at the data resource layer;

[0057] include:

[0058] (1) Data collection;

[0059] Sensor data: IoT sensors collect real-time environmental data such as temperature, humidity, gas concentration, and equipment operating status data in the mine;

[0060] Monitoring data: Use high-definition cameras and infrared cameras to collect video monitoring data inside the mine, including personnel activities, equipment status, and environmental changes;

[0061] Geological data: Collect data such as mine geological structure and ore deposit distribution through geological exploration equipment and store them in GIS (geographic information system) format;

[0062] Operation log: records the log data of various operations in the mine, including equipment operation records, personnel entry and exit records, etc.

[0063] (2) Data cleaning;

[0064] Deduplication: Use hash functions and Bloom filters to remove duplicate data.

[0065] Error correction: Identify and correct errors in data through rule engines and statistical methods, and use IQR (interquartile range) methods to detect and correct outliers;

[0066] Fill missing values: Use interpolation, regression models, or deep learning methods to fill missing values ​​and ensure data integrity. Use anomaly detection algorithms to identify and correct outliers.

[0067] (3) Data annotation;

[0068] Manual labeling: Establish a professional labeling team to manually label images, videos, and text data. For example, label the status of equipment in the mine (normal / faulty);

[0069] Semi-automatic labeling: Use existing pre-trained models for preliminary labeling, followed by manual review and correction to improve labeling efficiency.

[0070] (4) Data balance;

[0071] Oversampling: Use Synthetic Minority Over-sampling Technique to generate minority class samples and balance the dataset.

[0072] Undersampling: Randomly delete majority class samples to reduce the imbalance of the dataset.

[0073] Hybrid sampling: Combine oversampling and undersampling methods to achieve better data balance.

[0074] (5) Data enhancement;

[0075] Image enhancement: Increase the diversity of image data by rotating, scaling, cropping, flipping, etc.

[0076] Text enhancement: expand text data using methods such as synonym replacement, sentence rewriting, and antonym insertion;

[0077] Audio enhancement: Enhance the robustness of audio data by adding noise, changing pitch, changing speed, etc.

[0078] S3, the algorithm model layer performs preliminary training, fine-tuning, prompt word optimization, vector embedding, knowledge specification, and retrieval enhancement generation on a large corpus;

[0079] include:

[0080] (1) Pre-training;

[0081] Large-scale corpus: Use large-scale public corpora to conduct preliminary training on large models, and use pre-trained models such as BERT and RoBERTa to enable them to have basic language understanding and generation capabilities.

[0082] Multimodal pre-training: Combine multimodal data such as text, images, and audio for pre-training, and use multimodal pre-training models such as CLIP and DALL-E to improve the generalization ability of the model.

[0083] (2) Fine-tuning;

[0084] Specific task fine-tuning: Based on pre-training, fine-tune the model using specific datasets for the coal industry. Fine-tune the pre-trained model according to specific tasks in the coal industry (such as equipment fault diagnosis, safety monitoring, etc.). Use datasets unique to the coal industry, such as equipment operation logs, fault reports, etc.

[0085] Transfer learning: Use an existing pre-trained model as the initial weights and fine-tune it using a small amount of labeled data to reduce training time and data requirements.

[0086] (3) Prompt word optimization;

[0087] Prompt Engineering: Design efficient prompt word strategies to guide the model to generate more accurate and useful results. For equipment fault diagnosis tasks, specific prompt templates can be designed.

[0088] Zero-shot learning: Through carefully designed prompt words, the model can generate reasonable answers even if it has never seen similar tasks.

[0089] (4) Vector embedding;

[0090] Text embedding: Use pre-trained models such as BERT and Word2Vec to generate vector representations of text for easier model processing

[0091] Image embedding: Use convolutional neural networks such as ResNet and VGG to generate vector representations of images for image classification and retrieval.

[0092] Multimodal embedding: Combine the embedding vectors of text and image to generate a joint representation of multimodal data and improve the multimodal processing capabilities of the model.

[0093] (5) Knowledge norms;

[0094] Expert knowledge injection: Introduce the knowledge and rules of industry experts and inject this knowledge into the model through constrained optimization methods to ensure that the model output complies with industry standards and regulations.

[0095] Post-processing correction: After the model generates results, corrections are made through post-processing steps, such as using a rules engine to verify and adjust the model output.

[0096] (6) Retrieval Enhancement Generation

[0097] Information retrieval: Combine traditional information retrieval technology to retrieve relevant information from a large amount of historical data, and assist the model in generating more accurate answers. Use Elasticsearch for full-text retrieval and combine it with the BM25 algorithm for sorting.

[0098] Hybrid model: Combines the retrieval results and the output of the generative model to generate the final answer through weighted fusion, thus improving the accuracy and reliability of the model.

[0099] S4: Setting of API interface and application of user interface in the application service layer;

[0100] include:

[0101] (1) API interface;

[0102] Standardized API: Provides standardized API interfaces to facilitate access and integration of third-party systems and services. Provides RESTful API interfaces and supports data exchange in JSON format.

[0103] Version management: Use semantic version management to ensure API compatibility and stability.

[0104] (2) User interface;

[0105] Web application: Develop web applications and provide a graphical interface to facilitate users to view and manage data. Support a variety of chart displays, such as line charts, bar charts, heat maps, etc.

[0106] Mobile applications: Develop mobile applications that support iOS and Android platforms. Provide real-time data monitoring, alarm push and other functions.

[0107] Visualization tools: Integrated visualization tools support custom dashboards and report generation to help users understand data intuitively.

[0108] S5: Data encryption, access control, audit tracking and test verification are performed in the security, trust and testing layer.

[0109] include:

[0110] (1) Data encryption;

[0111] Transmission encryption: Use TLS / SSL protocol to encrypt data transmission to ensure the security of data during transmission.

[0112] Storage encryption: Use encryption algorithms such as AES-256 to encrypt stored data to prevent data leakage.

[0113] Key management: Use KMS (Key Management Service) to generate, store and manage keys to ensure key security.

[0114] (2) Access control;

[0115] Identity authentication: Use OAuth 2.0, OpenID Connect and other protocols for user identity authentication, and support multi-factor authentication (MFA).

[0116] Permission management: Use a role-based access control mechanism to define different roles and permissions to ensure that only authorized users can access specific data.

[0117] Fine-grained control: Supports fine-grained access control, based on field-level access control, ensures the principle of least privilege for data, and implements strict access control mechanisms to ensure that only authorized users can access specific data.

[0118] (3) Audit tracking;

[0119] Logging: Record all data access and operation logs, including user ID, operation time, operation type, operation results, etc.

[0120] Log analysis: Use ELK Stack (Elasticsearch, Logstash, Kibana) for log management and analysis, supporting real-time monitoring and historical queries.

[0121] Anomaly detection: Detect abnormal behavior in logs through machine learning algorithms to identify potential security threats in a timely manner.

[0122] (4) Test verification;

[0123] Unit testing: Perform unit testing on each module to ensure the correctness and robustness of the code. Use testing frameworks such as JUnit and pytest.

[0124] Integration testing: Perform integration testing on the entire system to ensure the coordination between modules. Use tools such as Selenium and Postman for automated testing.

[0125] Performance testing: Perform load testing and stress testing to evaluate the system's performance bottlenecks. Use tools such as JMeter and LoadRunner to simulate high-concurrency scenarios.

[0126] Security testing: Conduct penetration testing and vulnerability scanning to discover and fix potential security vulnerabilities. Use tools such as OWASPZAP and Nessus for security testing.

[0127] Based on the above method, a device for improving the efficiency and intelligence level of the mining industry in this embodiment includes: at least one memory and at least one processor;

[0128] The at least one memory is used to store a machine-readable program;

[0129] The at least one processor is used to call the machine-readable program to execute a method for improving the efficiency and intelligence level of the mining industry.

[0130] The above-mentioned specific implementations are only specific cases of the present invention. The patent protection scope of the present invention includes but is not limited to the above-mentioned specific implementations. Any technical solutions that conform to the above-mentioned specific implementations of the present invention and any appropriate changes or substitutions made by ordinary technicians in the relevant technical field shall fall within the patent protection scope of the present invention.

[0131] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for improving the efficiency and intelligence level of the mining industry, characterized in that: The steps are as follows: S1, computing resources, storage systems and grid architecture at the infrastructure layer; S2, perform data collection, data cleaning, data labeling, data balancing and data enhancement at the data resource layer; S3, the algorithm model layer performs preliminary training, fine-tuning, prompt word optimization, vector embedding, knowledge specification, and retrieval enhancement generation on a large corpus; S4: Setting of API interface and application of user interface in the application service layer; S5: Data encryption, access control, audit tracking and test verification are performed in the security, trust and testing layer.

2. A method for improving the efficiency and intelligence level of the mining industry according to claim 1, characterized in that: In step S1, when calculating resources, hardware configuration and cloud services are performed. GPU and TPU accelerators are configured in the hardware configuration, each node is configured with at least 1TB of memory and high SSD storage, and Kubernetes is used in the cloud service for container orchestration to achieve efficient management and scheduling of resources; In the storage system, distributed storage, object storage and cache layer are used. In the distributed storage, Hadoop HDFS or Ceph distributed storage system is deployed, and Erasure Coding technology is used to improve storage efficiency; The object storage uses an object storage service to store unstructured data and automatically archives cold data through a lifecycle management strategy; the cache layer introduces a Redis memory database as a cache layer; The network architecture uses InfiniBand network to connect various computing nodes, and uses SD-WAN technology to optimize wide area network connections.

3. A method for improving the efficiency and intelligence level of the mining industry according to claim 2, characterized in that: In step S2, during data collection, the temperature, humidity, gas concentration and equipment operation status data in the mine are collected through IoT sensors, and the video monitoring data in the mine is collected using cameras and infrared cameras, including personnel activities, equipment status and environmental changes; the geological structure and ore deposit distribution data of the mine are collected through geological exploration equipment and stored in GIS format; the log data of various operations in the mine are recorded, including equipment operation records and personnel entry and exit records; When cleaning data, use hash functions and Bloom filters to remove duplicate data, use rule engines and statistical methods to identify and correct errors in the data, use IQR methods to detect and correct outliers, use interpolation, regression models or deep learning methods to fill in missing values, and use anomaly detection algorithms to identify and correct outliers.

4. A method for improving the efficiency and intelligence level of the mining industry according to claim 3, characterized in that: When labeling data, a professional labeling team is established to manually label images, videos, and text data, using existing pre-trained models for preliminary labeling, followed by manual review and correction; When balancing data, oversampling is to use the Synthetic Minority Over-sampling Technique to generate minority class samples to balance the data set; undersampling is to randomly delete majority class samples to reduce the imbalance of the data set; mixed sampling is a method that combines oversampling and undersampling; Data enhancement includes image enhancement, text enhancement and audio enhancement. The image enhancement is to increase the diversity of image data by rotating, scaling, cropping and flipping methods. The text enhancement uses synonym replacement, sentence rewriting and antonym insertion methods to expand text data; the audio enhancement is to enhance the robustness of audio data by adding noise, changing pitch and changing speed.

5. A method for improving the efficiency and intelligence level of the mining industry according to claim 4, characterized in that: In step S3, a large-scale public corpus is used to preliminarily train the large model, using BERT and RoBERTa pre-trained models; pre-training is combined with text, image and audio multimodal data, using CLIP and DALL-E multimodal pre-trained models to improve the generalization ability of the model; Based on the pre-training, the model is fine-tuned using a specific data set for the coal industry. The pre-trained model is fine-tuned according to the specific tasks of the coal industry. The existing pre-trained model is used as the initial weight and fine-tuned with a small amount of labeled data. When the prompt words are optimized, the model is guided to generate more accurate and useful results. For equipment fault diagnosis tasks, specific prompt templates are designed.

6. A method for improving the efficiency and intelligence level of the mining industry according to claim 5, characterized in that: Vector embedding includes text embedding, image embedding and multimodal embedding. The text embedding is to generate a vector representation of text using the BERT and Word2Vec pre-trained models. The image embedding is to generate a vector representation of an image using the ResNet and VGG convolutional neural networks for image classification and retrieval. The multimodal embedding is to combine the embedding vectors of text and image to generate a joint representation of multimodal data. The knowledge specification is to introduce the knowledge and rules of industry experts, inject this knowledge into the model through the constraint optimization method, and correct it through the post-processing step after the model generates the results; When the retrieval enhancement is generated, relevant information is retrieved from historical data, the auxiliary model generates a reply, full-text retrieval is performed using Elasticsearch, and sorting is performed in combination with the BM25 algorithm. The retrieval results and the output of the generated model are combined to generate the final answer through weighted fusion.

7. A method for improving the efficiency and intelligence level of the mining industry according to claim 6, characterized in that: In step S4, a standardized API interface is provided to exchange data in JSON format, and semantic version management is adopted to make the API compatible and stable; In the user interface, develop web applications, provide graphical interfaces, develop mobile applications, support iOS and Android platforms, integrate visualization tools, and generate custom dashboards and reports.

8. A method for improving the efficiency and intelligence level of the mining industry according to claim 7, characterized in that: In step S5, when encrypting data, the TLS / SSL protocol is used to encrypt data transmission, the AES-256 encryption algorithm is used to encrypt stored data, and the KMS is used to generate, store and manage keys; When performing access control, OAuth 2.0 and OpenID Connect protocols are used for user authentication, and a role-based access control mechanism is adopted to define different roles and permissions for fine-grained access control.

9. A method for improving the efficiency and intelligence level of the mining industry according to claim 7, characterized in that: When auditing and tracking, all data access and operation logs are recorded, ELK Stack is used for log management and analysis, and machine learning algorithms are used to detect abnormal behaviors in logs; During test verification, Junit and pytest are used to test each module, and the entire system is integrated tested to ensure the collaboration between modules. Load testing and stress testing are performed to evaluate the performance bottlenecks of the system, and penetration testing and vulnerability scanning are performed to discover and fix potential security vulnerabilities.

10. A device for improving the efficiency and intelligence level of the mining industry, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to call the machine-readable program to execute the method according to any one of claims 1 to 9.

Images