Intelligent mining face distributed edge data real-time processing system

Abstract

This invention relates to a distributed edge data real-time processing system for intelligent mining faces, belonging to the field of intelligent coal mining technology. It includes a coal mining equipment group, a sensor cluster, an edge distributed cluster, and a cloud server. The sensor cluster collects real-time operating data from each fully mechanized mining machine and sends it to the edge distributed cluster. The edge distributed cluster preprocesses and stores the real-time operating data, manages the master node and multiple slave nodes, and performs intelligent diagnosis based on the preprocessed real-time operating data and the intelligent diagnostic model distributed by the cloud server. The cloud server stores the preprocessed real-time operating data and constructs the intelligent diagnostic model. By deploying the edge distributed cluster at the working face, some tasks from the cloud server are distributed to the cloud server, reducing the server's storage pressure and effectively reducing network latency during remote data transmission.

Description

Technical Field

[0001] This invention relates to the field of intelligent coal mining technology, and in particular to a distributed edge data real-time processing system for intelligent mining faces. Background Technology

[0002] Intelligentization is the core technology for efficient production in coal mining faces. With the in-depth application of various intelligent sensors in working faces, the amount of data is growing explosively at a rate of billions per day. By utilizing big data and artificial intelligence technologies, we can mine the potential value of the data, build real-time early warning and auxiliary decision-making models, and guide the efficient mining process.

[0003] Traditional coal mine face data processing systems transmit all data to a cloud server above ground via a ring network. The cloud server then processes and filters the data before storing it in a historical database. Simultaneously, the cloud server must handle numerous computational tasks, such as building excavation models and providing equipment fault warnings. This leads to inefficient allocation of computing resources and excessive network congestion. The massive data transmission and computational tasks hinder the further development of intelligent coal mine face systems. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a distributed edge data real-time processing system for intelligent mining operations. The technical solution of this invention is as follows:

[0005] A real-time distributed edge data processing system for intelligent mining faces includes a coal mining equipment group, a sensor cluster, an edge distributed cluster, and a cloud server;

[0006] The coal mining equipment group includes multiple fully mechanized mining machines on the working face; the sensor cluster includes multiple sensors of various types installed on the fully mechanized mining machines; the edge distributed cluster includes a master node and multiple slave nodes, which are connected to the underground ring network to form a network path; each sensor in the sensor cluster is connected to the master node and / or multiple slave nodes; the cloud server is connected to the master node and / or multiple slave nodes.

[0007] The sensor cluster is used to collect real-time operating data of each fully mechanized mining equipment and send it to the edge distributed cluster; the edge distributed cluster is used to preprocess and store the real-time operating data, manage the master node and multiple slave nodes, and perform intelligent diagnosis based on the preprocessed real-time operating data and the intelligent diagnostic model issued by the cloud server; the cloud server is used to store the preprocessed real-time operating data, construct the intelligent diagnostic model based on the preprocessed real-time operating data, and manage the edge distributed cluster.

[0008] Optionally, the coal mining equipment group includes a coal mining machine, hydraulic supports, a transfer conveyor, an emulsifying pump, and a belt conveyor;

[0009] The sensor cluster includes: speed sensors, current sensors, and temperature sensors installed on the traction motor and cutting motor of the coal mining machine; angle sensors, pressure sensors, and mining height sensors installed on the top beam and column of the hydraulic support; temperature sensors, current sensors, and voltage sensors installed on the motor of the transfer conveyor; temperature sensors and pressure sensors installed on the pump station and oil tank of the emulsion pump; and speed sensors and current sensors installed on the tail drum motor of the belt conveyor.

[0010] Optionally, when the edge distributed cluster preprocesses and stores the real-time operating data, upon detecting new real-time operating data, the master node divides the data preprocessing job into different tasks based on the amount of new real-time operating data, and queries computing resources from each slave node. Each slave node then reports its computing resources back to the master node. The master node determines the currently idle slave node based on the feedback and creates a data stream. The currently idle slave node receives the new real-time operating data, determines whether there are missing values ​​or noisy data in the new real-time operating data, and if so, processes the missing values ​​and noisy data to obtain pre-processed real-time operating data. It then extracts the effective features from the pre-processed real-time operating data as pre-processed real-time operating data and stores them. The master node records the topic and address of the second real-time operating data obtained by each slave node, compresses the pre-processed real-time operating data obtained by each slave node according to the time granularity, uses a sliding window to obtain the average value of all pre-processed real-time operating data within a time period, and then sends it to the cloud server.

[0011] Optionally, when the currently idle slave node determines whether there are missing values ​​and noisy data in the new real-time operating data, it uses a logical NOT operation to determine whether the new real-time operating data has missing values, and determines whether the new real-time operating data is noisy data according to a preset threshold range.

[0012] Optionally, when the currently idle slave node processes missing values ​​and noise data, it performs linear interpolation based on the valid operating condition data before and after the missing values ​​and noise data, and then resets the data at the location of the missing values ​​and noise data.

[0013] Optionally, the edge distributed cluster, when managing the master node and multiple slave nodes, includes the following:

[0014] 1. When the master node receives a task sent by the cloud server, it first divides the task into multiple jobs according to the size of the task data, and then allocates idle slave nodes with computing resources to perform the jobs according to the number of jobs. After all jobs have been completed, the master node cancels itself to release the computing resources it occupies.

[0015] 2. Each slave node periodically reports its own heartbeat signal and available computing resources to the master node. The master node periodically updates the status information of each slave node. The master node determines whether any slave node has failed based on the pre-configured heartbeat timeout threshold. When any slave node fails, the master node clears the data of the failed slave node locally and sends the fault information to the cloud server for display through the ring network.

[0016] Optionally, when the edge distributed cluster performs intelligent diagnosis based on preprocessed real-time operating condition data and the intelligent diagnostic model issued by the cloud server, it inputs the preprocessed real-time operating condition data into the intelligent diagnostic model issued by the cloud server, obtains the intelligent diagnostic result based on the output result of the intelligent diagnostic model issued by the cloud server, and sends the intelligent diagnostic result to the cloud server for display.

[0017] Optionally, when the cloud server constructs an intelligent diagnostic model based on preprocessed real-time operating condition data, it trains the intelligent diagnostic model using historically received preprocessed real-time operating condition data stored in its historical database to obtain a trained intelligent diagnostic model. The trained intelligent diagnostic model is then updated using preprocessed real-time operating condition data sent in real-time by the edge distributed cluster, and the updated intelligent diagnostic model is distributed to the master and slave nodes in the edge distributed cluster.

[0018] Optionally, when managing the edge distributed cluster, the cloud server queries the status information of the master node and each slave node from the master node in real time and displays the status information of the master node and each slave node.

[0019] All of the above-mentioned optional technical solutions can be combined arbitrarily, and the present invention will not provide a detailed description of the structure after each combination.

[0020] By means of the above solution, the beneficial effects of the present invention are as follows:

[0021] This invention deploys an edge-distributed cluster at the work surface, offloading some processing tasks from the cloud server (preprocessing of real-time operating data and application of intelligent diagnostic models) to the cloud server. This reduces server storage pressure and effectively minimizes network latency during remote data transmission. The interaction between the cloud server and the edge-distributed cluster enables node management of the master node and multiple slave nodes, achieving rational allocation of hardware resources.

[0022] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the system composition structure of the present invention.

[0024] Figure 2 This is a schematic diagram illustrating the interaction process between the edge distributed cluster and the cloud server in the system provided by the embodiments of the present invention. Detailed Implementation

[0025] The specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples. The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0026] like Figure 1 As shown in the figure, the intelligent mining face distributed edge data real-time processing system provided in this embodiment of the invention includes a coal mining equipment group, a sensor cluster, an edge distributed cluster, and a cloud server;

[0027] The coal mining equipment group includes multiple fully mechanized mining machines on the working face; the sensor cluster includes multiple sensors of various types installed on the fully mechanized mining machines; the edge distributed cluster includes a master node and multiple slave nodes, which are connected to the underground ring network to form a network path; each sensor in the sensor cluster is connected to the master node and / or multiple slave nodes; the cloud server is connected to the master node and / or multiple slave nodes; the sensor cluster is used to collect real-time operating data of each fully mechanized mining machine and send it to the edge distributed cluster; the edge distributed cluster is used to preprocess and store the real-time operating data, manage the master node and multiple slave nodes, and perform intelligent diagnosis based on the preprocessed real-time operating data and the intelligent diagnostic model issued by the cloud server; the cloud server is used to store the preprocessed real-time operating data, construct the intelligent diagnostic model based on the preprocessed real-time operating data, and manage the edge distributed cluster.

[0028] In specific implementation of this invention, the master and slave nodes in the edge distributed cluster are inexpensive and easily scalable Raspberry Pi edge computers. After achieving network communication between the edge computers through an underground ring network, a distributed real-time data processing cluster is formed by deploying the Apache community's distributed computing framework (Hadoop) and its components (Kafka, Flink).

[0029] The coal mining equipment group includes a coal mining machine, hydraulic supports, a transfer conveyor, an emulsifying pump, and a belt conveyor. The sensor cluster includes: speed sensors, current sensors, and temperature sensors installed on the traction motor and cutting motor of the coal mining machine to collect data on the traction speed, cutting current, and cutting temperature of the coal mining machine; angle sensors, pressure sensors, and mining height sensors installed on the top beam and columns of the hydraulic supports to collect data on the angle of the top beam / base, column pressure, and support mining height; temperature sensors, current sensors, and voltage sensors installed on the motor of the transfer conveyor to collect data on the temperature, current, and voltage of the transfer conveyor; temperature sensors and pressure sensors installed on the pump station and oil tank of the emulsifying pump to collect data on bearing temperature, oil temperature, and oil pressure; and speed sensors and current sensors installed on the tail drum motor of the belt conveyor to collect data on belt speed and motor current. The real-time operating data collected by the sensor cluster is transmitted to an edge distributed cluster via CAN / Modbus / OPC protocols.

[0030] As a specific implementation, when the edge distributed cluster preprocesses and stores the real-time operational data, the cloud server sends a task to the master node upon detecting new real-time operational data. Upon receiving the task, the master node automatically triggers a Flink data preprocessing streaming job and starts the job manager to coordinate and manage the entire job. Specifically, the master node divides the data preprocessing job into different tasks based on the volume of the new real-time operational data and queries the slave nodes for computing resources. Each slave node reports its computing resources back to the master node. Based on the feedback, the master node determines the currently idle slave nodes and creates a data stream, which is used to allocate the workload of each slave node for that task. The currently idle slave nodes receive the new real-time operational data, determine whether there are missing values ​​or noisy data, and if so, process the missing values ​​and noisy data to obtain pre-processed real-time operational data. They then extract the valid features from the pre-processed real-time operational data as pre-processed real-time operational data and store them. Valid features may include numerical values, timestamps, and variable names. By extracting effective features, the data format of real-time operating data collected by various sensors in the sensor cluster can be standardized. When storing preprocessed real-time operating data, the master or slave nodes store it in a Kafka message queue. The master node records the topic and address of the preprocessed real-time operating data obtained by each slave node. It then compresses the preprocessed real-time operating data obtained by each slave node according to time granularity, uses a sliding window to obtain the average value of all preprocessed real-time operating data within a time period, and sends it to the cloud server. The topic represents the data summary, and the address represents the data storage location. By compressing according to time granularity and calculating the average value within the sliding window, the problem of inconsistent acquisition timing can be solved. The master node can import the processed real-time operating data into the historical database of the cloud server through a JDBC interface with user-configured parameters. The cloud server stores the preprocessed real-time operating data in the historical database.

[0031] Optionally, when determining whether new real-time operating condition data contains missing values ​​and noise, the currently idle slave node uses a logical NOT operation to determine whether the new real-time operating condition data contains missing values, and determines whether the new real-time operating condition data is noise based on a preset threshold range. Further, when processing missing values ​​and noise data, the currently idle slave node performs linear interpolation based on the valid operating condition data before and after the missing values ​​and noise data, and then resets the data at the positions where the missing values ​​and noise data are located.

[0032] By preprocessing the real-time operating data as described above, four problems with the real-time operating data collected by the sensor cluster can be solved: (1) The real-time operating data is noisy due to interference from factors such as humidity and vibration. (2) Due to environmental complexity and equipment operation problems, some real-time operating data may be missing or lost, resulting in an incomplete dataset. (3) Due to signal transmission delay, inconsistent acquisition frequency, or equipment synchronization problems, the timing of real-time operating data collected by different sensors may be inaccurate. (4) Due to differences in data acquisition methods, frequencies, formats, etc., between different devices, the real-time operating data format may be inconsistent.

[0033] As a specific implementation method, the master node and slave nodes of the cluster are customized when building an edge distributed cluster. This embodiment of the invention includes the following aspects when managing the master node and multiple slave nodes:

[0034] 1. When the master node receives a task sent by the cloud server, it first divides the task into multiple jobs according to the size of the task data, and then allocates idle slave nodes with computing resources to perform the jobs according to the number of jobs. After all jobs have been completed, the master node cancels itself to release the computing resources it occupies.

[0035] 2. Each slave node periodically reports its heartbeat signal and available computing resources to the master node. The master node periodically updates the status information (resource utilization, load, etc.) of each slave node. Based on a pre-configured heartbeat timeout threshold, the master node determines whether any slave node has failed. When any slave node fails, the master node locally deletes the data of the failed slave node and sends the fault information to the cloud server via the ring network for display. After receiving the fault information, the cloud server dispatches technical personnel to resolve the problem to ensure the normal operation of the edge distributed cluster.

[0036] In this embodiment of the invention, the cloud server uses the server in the ground control center as its hardware foundation and is responsible for building the data mining model. Users log in to the cloud server through the front-end interface of the coal mine ground control center to view the real-time working conditions of the working face, build an intelligent diagnostic model, and then distribute the built intelligent diagnostic model to the edge distributed cluster for computation. Specifically, when building the intelligent diagnostic model based on preprocessed real-time working condition data, the cloud server trains the intelligent diagnostic model using historically received preprocessed real-time working condition data stored in its historical database to obtain a trained intelligent diagnostic model. The trained intelligent diagnostic model is then updated using preprocessed real-time working condition data sent in real-time by the edge distributed cluster, and the updated intelligent diagnostic model is distributed to the master and slave nodes in the edge distributed cluster. When performing intelligent diagnosis based on the preprocessed real-time working condition data and the intelligent diagnostic model distributed by the cloud server, the edge distributed cluster inputs the preprocessed real-time working condition data into the intelligent diagnostic model distributed by the cloud server, obtains the intelligent diagnostic result based on the output of the intelligent diagnostic model distributed by the cloud server, and sends the intelligent diagnostic result to the cloud server for display.

[0037] Because training machine learning models requires significant computing resources, this embodiment of the invention places the complex task of building the intelligent diagnostic model on a cloud server. Once the intelligent diagnostic model is built, the cloud server distributes the model file to the edge distributed cluster. When intelligent diagnosis is needed, the edge distributed cluster inputs preprocessed real-time operating data into the intelligent diagnostic model. The intelligent diagnostic model outputs the diagnostic results and sends them to the cloud server, which displays the results in real time. If manual maintenance is required, it prompts staff for assistance.

[0038] The embodiments of this invention do not specifically limit the type and application of intelligent diagnostic models. For example, an intelligent diagnostic model can be a fault diagnosis model for diagnosing coal mining machine faults, a classification model for identifying fault types, etc. Figure 2 As shown, it is a schematic diagram of the interaction process between the edge distributed cluster and the cloud server in the system provided by the embodiment of the present invention.

[0039] In addition, the cloud server in this embodiment of the invention is also responsible for managing the edge distributed cluster. Specifically, when managing the edge distributed cluster, the cloud server queries the status information of the master node and each slave node from the master node in real time, and displays the status information of the master node and each slave node.

[0040] In summary, the system provided by this invention distributes some data preprocessing and intelligent diagnostic tasks from the cloud server to the edge distributed cluster. After preprocessing, various real-time working condition data of the working face are temporarily stored on the master node or slave node in the edge distributed cluster, and also stored in the historical database of the cloud server on the well. This can ensure the real-time performance of data processing, reduce network congestion, and relieve the computing pressure on the cloud server.

[0041] The system provided in this embodiment of the invention has the following characteristics:

[0042] 1. Deploying an edge-distributed cluster on the work surface to offload some of the storage and computing tasks from the cloud server can effectively improve the real-time performance of data processing. By setting the master and slave nodes as inexpensive and easily scalable Raspberry Pi edge computers, the overall system cost remains low.

[0043] 2. By setting up an edge distributed cluster, load balancing can be achieved during the computing process, avoiding a situation where one computer consumes huge computing resources while other computers do not utilize them.

[0044] 3. Free cloud servers from cumbersome data transmission and processing, and apply computing resources mainly to the mining and construction of intelligent diagnostic models.

[0045] 4. The edge distributed cluster distributes data, which can ensure the real-time nature of the data and store the data in the historical database of the cloud server through data compression algorithms, thus avoiding the increase of storage pressure on the historical database due to excessive collection frequency.

[0046] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A distributed edge data real-time processing system for intelligent mining faces, characterized in that, This includes coal mining equipment clusters, sensor clusters, edge distributed clusters, and cloud servers; The coal mining equipment group includes multiple fully mechanized mining machines on the working face; the sensor cluster includes multiple sensors of various types installed on the fully mechanized mining machines; the edge distributed cluster includes a master node and multiple slave nodes, which are connected to the underground ring network to form a network path; each sensor in the sensor cluster is connected to the master node and / or multiple slave nodes; the cloud server is connected to the master node and / or multiple slave nodes. The sensor cluster is used to collect real-time operating data of each fully mechanized mining device and send it to the edge distributed cluster. The edge distributed cluster is used to preprocess and store the real-time operating data, manage the master node and multiple slave nodes, and perform intelligent diagnosis based on the preprocessed real-time operating data and the intelligent diagnostic model issued by the cloud server. The cloud server is used to store preprocessed real-time operating condition data, construct intelligent diagnostic models based on the preprocessed real-time operating condition data, and manage the edge distributed cluster. When the edge distributed cluster preprocesses and stores the real-time operating data, the cloud server sends a task to the master node when new real-time operating data is detected. After receiving the task, the master node automatically triggers the Flink data preprocessing streaming job and starts the job manager to coordinate and manage the entire job. Specifically, the master node divides the data preprocessing job into different tasks according to the amount of new real-time operating data and queries the computing resources of each slave node. Each slave node feeds back the computing resources to the master node. The master node determines the currently idle slave node based on the feedback results and then creates a data stream. The currently idle slave node receives new real-time operational data, determines whether there are missing values ​​or noisy data in the new real-time operational data, and if so, processes the missing values ​​and noisy data to obtain pre-processed real-time operational data. It then extracts the valid features from the pre-processed real-time operational data as pre-processed real-time operational data and stores it. When storing the pre-processed real-time operational data, the master node or slave node stores it in a Kafka message queue. The master node records the topic and address of the real-time operational data obtained by each slave node, compresses the pre-processed real-time operational data obtained by each slave node according to the time granularity, uses a sliding window to obtain all the pre-processed real-time operational data within a time period, calculates its average, and then sends it to the cloud server.

2. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, The coal mining equipment group includes a coal mining machine, hydraulic supports, a transfer conveyor, an emulsifying pump, and a belt conveyor. The sensor cluster includes: speed sensors, current sensors, and temperature sensors installed on the traction motor and cutting motor of the coal mining machine; angle sensors, pressure sensors, and mining height sensors installed on the top beam and column of the hydraulic support; temperature sensors, current sensors, and voltage sensors installed on the motor of the transfer conveyor; temperature sensors and pressure sensors installed on the pump station and oil tank of the emulsion pump; and speed sensors and current sensors installed on the tail drum motor of the belt conveyor.

3. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, When the currently idle slave node determines whether there are missing values ​​or noisy data in the new real-time operating data, it uses a logical NOT operation to determine whether the new real-time operating data has missing values, and determines whether the new real-time operating data is noisy data based on a preset threshold range.

4. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, When the currently idle slave node processes missing values ​​and noise data, it performs linear interpolation based on the valid operating condition data before and after the missing values ​​and noise data, and then resets the data at the location of the missing values ​​and noise data.

5. The intelligent mining face distributed edge data real-time processing system according to claim 1, 3, or 4, characterized in that, When managing the master node and multiple slave nodes, the edge distributed cluster includes the following:

1. When the master node receives a task sent by the cloud server, it first divides the task into multiple jobs according to the size of the task data, and then allocates idle slave nodes with computing resources to perform the jobs according to the number of jobs. After all jobs have been completed, the master node cancels itself to release the computing resources it occupies.

2. Each slave node periodically reports its own heartbeat signal and available computing resources to the master node. The master node periodically updates the status information of each slave node. The master node determines whether any slave node has failed based on the pre-configured heartbeat timeout threshold. When any slave node fails, the master node clears the data of the failed slave node locally and sends the fault information to the cloud server for display through the ring network.

6. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, When performing intelligent diagnosis based on preprocessed real-time operating data and the intelligent diagnostic model issued by the cloud server, the edge distributed cluster inputs the preprocessed real-time operating data into the intelligent diagnostic model issued by the cloud server, obtains the intelligent diagnostic result based on the output of the intelligent diagnostic model issued by the cloud server, and sends the intelligent diagnostic result to the cloud server for display.

7. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, When constructing an intelligent diagnostic model based on preprocessed real-time operating data, the cloud server trains the intelligent diagnostic model using historically received preprocessed real-time operating data stored in its historical database to obtain a trained intelligent diagnostic model. The trained intelligent diagnostic model is then updated using preprocessed real-time operating data sent in real-time by the edge distributed cluster, and the updated intelligent diagnostic model is distributed to the master and slave nodes in the edge distributed cluster.

8. The intelligent mining face distributed edge data real-time processing system according to claim 1, characterized in that, When managing the edge distributed cluster, the cloud server queries the status information of the master node and each slave node from the master node in real time and displays the status information of the master node and each slave node.

Images