Mine safety intelligent monitoring method and system based on multi-source data fusion

Abstract

The invention discloses a mine safety intelligent monitoring method and system based on multi-source data fusion, and the method comprises the steps: collecting the data of each sensor of a mine, carrying out the preprocessing of the multi-source data, carrying out the spatial reconstruction of the processed multi-source data, and carrying out the separation of data features through a multi-window sliding scanning mechanism. And expanding the diversity of the feature space through a four-way scanning mechanism and a channel shuffling mechanism, and finally performing deep feature fusion and extraction on the reconstructed multi-source data by selecting a state space model to predict the safety conditions of the mining equipment and the environment. According to the method, the state space model is used as a main body framework of an intelligent monitoring algorithm to reduce the calculation complexity of the model, the reconstructed multi-source data is input into the improved state space model, the implicit relation among the multi-source data is extracted, and the mine safety condition is estimated; the spatial diversity of multi-source data samples and multi-source heterogeneous features is increased, and the feature extraction efficiency and the calculation efficiency and prediction capability of a prediction model are improved.

Description

Technical Field

[0001] The invention relates to a mine safety intelligent monitoring method and system based on multi-source data fusion, belonging to the technical field of safety monitoring. Background Art

[0002] In the process of coal mining and production, in order to fully grasp the environmental conditions in the mine and realize the judgment of the safety status of the mine, multiple or multiple sensors are needed to monitor the environmental parameters of the mine and perform data fusion. By comprehensively analyzing the data collected by the coal mine multi-sensor monitoring system, extracting feature data, and establishing a multi-parameter fusion mine safety warning, it is of great significance to improve coal mine production safety.

[0003] Existing data analysis methods, such as deep learning and big data technology, are widely used in the field of mine safety monitoring to better handle complex data and scale performance. Typical analysis methods include fuzzy neural network-based methods, transformer-based methods, recurrent neural network (RNN)-based methods, convolutional neural network (CNN)-based methods, and graph neural network (GAN)-based methods. Although these methods can process complex data, they have high computational complexity, low computational efficiency for long sequence data, and cannot model sequence information outside a limited window; in addition, these methods usually directly feed different types of data into the network for training, and then fuse the features of different types of data, which has low fitting efficiency and cannot capture the deep feature relationship between multi-source data, so they cannot accurately estimate the real situation of the equipment and the environment.

[0004] In summary, the current mine safety monitoring has the following main defects: (1) The computational efficiency of long sequence data is low, and it is impossible to model the sequence information outside the limited window; (2) The sample space is single, and different types of data lack interaction, which cannot capture the deep characteristic relationship between multi-source data, resulting in low model fitting efficiency. Summary of the invention

[0005] The purpose of the invention is to provide a method and system for intelligent mine safety monitoring based on multi-source data fusion, which can improve data feature extraction and calculation efficiency, strengthen the fusion between data, and ultimately improve the accuracy of mine safety prediction, thereby providing protection for safe production in coal mines.

[0006] In order to achieve the above object, the present invention provides a mine safety intelligent monitoring method based on multi-source data fusion, comprising the following steps:

[0007] S1. Preprocessing the data collected by each sensor, including cleaning, feature selection, feature transformation and data conversion, converting the data into a form suitable for feature extraction to obtain multi-source data presenting three dimensions, namely, time dimension, location dimension and sensor type dimension;

[0008] S2, the obtained multi-source data enters two branches at the same time, namely the 4D selective scanning branch and the window cropping branch; the 4D selective scanning branch reads data from the upper left corner, upper right corner, lower left corner, and upper right corner of each type of sensor data in a top-down and left-to-right order; the window cropping branch first divides the window and divides each feature F of size H×W into m Crop into multiple independent sub-windows of size HW / mn, where m and n are the height and width of the cropped sub-window, respectively, and then perform feature extraction on the sub-windows;

[0009] S3, sub-window feature extraction stage, expand each type of sensor data along the time dimension and position dimension, and perform four-way scanning in the order of upper left to lower right, lower right to upper left, upper right to lower left, and lower left to upper right corner to obtain the input of the state space model, and realize the spatial reconstruction of various types of sensor data, that is, multi-source data features;

[0010] S4. Input the spatially reconstructed multi-source data feature sequence into the state space model and perform mine safety status prediction.

[0011] Furthermore, the specific process of S1 is as follows:

[0012] S1-1. Data cleaning: Use the visualization tool describe() method to check the basic information of the data set, use the isnull() method to filter out missing values, use visualization tools to identify outliers, and use the duplicated() method to filter out duplicate values; ensure data quality, reduce errors, improve analysis results, reduce bias, and improve data availability;

[0013] S1-2. Feature selection: PCA principal component analysis is used to reduce the dimension of features, retain important information, eliminate irrelevant variables, and reduce redundant data dimensions to improve modeling efficiency and accuracy. The dimensionality reduction steps are as follows:

[0014] S1-2.1, organize each type of sensor data into a matrix with p rows and q columns;

[0015] S1-2.2, average each row of data, i.e., the features at different moments;

[0016] S1-2.3. Find the covariance matrix of the matrix Among them, XX represents the input data;

[0017] S1-2.4, find the eigenvalues ​​and eigenvectors of the covariance matrix E;

[0018] S1-2.5. Arrange the eigenvectors from high to low according to their corresponding eigenvalues ​​to form a matrix, and select the first k rows to form a new matrix P; obtain Y = PX, which is the data after dimension reduction to k dimensions;

[0019] S1-3, feature transformation: standardize the data to eliminate the dimensional differences between features and improve the stability and convergence speed of model training. The specific standardization process is as follows:

[0020]

[0021] Among them, X represents sensor data, X min Represents the maximum value of the data, X max Represents the minimum value of the data;

[0022] S1-4, data conversion: convert the data into a form suitable for model training. First, export the sensor data of the same type at multiple locations in the mine, splice them by column, and obtain a two-dimensional matrix U. Each row of U is the sensor data at a different time, and the column of U is the sensor data at different locations. Then, export the vibration sensor two-dimensional data U through the local area network. a , gas sensor two-dimensional data U b , sound sensor two-dimensional data U c , and the humidity sensor two-dimensional data D, use the concate() method to splice them to obtain the three-dimensional data X, that is, multi-source data.

[0023] Furthermore, S2 also includes performing maximum pooling on the features scanned by the scanning branch, extracting features of different windows, and then sending the extracted multi-windows to the 4D selection module, communicating between sub-windows through the state space model, and then restoring the feature size of multi-source data through deconvolution. The specific formula is as follows:

[0024] F i-2 =windows(F i-3 )

[0025] F i-1 = pooling(F i-2 )

[0026] F i =deconv(F i-1 )

[0027] Among them, F iRepresents the features of the current layer, windows represents the window cropping operation, pooling represents the pooling operation, and deconv represents the deconvolution operation.

[0028] Furthermore, the specific process of the S3 four-way scan is as follows:

[0029] The spliced ​​multi-source data is expanded along rows and columns, and then scanned along four different directions. The sensor data is straightened, rotated, and flipped using the reshape(), transpose(), and flip() methods. The data is read from different directions, specifically from the top left to the bottom right, the bottom right to the top left, the top right to the bottom left, and the bottom left to the top right corner of the two-dimensional data, and the sensor data is read from top to bottom and from left to right. In this way, multi-source data at the same time can be obtained, and data of the same data type at different time points can also be obtained, so that the feature space is not limited to the same time or the same type.

[0030] Furthermore, the specific process of S4 is as follows:

[0031] S4-1. Input the multi-source feature sequence after spatial reconstruction into the state space model. The state space model maps the multi-source data through a hidden potential state, which is defined by four model parameters: (A, B, C, D). The formula for sequence-to-sequence conversion is described as:

[0032] h′(t)=Ah(t)+Bx(t)

[0033] y(t)=Ch(t)+Dx(t);

[0034] Among them, x(t) represents the mapping input sequence, h′(t) represents the potential state at the current time t, y(t) represents the prediction sequence, A, B, C, D are model parameters, N represents the time state;

[0035] S4-2, the state space model prediction consists of linear mapping and softmax function. The features extracted by the state space model are input into the linear mapping layer, the danger index at the current moment is output, and the danger index is input into the softmax activation function. If the obtained value is greater than 0.5, an alarm is activated; the formulas for adaptive average pooling and linear mapping are as follows:

[0036] F i = liner(F i-1 )

[0037] F i+1 =softmax(F i );

[0038] In the formula, F iRepresents the features of the current layer, liner represents the linear mapping layer, and softmax produces category probabilities.

[0039] A mine safety intelligent monitoring system based on multi-source data fusion, characterized by comprising a data acquisition module, a data processing module, an intelligent analysis module and an alarm feedback module;

[0040] The data acquisition module integrates a vibration sensor, a concentration sensor, a sound sensor, and a humidity sensor, and is used to collect mining environmental parameters in real time, including vibration signals of equipment, gas concentration data signals, sound data signals, and air humidity data signals;

[0041] The data processing module is used to pre-process the collected data, including denoising, normalization and format conversion, and is also used for data storage and management;

[0042] The intelligent analysis module uses advanced deep learning algorithms to conduct in-depth analysis of the fused data to identify potential safety hazards, generate environmental status assessment reports, and predict possible dangerous situations;

[0043] The alarm feedback module is used to issue an alarm when the intelligent analysis module detects an abnormal situation, and to display the real-time status of the current mining environment and specific safety hazard information to the operator through a visual interface, and to send the alarm information to the mobile terminal.

[0044] The present invention collects data from various sensors in the mine, pre-processes the multi-source data, and then spatially reconstructs the processed multi-source data. Then, the data features are separated by means of a multi-window sliding scanning mechanism to increase the in-line nature of the local multi-source data. The feature space diversity is then expanded by a four-way scanning mechanism and a channel shuffling mechanism. Finally, the state space model is selected to perform deep feature fusion and extraction on the reconstructed multi-source data to predict the safety status of the mine equipment and environment. The present invention uses the state space model as the main framework of the intelligent monitoring algorithm to reduce the model calculation complexity, inputs the reconstructed multi-source data into the improved state space model, extracts the implicit connection between the multi-source data, and estimates the safety status of the mine, thereby increasing the spatial diversity of multi-source data samples and multi-source heterogeneous features, improving the feature extraction efficiency and the calculation efficiency and prediction ability of the prediction model, and providing a guarantee for coal mine safety production. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of the present invention;

[0046] Figure 2 It is a flow chart of the 4D selective scanning branch and the window clipping branch in the present invention;

[0047] Figure 3 It is a schematic diagram of the four-way scanning method in the present invention. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, a mine safety intelligent monitoring method based on multi-source data fusion includes the following steps:

[0050] S1. Preprocessing the data collected by each sensor, including cleaning, feature selection, feature transformation and data conversion, converting the data into a form suitable for feature extraction to obtain multi-source data presenting three dimensions, namely, time dimension, location dimension and sensor type dimension;

[0051] S2, the obtained multi-source data enters two branches at the same time, namely the 4D selective scanning branch and the window cropping branch; the 4D selective scanning branch reads data from the upper left corner, upper right corner, lower left corner, and upper right corner of each type of sensor data in a top-down and left-to-right order; the window cropping branch first divides the window and divides each feature F of size H×W into m Crop into multiple independent sub-windows of size HW / mn, where m and n are the height and width of the cropped sub-window, respectively, and then perform feature extraction on the sub-windows;

[0052] S3, sub-window feature extraction stage, expand each type of sensor data along the time dimension and position dimension, and perform four-way scanning in the order of upper left to lower right, lower right to upper left, upper right to lower left, and lower left to upper right corner to obtain the input of the state space model, and realize the spatial reconstruction of various types of sensor data, that is, multi-source data features;

[0053] S4. Input the spatially reconstructed multi-source data feature sequence into the state space model and perform mine safety status prediction.

[0054] Furthermore, the specific process of S1 is as follows:

[0055] S1-1. Data cleaning: Use the visualization tool describe() method to check the basic information of the data set, use the isnull() method to filter out missing values, use visualization tools to identify outliers, and use the duplicated() method to filter out duplicate values; ensure data quality, reduce errors, improve analysis results, reduce bias, and improve data availability;

[0056] S1-2. Feature selection: PCA principal component analysis is used to reduce the dimension of features, retain important information, eliminate irrelevant variables, and reduce redundant data dimensions to improve modeling efficiency and accuracy. The dimensionality reduction steps are as follows:

[0057] S1-2.1, organize each type of sensor data into a matrix with p rows and q columns;

[0058] S1-2.2, average each row of data, i.e., the features at different moments;

[0059] S1-2.3. Find the covariance matrix of the matrix Among them, XX represents the input data;

[0060] S1-2.4, find the eigenvalues ​​and eigenvectors of the covariance matrix E;

[0061] S1-2.5. Arrange the eigenvectors from high to low according to their corresponding eigenvalues ​​to form a matrix, and select the first k rows to form a new matrix P; obtain Y = PX, which is the data after dimension reduction to k dimensions;

[0062] S1-3, feature transformation: standardize the data to eliminate the dimensional differences between features and improve the stability and convergence speed of model training. The specific standardization process is as follows:

[0063]

[0064] Among them, X represents sensor data, X min Represents the maximum value of the data, X max Represents the minimum value of the data;

[0065] S1-4, data conversion: convert the data into a form suitable for model training. First, export the sensor data of the same type at multiple locations in the mine, splice them by column, and obtain a two-dimensional matrix U. Each row of U is the sensor data at a different time, and the column of U is the sensor data at different locations. Then, export the vibration sensor two-dimensional data U through the local area network. a , gas sensor two-dimensional data U b , sound sensor two-dimensional data U c , and humidity sensor two-dimensional data U d , use the concate() method to splice and obtain the three-dimensional data X, that is, multi-source data.

[0066] like Figure 2 As shown in FIG. 1 , since the data division destroys the integrity of the original multi-source data, it is necessary to re-establish the communication between multiple local adjacent types of multi-source data. In S2, the features clipped by the window clipping branch are subjected to maximum pooling to extract the features of different windows, and then the extracted multi-windows are further sent to the 4D selection module to communicate between the sub-windows through the state space model, and then the multi-source data feature size is restored by deconvolution. The specific formula is as follows:

[0067] F i-2 =windows(F i-3 )

[0068] F i-1 = pooling(F i-2 )

[0069] F i =deconv(F i-1 )

[0070] Among them, F i Represents the features of the current layer, windows represents the window cropping operation, pooling represents the pooling operation, and deconv represents the deconvolution operation.

[0071] In the 4D selective scanning process, in order to allow the model to have more diverse input sequences, the feature space is not limited to the same time or the same type. The selective scanning of the state space model is extended by a four-way scanning mechanism, such as Figure 3 As shown, the specific process of four-way scanning is:

[0072] The spliced ​​multi-source data is expanded along rows and columns, and then scanned along four different directions. The sensor data is straightened, rotated, and flipped using the reshape(), transpose(), and flip() methods. The data is read from different directions, specifically from the top left to the bottom right, the bottom right to the top left, the top right to the bottom left, and the bottom left to the top right corner of the two-dimensional data, and the sensor data is read from top to bottom and from left to right. In this way, multi-source data at the same time can be obtained, and data of the same data type at different time points can also be obtained, so that the feature space is not limited to the same time or the same type.

[0073] Furthermore, the specific process of S4 is as follows:

[0074] S4-1. Input the multi-source feature sequence after spatial reconstruction into the state space model. The state space model maps the multi-source data through a hidden potential state, which is defined by four model parameters: (A, B, C, D). The formula for sequence-to-sequence conversion is described as:

[0075] h′(t)=Ah(t)+Bx(t)

[0076] y(t)=Ch(t)+Dx(t);

[0077] Among them, x(t) represents the mapping input sequence, h′(t) represents the potential state at the current time t, y(t) represents the prediction sequence, A, B, C, D are model parameters, N represents the time state;

[0078] S4-2, the state space model prediction consists of linear mapping and softmax function. The features extracted by the state space model are input into the linear mapping layer, the danger index at the current moment is output, and the danger index is input into the softmax activation function. If the obtained value is greater than 0.5, an alarm is activated; the formulas for adaptive average pooling and linear mapping are as follows:

[0079] F i = liner(F i-1 )

[0080] F i+1 =softmax(F i );

[0081] In the formula, F i Represents the features of the current layer, liner represents the linear mapping layer, and softmax produces category probabilities.

[0082] The present invention also provides a mine safety intelligent monitoring system based on multi-source data fusion, including a data acquisition module, a data processing module, an intelligent analysis module and an alarm feedback module;

[0083] The data acquisition module integrates a vibration sensor, a concentration sensor, a sound sensor, and a humidity sensor, and is used to collect mining environmental parameters in real time, including vibration signals of equipment, gas concentration data signals, sound data signals, and air humidity data signals;

[0084] The data processing module is used to pre-process the collected data, including denoising, normalization and format conversion, and is also used for data storage and management;

[0085] The intelligent analysis module uses advanced deep learning algorithms to conduct in-depth analysis of the fused data to identify potential safety hazards, generate environmental status assessment reports, and predict possible dangerous situations;

[0086] The alarm feedback module is used to issue an alarm when the intelligent analysis module detects an abnormal situation, and to display the real-time status of the current mining environment and specific safety hazard information to the operator through a visual interface, and to send the alarm information to the mobile terminal.

[0087] Through the effective fusion and intelligent analysis of multi-source data, the present invention can realize dynamic monitoring and early warning of the mine environment, improve the scientificity and effectiveness of safety management, and provide comprehensive safety protection for mine operations.

Claims

1. A mine safety intelligent monitoring method based on multi-source data fusion, characterized in that: The steps include: S1. Preprocessing the data collected by each sensor, including cleaning, feature selection, feature transformation and data conversion, converting the data into a form suitable for feature extraction to obtain multi-source data presenting three dimensions, namely, time dimension, location dimension and sensor type dimension; S2, the obtained multi-source data enters two branches at the same time, namely the 4D selective scanning branch and the window cropping branch; the 4D selective scanning branch reads data from the upper left corner, upper right corner, lower left corner, and upper right corner of each type of sensor data in a top-down and left-to-right order; the window cropping branch first divides the window and divides each feature F of size H×W into m Crop into multiple independent sub-windows of size HW / mn, where m and n are the height and width of the cropped sub-window, respectively, and then perform feature extraction on the sub-windows; S3, sub-window feature extraction stage, expand each type of sensor data along the time dimension and position dimension, and perform four-way scanning in the order of upper left to lower right, lower right to upper left, upper right to lower left, and lower left to upper right corner to obtain the input of the state space model, and realize the spatial reconstruction of various types of sensor data, that is, multi-source data features; S4. Input the spatially reconstructed multi-source data feature sequence into the state space model and perform mine safety status prediction.

2. The mine safety intelligent monitoring method based on multi-source data fusion according to claim 1 is characterized in that: The specific process of S1 is as follows: S1-1. Data cleaning: Use the describe() method of the visualization tool to check the basic information of the data set, use the isnull() method to filter out missing values, use visualization tools to identify outliers, and use the duplicated() method to filter out duplicate values; S1-2. Feature selection: Use PCA principal component analysis to reduce the dimension of features. The steps of dimension reduction are as follows: S1-2.1, organize each type of sensor data into a matrix with p rows and q columns; S1-2.2, average each row of data, i.e., the features at different moments; S1-2.

3. Find the covariance matrix of the matrix Among them, XX represents the input data; S1-2.4, find the eigenvalues ​​and eigenvectors of the covariance matrix E; S1-2.

5. Arrange the eigenvectors from high to low according to their corresponding eigenvalues ​​to form a matrix, and select the first k rows to form a new matrix P; obtain Y = PX, which is the data after dimension reduction to k dimensions; S1-3. Feature transformation: Standardize the data. The specific standardization process is as follows: Among them, X represents sensor data, X min Represents the maximum value of the data, X max Represents the minimum value of the data; S1-4, data conversion: convert the data into a form suitable for model training. First, export the sensor data of the same type at multiple locations in the mine, splice them by column, and obtain a two-dimensional matrix U. Each row of U is the sensor data at a different time, and the column of U is the sensor data at different locations. Then, export the vibration sensor two-dimensional data U through the local area network. a , gas sensor two-dimensional data Ub, sound sensor two-dimensional data U c , and humidity sensor two-dimensional data U d , use the concate() method to splice and obtain the three-dimensional data X, that is, multi-source data.

3. The mine safety intelligent monitoring method based on multi-source data fusion according to claim 2 is characterized in that: In the S2, the features after being clipped by the window clipping branch are also subjected to maximum pooling to extract features of different windows, and then the extracted multi-windows are further sent to the 4D selection module to communicate between sub-windows through the state space model, and then the feature size of multi-source data is restored by deconvolution. The specific formula is as follows: F i-2 =windows(F i-3 ) F i-1 =pooling(F i-2 ) F i =deconv(F i-1 ) Among them, F i Represents the features of the current layer, windows represents the window cropping operation, pooling represents the pooling operation, and deconv represents the deconvolution operation.

4. The mine safety intelligent monitoring method based on multi-source data fusion according to claim 3 is characterized in that: The specific process of the S3 four-way scanning is as follows: The spliced ​​multi-source data is expanded along rows and columns, and then scanned along four different directions. The reshape(), transpose(), and flip() methods are used to straighten, rotate, and flip the sensor data. The data is read from different directions, specifically from the upper left to the lower right, the lower right to the upper left, the upper right to the lower left, and the lower left to the upper right corner of the two-dimensional data, and the sensor data is read from top to bottom and from left to right.

5. The mine safety intelligent monitoring method based on multi-source data fusion according to claim 4 is characterized in that: The specific process of S4 is as follows: S4-1. Input the multi-source feature sequence after spatial reconstruction into the state space model. The state space model maps the multi-source data through a hidden potential state, which is defined by four model parameters: (A, B, C, D). The formula for sequence-to-sequence conversion is described as: h′(t)=Ah(t)++Bx(t) y(t)=Ch(t)++Dx(t); Among them, x(t) represents the mapping input sequence, h′(t) represents the potential state at the current time t, y(t) represents the prediction sequence, A, B, C, D are model parameters, N represents the time state; S4-2, the state space model prediction consists of linear mapping and softmax function. The features extracted by the state space model are input into the linear mapping layer, the danger index at the current moment is output, and the danger index is input into the softmax activation function. If the obtained value is greater than 0.5, an alarm is activated; the formulas for adaptive average pooling and linear mapping are as follows: F i =liner(F i-1 ) F i+1 =softmax(F i ); In the formula, F i Represents the features of the current layer, liner represents the linear mapping layer, and softmax produces category probabilities.

6. A mine safety intelligent monitoring system based on multi-source data fusion, characterized in that: It includes data acquisition module, data processing module, intelligent analysis module and alarm feedback module; The data acquisition module integrates a vibration sensor, a concentration sensor, a sound sensor, and a humidity sensor, and is used to collect mining environmental parameters in real time, including vibration signals of equipment, gas concentration data signals, sound data signals, and air humidity data signals; The data processing module is used to pre-process the collected data, including denoising, normalization and format conversion, and is also used for data storage and management; The intelligent analysis module uses advanced deep learning algorithms to conduct in-depth analysis of the fused data to identify potential safety hazards, generate environmental status assessment reports, and predict possible dangerous situations; The alarm feedback module is used to issue an alarm when the intelligent analysis module detects an abnormal situation, and to display the real-time status of the current mining environment and specific safety hazard information to the operator through a visual interface, and to send the alarm information to the mobile terminal.

Images