Coal mine underground dust concentration monitoring method and system based on multi-modal data fusion

Abstract

The invention relates to the technical field of coal mine safety monitoring, in particular to an underground coal mine dust concentration monitoring method and system based on multi-modal data fusion, and the method comprises the steps: synchronously collecting dust concentration time sequence data, dust image data, sound wave signal data and environmental parameters through a multi-modal sensor array deployed in an underground coal mine; carrying out preprocessing and feature extraction on the collected data; predicting the decomposed high-frequency and low-frequency component signals by adopting a long short-term memory neural network and a grey Markov model; when the environment humidity is greater than 80%, carrying out light scattering compensation on the predicted value; inputting various predicted values into an improved D-S evidence theory fusion device, and outputting a fusion dust concentration monitoring value; and when the threshold value is exceeded or the temperature and humidity composite condition is reached, an acousto-optic alarm is triggered and a spraying dust-settling device is started. According to the method, the problems of low precision of a single sensor, multi-source data conflict, high-humidity environment measurement deviation and insufficient time sequence and fusion precision in underground coal mine dust concentration monitoring can be solved.

Description

Technical Field

[0001] This invention relates to the field of coal mine safety monitoring technology, and in particular to a method and system for monitoring underground dust concentration in coal mines based on multimodal data fusion. Background Technology

[0002] In the early days, monitoring of dust concentration in coal mines primarily relied on single dust concentration sensors for data acquisition and analysis. While these sensors could reflect dust concentration to some extent, they only provided limited time-series data and could not comprehensively capture the complex characteristics of dust in terms of spatial distribution and particle size, resulting in significant limitations in the monitoring results. Furthermore, early data processing methods were relatively simple, lacking effective data fusion and in-depth analysis techniques, making it difficult to extract valuable correlations from massive amounts of data to accurately assess the dynamic trends and potential risks of dust concentration.

[0003] With the increasing mechanization of coal mining and the improvement of safety standards, the limitations of single-sensor monitoring have become increasingly apparent, prompting the industry to explore multi-source data collaborative monitoring technologies. Various types of sensors, such as image sensors and acoustic sensors, are being introduced to monitor dust concentration in underground coal mines to enrich data sources and obtain more multi-dimensional dust-related information. Simultaneously, data processing technologies have also advanced, such as using filtering and noise reduction methods to perform preliminary processing of the collected data, and employing simple statistical analysis and prediction models to conduct preliminary trend analysis of dust concentration. However, these methods mostly involve independent processing and analysis of different types of data, and have not yet formed an effective multi-modal data fusion mechanism, failing to fully leverage the advantages of multi-source data.

[0004] Currently, the application of multimodal data fusion technology in coal mine dust monitoring is gradually increasing. Some solutions attempt to integrate multi-source data using traditional fusion algorithms (such as weighted average and classic DS evidence theory), but several problems remain: First, the heterogeneity of multimodal data leads to low fusion accuracy; second, traditional fusion rules are insufficient for handling highly conflicting data (such as outliers when sensors are obstructed by dust), easily resulting in distorted fusion results; third, the impact of high humidity in underground environments on dust particle size is not specifically addressed, as humidity causes dust agglomeration and increased particle size, thus affecting light scattering and sound wave propagation characteristics, and existing solutions lack dynamic compensation mechanisms for such environmental interference; fourth, the prediction of time-series data does not distinguish between high-frequency fluctuations and low-frequency trends, and a single prediction model cannot take both characteristics into account. These problems limit the effectiveness and value of underground coal mine dust concentration monitoring technology in practical applications.

[0005] Patent CN119246355A discloses a dust concentration monitoring system and method for mining applications. It employs a multi-mode sensor combining electrostatic induction and optics to detect dust concentration, and improves measurement stability in complex environments through signal amplification and filtering, temperature compensation, and adaptive filtering. However, this invention lacks layered processing and fusion of multi-modal data such as time series, images, and acoustic waves, and does not correct the concentration value at the physical mechanism level using a humidity-particle size correlation model.

[0006] Therefore, this invention discloses a method and system for monitoring underground dust concentration in coal mines based on multimodal data fusion. Summary of the Invention

[0007] The purpose of this invention is to address the problems in existing technologies, such as low accuracy of single sensors, difficulty in processing high-conflict data during multimodal data fusion, measurement deviation caused by dust particle size changes in high humidity environments, and insufficient accuracy of time-series prediction and multi-source fusion. This invention proposes a method and system for monitoring underground dust concentration in coal mines based on multimodal data fusion.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a method for monitoring underground dust concentration in coal mines based on multimodal data fusion, comprising the following steps:

[0009] Step S1: Simultaneously collect time-series data of dust concentration, dust image data, acoustic signal data, and environmental parameters through a multi-modal sensor array deployed underground in the coal mine;

[0010] Step S2: Perform variational mode decomposition on the dust concentration time series data to generate high-frequency component signals and low-frequency component signals; use a convolutional neural network to extract the spatial distribution features of dust from the dust image data and output the image dust concentration value; extract the dust particle size distribution features from the acoustic signal data through wavelet packet transform and output the acoustic dust concentration value.

[0011] Step S3: Use a long short-term memory neural network to predict the high-frequency component signal and obtain the time-series high-frequency prediction value; use a grey Markov model to predict the low-frequency component signal and obtain the time-series low-frequency prediction value; when the ambient humidity is >80%, based on multi-angle light scattering measurement data, the time-series high-frequency prediction value and the time-series low-frequency prediction value are corrected according to the humidity-particle size correlation model.

[0012] Step S4: Input the time series high-frequency prediction value, time series low-frequency prediction value, image dust concentration value, and acoustic dust concentration value into the improved DS evidence theory fusion rule, and output the fused dust concentration monitoring value.

[0013] Step S5: When the dust concentration monitoring value exceeds the preset safety threshold, or the temperature is >40℃ and the humidity is >85%, an audible and visual alarm is triggered and the spray dust suppression device is activated.

[0014] A coal mine underground dust concentration monitoring system based on multimodal data fusion includes:

[0015] Multimodal data acquisition module, data processing module, prediction and compensation module, multi-source fusion module, and early warning and control module;

[0016] The multimodal data acquisition module includes:

[0017] The multi-modal sensor array for mining meets the explosion-proof standards for underground coal mines and is deployed in tunneling faces, coal mining faces, and transport roadways.

[0018] The synchronous acquisition unit is used to synchronously acquire dust concentration time-series data, dust image data, acoustic signal data, and environmental parameters at a preset frequency;

[0019] The data transmission unit is used to transmit the collected data to the data processing module and supports local caching.

[0020] The data processing module includes:

[0021] The time-series data processing unit is used to decompose the dust concentration time-series data and output high-frequency component signals and low-frequency component signals.

[0022] The image data processing unit is used to extract features from dust image data and output the dust concentration value in the image.

[0023] The acoustic signal processing unit is used to extract features from acoustic signals and output acoustic dust concentration values.

[0024] The prediction and compensation module includes:

[0025] The modal prediction unit, deployed on the GPU acceleration module of the edge computing node, integrates a long short-term memory neural network computing component and a gray Markov computing component, and is used to predict high-frequency component signals and low-frequency component signals respectively, and output time-series high-frequency prediction values ​​and time-series low-frequency prediction values.

[0026] The humidity compensation unit is used to correct the predicted value based on the humidity-particle size correlation model and the principle of light scattering when the ambient humidity exceeds a preset threshold.

[0027] The multi-source fusion module includes:

[0028] The data adaptation unit is used to normalize the input multi-source data;

[0029] The fusion computing unit is used to fuse multi-source data according to the DS evidence theory fusion rules and output the fused dust concentration monitoring value.

[0030] The early warning control module includes:

[0031] The threshold judgment unit is used to judge the monitoring value of the fused dust concentration and environmental parameters;

[0032] An alarm unit is used to trigger an audible and visual alarm when the warning conditions are met.

[0033] The dust suppression control unit is used to activate the spray dust suppression device when the warning conditions are met;

[0034] The data recording unit is used to record warning-related data and generate logs.

[0035] The beneficial effects of the technical solution provided by this invention include at least the following:

[0036] This invention, by deploying a multimodal sensor array to synchronously collect data, can obtain multidimensional characteristics such as dust spatial distribution and particle size distribution, breaking through the monitoring limitations of a single sensor and providing comprehensive data support for accurate monitoring.

[0037] This invention separates high-frequency and low-frequency component signals through variational mode decomposition and uses long short-term memory neural networks and grey Markov models for targeted mode prediction. Combined with humidity-particle size correlation models and Mie scattering theory for humidity compensation, it can improve prediction accuracy and correct the interference of dust agglomeration on measurement in high humidity environments.

[0038] This invention, through an improved DS evidence theory fusion rule, can effectively handle highly conflicting data and avoid the distortion of results from traditional fusion algorithms.

[0039] This invention, through its mine-use explosion-proof sensor, dual-redundant data transmission, and local cache design, can adapt to the complex environment of underground coal mines, ensuring the continuity and security of data acquisition and transmission. Attached Figure Description

[0040] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a flowchart of a method for monitoring underground dust concentration in coal mines based on multimodal data fusion, provided in an embodiment of the present invention.

[0042] Figure 2 This is a system architecture diagram of a coal mine underground dust concentration monitoring system based on multimodal data fusion, provided in an embodiment of the present invention. Detailed Implementation

[0043] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the coal mine underground dust concentration monitoring method and system based on multimodal data fusion proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0044] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0045] The following examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0046] The following description, in conjunction with the accompanying drawings, details the specific scheme of the coal mine underground dust concentration monitoring method and system based on multimodal data fusion provided by this invention.

[0047] Please see Figure 1 The diagram illustrates a flowchart of a method for monitoring underground dust concentration in coal mines based on multimodal data fusion, according to an embodiment of the present invention. The method includes the following steps:

[0048] Step S1: Simultaneously collect time-series data of dust concentration, dust image data, acoustic signal data, and environmental parameters through a multi-modal sensor array deployed underground in the coal mine;

[0049] Step S1 further includes the following sub-steps:

[0050] S1-1, a multi-modal sensor array is deployed at preset intervals in the underground tunneling face, coal mining face and transportation roadway of the coal mine. The multi-modal sensor array includes a full dust sensor, a respiration dust sensor, an industrial camera, a sound wave sensor, a temperature and humidity composite sensor and a multi-angle light scattering sensor. The sensor housing protection level meets the mining explosion-proof standard.

[0051] S1-2, Initialize the multimodal sensor array, unify the timestamps and calibrate the accuracy, and set the appropriate sampling parameters according to the sensor type;

[0052] S1-3, activate the synchronous acquisition mechanism to collect dust concentration time-series data, dust image data, acoustic signal data and environmental parameters at a preset frequency;

[0053] S1-4 transmits the collected raw data to the edge computing node through dual redundant communication channels and enables a local caching mechanism to prevent transmission interruption.

[0054] It should be noted that the deployment of the multimodal sensor array is rationally planned based on the actual geographical environment and dust generation source distribution in the coal mine, ensuring comprehensive coverage of key areas such as the tunneling face, coal mining face, and transportation roadways to obtain representative monitoring data.

[0055] The preset interval can be flexibly determined based on factors such as the width and height of the tunnel and the distribution of dust sources. At the tunnel face, a sensor array can be deployed every 10-20 meters to ensure effective coverage of areas where dust is prone to accumulate.

[0056] Total dust sensors and inhalable dust sensors are based on capacitive or light scattering principles and are used to measure the total concentration of all dust and the concentration of inhalable dust in coal mines, respectively. Their measurement range is usually 0-1000 mg / m³, and the accuracy can reach ±10%.

[0057] The industrial camera uses a high-resolution, low-light image sensor with infrared illumination, which can clearly capture the distribution of dust in space. Its resolution is no less than 1920×1080 and its frame rate is no less than 25fps.

[0058] The operating frequency range of acoustic wave sensors is generally 20kHz - 100kHz. By transmitting and receiving sound waves, they analyze the changes in the propagation characteristics of sound waves in dusty media to obtain information on dust particle size distribution.

[0059] The temperature and humidity composite sensor uses capacitive or resistive sensing elements to monitor the ambient temperature and humidity in real time. The temperature measurement range is -20℃ to 60℃ with an accuracy of ±0.5℃, and the humidity measurement range is 0% - 100% RH with an accuracy of ±3% RH.

[0060] Multi-angle light scattering sensors utilize multiple photodetectors internally to receive scattered light signals from different angles (90° and 270°), enabling optical measurement of dust concentration and providing fundamental data for subsequent humidity compensation. The measured particle size range is typically 0.1μm to 10μm. All sensors are equipped with enclosures that meet mining explosion-proof standards to ensure safe operation in the explosive environments of underground coal mines.

[0061] The project collects time-series data on dust concentration, dust image data, acoustic signal data, and environmental parameters. The aim is to construct a basic dataset for dust concentration monitoring through multi-dimensional data complementarity. The time-series data on dust concentration reflects the dynamic changes in concentration, the dust image data reflects the spatial distribution, the acoustic signal data is associated with particle size characteristics, and the environmental parameters provide a basis for subsequent humidity compensation and early warning judgment.

[0062] Initializing the multimodal sensor array mainly involves setting initial parameters for each sensor, such as setting the sensor's range, resolution, and communication address; and establishing a communication connection to ensure a smooth communication link between the sensors and the data acquisition system.

[0063] The process of unifying timestamps is achieved by sending a synchronization clock signal to all sensors or by using a time synchronization protocol such as Network Time Protocol (NTP) to ensure that the data collected by each sensor is consistent in terms of timestamps, with the error controlled within the millisecond level.

[0064] Calibration accuracy is achieved by periodically calibrating the sensor using a standard dust concentration solution or calibration instrument, once every quarter or semi-annually, to ensure the accuracy of the measurement data.

[0065] After the synchronous acquisition mechanism is activated, each sensor works collaboratively at a preset frequency to synchronously acquire dust concentration time-series data, dust image data, acoustic signal data, and environmental parameters. For dust concentration time-series data, when it is necessary to capture rapid changes in dust concentration, the acquisition frequency can be set to 1-10 times per second. For dust image data, considering the large amount of image data and the relatively slow diffusion speed of dust in space, the acquisition frequency can be set to 0.1-1 times per second. The acquisition frequency of acoustic signal data is determined according to the operating frequency of the acoustic sensor and the dust particle size measurement requirements, usually in the range of kHz to MHz. The acquisition frequency of environmental parameter data is relatively low, generally 1-10 times per minute is sufficient to meet the monitoring requirements.

[0066] The collected raw data is transmitted through dual redundant communication channels, which combine industrial Ethernet and a wireless mesh network to ensure reliable data transmission. Industrial Ethernet features high transmission speed and stability, making it suitable for rapid transmission of large amounts of data; the wireless mesh network provides flexible network coverage in complex underground environments where cabling is difficult, serving as a supplement and backup to the wired network.

[0067] During data transmission, a local caching mechanism is enabled. When the communication channel is interrupted due to a fault, the collected data can be automatically stored in the storage device of the sensor or a nearby edge computing node. After communication is restored, the cached data is fully transmitted to the edge computing node, thus ensuring data integrity. The storage capacity of the local caching mechanism should be reasonably configured according to the data acquisition frequency and the possible duration of the transmission interruption, generally storing 12 hours of data.

[0068] Step S2: Perform variational mode decomposition on the dust concentration time series data to generate high-frequency component signals and low-frequency component signals; use a convolutional neural network to extract the spatial distribution features of dust from the dust image data and output the image dust concentration value; extract the dust particle size distribution features from the acoustic signal data through wavelet packet transform and output the acoustic dust concentration value.

[0069] Step S2 further includes the following sub-steps:

[0070] S2-1, Processing the dust concentration time series data, including preprocessing and decomposing the total dust concentration time series data and the respirable dust concentration time series data respectively, using moving average filtering to remove impulse noise, and using wavelet threshold noise reduction to suppress high-frequency interference;

[0071] S2-2, variational mode decomposition is used to decompose the preprocessed dust concentration time series data. The number of decomposition layers and the penalty coefficient are set. The high-frequency component signal and the low-frequency component signal are separated by iterative solution through the alternating direction multiplier method. The high-frequency component signal corresponds to the dust concentration fluctuation characteristics, and the low-frequency component signal corresponds to the dust concentration trend characteristics.

[0072] S2-3, preprocessing the dust image data, including median filtering for noise reduction, histogram equalization to enhance contrast, and using the GrabCut algorithm to segment the dust region;

[0073] S2-4: Input the preprocessed dust image data into a convolutional neural network, output the dust concentration regression value through regression analysis of a fully connected layer, optimize the nonlinear feature extraction using the ReLU activation function, and output the dust concentration regression value as the image dust concentration value.

[0074] S2-5 performs wavelet packet transform on the acoustic signal data, extracts frequency band features, and outputs acoustic dust concentration values ​​through regression model mapping.

[0075] It should be noted that the preprocessing uses a moving average filtering method to remove impulse noise. The length of the moving window can be determined based on the data sampling frequency and the characteristics of dust concentration variation, and is generally between 5 and 15 sampling points. For example, for data collected 10 times per second, the moving window length can be set to 10 sampling points to smooth the data and remove sudden impulse interference.

[0076] High-frequency interference is suppressed by using wavelet threshold denoising. Appropriate wavelet basis functions (such as db4 or sym5) and threshold calculation methods (such as fixed threshold or adaptive threshold) are selected to decompose and reconstruct the data, effectively reducing the interference of high-frequency noise on dust concentration measurement, and making the data more smoothly and accurately reflect the actual trend of dust concentration change.

[0077] Variational mode decomposition (VMD) is used to decompose the preprocessed dust concentration time series data. The number of decomposition layers is usually determined based on the complexity of the dust concentration signal; for underground coal mine dust concentration signals, 3-6 layers are typically used. The penalty coefficient is used to control the sparsity during the decomposition process and can be determined between 100 and 1000 based on empirical formulas or cross-validation. Iterative solutions using the alternating direction multiplier method are then employed to separate the high-frequency and low-frequency components.

[0078] The high-frequency component signal mainly reflects the short-term fluctuations of dust concentration and the changing characteristics of fine particulate matter, while the low-frequency component signal reflects the long-term trend and overall change law of dust concentration, providing a basis for subsequent submodal dynamic prediction.

[0079] Median filtering is used to denoise dust image data. By selecting an appropriate filter window size, such as a 3×3 or 5×5 window, the image data is scanned pixel by pixel, and the median value within the window is used to replace the center pixel value, effectively removing random noise such as salt-and-pepper noise from the image.

[0080] Histogram equalization enhances image contrast by statistically analyzing and adjusting the grayscale values ​​of image pixels, making the boundary between dusty areas and the background clearer.

[0081] The GrabCut algorithm is used to segment the dust region. GrabCut is a graph theory-based image segmentation method that constructs a Gaussian mixture model for the foreground and background and uses a graph cut algorithm to segment the image into dust and non-dust regions, accurately extracting the spatial distribution features of dust and providing more accurate image information for subsequent image dust concentration inversion.

[0082] A convolutional neural network (CNN) consists of multiple convolutional layers, pooling layers, and fully connected layers. In the convolutional layers, convolutional kernels of different sizes (such as 3×3 or 5×5) are used to perform convolution operations on the image, extracting features such as edges and textures. After multiple convolution and pooling operations, the extracted features are input into the fully connected layer for regression analysis, outputting a dust concentration regression value.

[0083] The ReLU activation function can effectively solve the gradient vanishing problem, accelerate the training speed of the network, improve the network's ability to fit the nonlinear features of complex dust in the image, and finally output the dust concentration regression value as the image dust concentration value, which can more accurately reflect the actual dust concentration level in the image.

[0084] Wavelet packet transform is performed on the acoustic signal data. By selecting appropriate wavelet basis functions (symlet or coiflet) and the number of decomposition levels (typically 3-5 levels), the acoustic signal is decomposed into sub-band signals of different frequency bands. Each sub-band signal corresponds to a different frequency range. By calculating the energy characteristics or variance characteristics of each sub-band signal, frequency band features that characterize the dust particle size distribution are extracted. Then, a regression model (linear regression or support vector regression model) is constructed, using the extracted frequency band features as input variables and the acoustic dust concentration value as the output variable. The trained regression model maps the relationship between the two, thereby outputting the acoustic dust concentration value.

[0085] Step S3: Use a long short-term memory neural network to predict high-frequency component signals and obtain time-series high-frequency predicted values; use a grey Markov model to predict low-frequency component signals and obtain time-series low-frequency predicted values; when the ambient humidity is >80%, the predicted values ​​are corrected based on multi-angle light scattering measurement data and according to the humidity-particle size correlation model.

[0086] Step S3 further includes the following sub-steps:

[0087] S3-1 inputs the high-frequency component signal into the long short-term memory neural network, dynamically adjusts the temporal feature weights through the hidden layer neurons, and outputs the temporal high-frequency prediction value.

[0088] S3-2, establish a grey prediction model GM (1,1) for low-frequency components, generate an initial prediction sequence, divide the prediction residuals into a preset number of states, construct a Markov state transition matrix, correct the GM (1,1) prediction value, and output the time series low-frequency prediction value.

[0089] S3-3 monitors ambient humidity in real time using a humidity sensor. When the humidity exceeds 80%, it activates multi-angle light scattering compensation and calls the humidity-particle size correlation model to correct dust particle size parameters. The formula for the humidity-particle size correlation model is as follows:

[0090]

[0091] Where D represents the wet particle size; Indicates the dry particle size; h indicates the percentage of humidity. α represents the hygroscopic expansion coefficient of coal dust; α represents the hygroscopic nonlinear coefficient.

[0092] S3-4, based on the corrected dust particle size parameters, the particle size compensation factor is calculated using Mie scattering theory. Combined with the humidity correlation factor and the intensity of dual-angle scattered light, humidity compensation is applied to the time-series high-frequency and low-frequency predicted values. The compensation formula is expressed as:

[0093]

[0094] in, This represents the predicted value to be compensated. Indicates the particle size compensation factor; Indicates the humidity correlation factor; This represents the light intensity received at a 90° scattering angle; The value represents the light intensity received at a scattering angle of 270°; C represents the dust concentration value after compensation.

[0095] It should be noted that the Long Short-Term Memory (LSTM) neural network can be configured with 2-3 hidden layers, each containing 50-100 neurons. The training process uses the Adam optimization algorithm, with a learning rate typically set to 0.001-0.01. Through multiple iterations of training on the training set, the network can learn the short-term fluctuation characteristics of dust concentration in high-frequency component signals, and then output time-series high-frequency prediction values. The temporal resolution of the prediction values ​​matches the sampling frequency of the original data. For example, if the original data is sampled 10 times per second, the prediction values ​​can also provide 10 prediction points per second, promptly reflecting the rapid changes in dust concentration.

[0096] The grey prediction model GM (1,1) is based on time series data of low-frequency component signals. It generates an initial prediction sequence by accumulating and generating a sequence and constructing a grey differential equation. When the prediction residual is divided into a preset number of states, it can be divided into 3 to 5 states according to the distribution of the residual.

[0097] The Markov state transition matrix is ​​an n×n matrix (where n is the number of states). The elements in the matrix represent the probability of the residual transitioning from one state to another; these probabilities are obtained by statistically analyzing historical residual data. When using this matrix to correct the GM(1,1) prediction, the matrix predicts the residual state at the next time step based on the current residual state, thereby adjusting the initial prediction sequence and outputting a low-frequency time-series prediction value to account for the long-term trend of dust concentration. The time span of the prediction values ​​can be set according to actual needs, such as providing a prediction value every minute or hour to reflect the overall trend of dust concentration changes.

[0098] When the ambient humidity exceeds 80%, the activated multi-angle light scattering compensation mechanism is based on the light signals received by the multi-angle light scattering sensor at scattering angles of 90° and 270°.

[0099] In the humidity-particle size correlation model, the hygroscopic expansion coefficient β of coal dust is calibrated by the hygroscopic test in GB / T 29146. For anthracite, β=0.25, and for lignite, β=0.82. The hygroscopic nonlinear coefficient α is usually less than 0.1.

[0100] Mie scattering theory is a physical model describing the interaction between electromagnetic waves and spherical particles. Its calculation process requires consideration of parameters such as particle size, relative refractive index, and wavelength of incident light. In the calculation process, the relative refractive index of the particles can be determined based on the physical properties of coal dust, while the wavelength of the incident light is selected as a suitable fixed value or integrated calculation based on the operating wavelength range of the light scattering sensor (400-700nm).

[0101] The humidity correlation factor is derived by analyzing the effect of humidity on the intensity of scattered light. It typically ranges from 0 to 1, reflecting the comprehensive impact of humidity on light scattering. The selection of 90° and 270° scattering angles is based on the sensitivity and complementarity of the light scattering signal to changes in dust particle size.

[0102] Step S4: Input the time series high-frequency prediction value, time series low-frequency prediction value, image dust concentration value, and acoustic dust concentration value into the improved DS evidence theory fusion rule, and output the fused dust concentration monitoring value.

[0103] Step S4 further includes the following sub-steps:

[0104] S4-1, normalize the time series high-frequency prediction value, time series low-frequency prediction value, image dust concentration value and acoustic dust concentration value, and unify the dimensions to mg / m³; define the identification framework Θ = {dust concentration is normal, dust concentration exceeds the limit}, and set the limit threshold to 10mg / m³.

[0105] S4-2, Construct the basic probability allocation function, set the initial weights based on the historical accuracy of each data source, and calculate the inter-evidence conflict coefficient using the Jousselme distance formula;

[0106] S4-3, using an improved DS evidence theory fusion rule, weighted average redistribution of conflicting evidence, and iterative fusion according to a preset number of times to obtain the final basic probability allocation, the improved DS evidence theory fusion rule formula is expressed as follows;

[0107]

[0108] in, Represents the improved basic probability assignment; (1- )·m(A) represents the portion of the original evidence retained, with a weight of (1-γ). The larger γ is, the less original information is retained; the smaller γ is, the closer it is to the traditional Dempster rule. Indicates the conflict redistribution portion. Let w represent the conflict coefficient and w represent the weight of the credibility of the evidence. This represents the average support of all evidence for proposition A;

[0109] S4-4 Calculate the trust function and likelihood function of the fusion result, and take the expected value of the trust function as the fusion dust concentration monitoring value.

[0110] It should be noted that when normalizing the time-series high-frequency predicted values, time-series low-frequency predicted values, image dust concentration values, and acoustic dust concentration values, a linear normalization method is used to uniformly map each data value to the range of 0 - 1, and then convert it to the mg / m³ dimension by multiplying by the corresponding dimensional coefficient. For example, for high-frequency predicted values, if their original range is 0-500 μg / m³, then after normalization, multiplying by 0.001 yields the corresponding mg / m³ value.

[0111] The identification framework Θ = {normal dust concentration, dust concentration exceeding the limit} is set based on the safety standards for dust concentration in underground coal mines and the needs of actual application scenarios. The threshold for exceeding the limit is set at 10mg / m³, referring to the limit requirements for dust concentration in the relevant industry standard (GBZ2.1-2019). When the integrated dust concentration monitoring value exceeds this threshold, it is judged as a dust concentration exceeding the limit state, and corresponding early warning measures need to be taken.

[0112] When constructing the basic probability allocation function, the historical accuracy of each data source is derived from the statistical analysis of a large amount of test data from various sensors and prediction models in the early stages.

[0113] When calculating the conflict coefficient between pieces of evidence using the Jousselme distance formula, it is possible to effectively measure the degree of conflict between different pieces of evidence. The parameter values ​​in the formula are generally between 0 and 1.

[0114] When using the improved DS evidence theory fusion rule, the value of γ generally ranges from 0.7 to 0.95, and can be adjusted according to the importance of preserving the original evidence and redistributing conflicts in the actual application scenario. The evidence credibility weight w is determined by comprehensively considering factors such as the stability and reliability of each data source. For example, the weight of time series high-frequency prediction values ​​is 0.3-0.4, the weight of time series low-frequency prediction values ​​is 0.2-0.3, the weight of image dust concentration values ​​is 0.1-0.2, and the weight of acoustic dust concentration values ​​is 0.1-0.2. The average support of all evidence for proposition A is obtained by calculating the average of the basic probability distributions of each piece of evidence for proposition A. When iterating fusion according to a preset number of times, the preset number of times is generally between 10 and 100. Each iteration updates the basic probability distribution according to the improved DS evidence theory fusion rule until the set number of iterations is reached or the convergence condition is met, thereby obtaining the final basic probability distribution and achieving effective processing and fusion of conflicting evidence.

[0115] The trust function represents the degree of confidence in the proposition being true, while the likelihood function represents the degree of support for the evidence if the proposition is false. Using the expected value of the trust function as the fused dust concentration monitoring value more directly reflects the degree of confidence in the dust concentration status. The expected value is calculated by weighting the values ​​of the trust function across all possible states, with the weights being the probabilities of each state. This yields a comprehensive dust concentration monitoring value that represents the fused result and more accurately reflects the actual dust concentration situation in underground coal mines.

[0116] Step S5: When the dust concentration monitoring value exceeds the preset safety threshold, or the temperature is >40℃ and the humidity is >85%, an audible and visual alarm is triggered and the spray dust suppression device is activated.

[0117] Step S5 further includes the following sub-steps:

[0118] S5-1 compares the integrated dust concentration monitoring value with the preset safety thresholds of the tunneling face, coal mining face and transportation roadway, and simultaneously determines the composite conditions of ambient temperature > 40℃ and humidity > 85%.

[0119] S5-2, when any of the conditions of the composite conditions are met, the sound and light alarm device is triggered, and a 110dB buzzing sound and red warning light that conform to the standard of mine sound and light alarm are emitted. The sound and light alarm lasts for 30 seconds, and the spray dust suppression continues until the fusion dust concentration monitoring value is lower than the safety threshold.

[0120] S5-3 sends an early warning signal to the central control room via the underground industrial Ethernet, and at the same time starts the spray dust suppression device to perform dust suppression operation according to the appropriate pressure and coverage area;

[0121] S5-4 records multimodal data for one hour before and after the alarm time, generates an event log, and uploads it to the central control room.

[0122] It should be noted that the safety threshold for dust concentration at the tunneling face can be set slightly lower than that at the coal face, depending on the intensity of the dust source and ventilation conditions, in order to more strictly monitor the dust risk during the tunneling process; the thresholds for ambient temperature and humidity are determined based on the normal operating environment requirements of the underground equipment and the moisture absorption characteristics of the dust.

[0123] The combined conditions of temperature > 40℃ and humidity > 85% are usually critical states in which underground dust is prone to agglomeration, exacerbating equipment failure and affecting the health of workers.

[0124] When the audible and visual alarm device is triggered, the 110dB buzzer sound emitted is set according to the sound intensity requirements in the standard for mine audible and visual alarm devices, and can be clearly heard in noisy underground environments; the red warning light is selected based on the sensitivity of the human eye to red light, and its flashing frequency can be set to 1-2 times per second to enhance the warning effect.

[0125] The 30-second duration of the audible and visual alarm is intended to ensure that operators receive sufficient attention while avoiding information redundancy and interference caused by prolonged alarms.

[0126] The starting pressure of the spray dust suppression device is generally set between 0.2 and 0.5 MPa to ensure that the spray can effectively cover the dusty area. The dust suppression operation will continue until the dust concentration monitoring value is lower than the safety threshold. This process may take anywhere from a few minutes to tens of minutes, depending on the degree to which the dust concentration exceeds the limit and the ventilation conditions underground.

[0127] The underground industrial Ethernet network employs a redundant network topology (such as a ring or dual-star configuration) to ensure the reliability of early warning signal transmission. The early warning signal includes the specific location of dust concentration exceeding limits, the exceeding value, and information on environmental parameters.

[0128] The dust suppression spray device is activated by sending control commands to it. The device will carry out dust suppression operations according to the preset spray pattern (such as fan spray, mist spray, etc.). The coverage area of ​​the spray is pre-determined based on the cross-sectional size of the tunnel and the dust diffusion characteristics, and it can generally cover more than 80% of the tunnel cross-section.

[0129] Multimodal data was recorded one hour before and after the alarm. To provide sufficient data support for accident investigation and cause analysis, the data recording frequency was consistent with the original data acquisition frequency to ensure data integrity and continuity.

[0130] The event log includes information such as the time and location of the alarm, dust concentration change curves, environmental parameter change curves, and the measures taken. It is uploaded to the central control room in a standard data format (such as XML or JSON). The central control room is equipped with data storage and analysis software that can receive and process this data in real time, providing management personnel with decision support and a basis for post-event traceability.

[0131] Please see Figure 2 This diagram illustrates a system architecture of a coal mine underground dust concentration monitoring system based on multimodal data fusion, according to an embodiment of the present invention. The coal mine underground dust concentration monitoring system based on multimodal data fusion includes:

[0132] Multimodal data acquisition module, data processing module, prediction and compensation module, multi-source fusion module, and early warning and control module;

[0133] The multimodal data acquisition module includes:

[0134] The multi-modal sensor array for mining meets the explosion-proof standards for underground coal mines and is deployed in tunneling faces, coal mining faces, and transport roadways.

[0135] The synchronous acquisition unit is used to synchronously acquire dust concentration time-series data, dust image data, acoustic signal data, and environmental parameters at a preset frequency;

[0136] The data transmission unit is used to transmit the collected data to the data processing module and supports local caching.

[0137] The data processing module includes:

[0138] The time-series data processing unit is used to decompose the dust concentration time-series data and output high-frequency component signals and low-frequency component signals.

[0139] The image data processing unit is used to extract features from dust image data and output the dust concentration value in the image.

[0140] The acoustic signal processing unit is used to extract features from acoustic signals and output acoustic dust concentration values.

[0141] The prediction and compensation module includes:

[0142] The modal prediction unit, deployed on the GPU acceleration module of the edge computing node, integrates a long short-term memory neural network computing component and a gray Markov computing component, and is used to predict high-frequency component signals and low-frequency component signals respectively, and output time-series high-frequency prediction values ​​and time-series low-frequency prediction values.

[0143] The humidity compensation unit is used to correct the predicted value based on the humidity-particle size correlation model and the principle of light scattering when the ambient humidity exceeds a preset threshold.

[0144] The multi-source fusion module includes:

[0145] The data adaptation unit is used to normalize the input multi-source data;

[0146] The fusion computing unit is used to fuse multi-source data according to the DS evidence theory fusion rules and output the fused dust concentration monitoring value.

[0147] The early warning and control module includes:

[0148] The threshold judgment unit is used to judge the monitoring value of the fused dust concentration and environmental parameters;

[0149] An alarm unit is used to trigger an audible and visual alarm when the warning conditions are met.

[0150] The dust suppression control unit is used to activate the spray dust suppression device when the warning conditions are met;

[0151] The data recording unit is used to record warning-related data and generate logs.

[0152] In this way, a method and system for monitoring underground dust concentration in coal mines based on multimodal data fusion can be realized.

[0153] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A coal mine underground dust concentration monitoring method based on multi-modal data fusion, characterized in that, The method comprises: Step S1, synchronously collecting dust concentration time series data, dust image data, sound wave signal data and environmental parameters through a multi-modal sensor array deployed underground in a coal mine; Step S2, performing variational mode decomposition on the dust concentration time series data to generate high-frequency component signals and low-frequency component signals; using a convolutional neural network to extract dust spatial distribution features from the dust image data and output image dust concentration values; and extracting dust particle size distribution features from the sound wave signal data through wavelet packet transform and outputting acoustic dust concentration values; Step S3, using a long short-term memory neural network to predict the high-frequency component signals to obtain time series high-frequency prediction values; using a grey Markov model to predict the low-frequency component signals to obtain time series low-frequency prediction values; when the environmental humidity is greater than 80%, correcting the time series high-frequency prediction values and the time series low-frequency prediction values based on multi-angle light scattering measurement data according to a humidity-particle size correlation model; Step S4, inputting the time series high-frequency prediction values, the time series low-frequency prediction values, the image dust concentration values and the acoustic dust concentration values into an improved D-S evidence theory fusion rule to output a fused dust concentration monitoring value; Step S5, when the fused dust concentration monitoring value exceeds a preset safety threshold or the temperature is greater than 40 DEG C and the humidity is greater than 85%, triggering an audible and light alarm and starting a spray dust-settling device.

2. The coal mine underground dust concentration monitoring method based on multi-modal data fusion according to claim 1, characterized in that: wherein in step S1, the following sub-steps are further included: S1-1, arranging the multi-modal sensor array at a preset interval on a coal mine underground heading face, a coal mining face and a transportation roadway, the multi-modal sensor array comprising a total dust sensor, a respirable dust sensor, an industrial camera, a sound wave sensor, a temperature and humidity composite sensor and a multi-angle light scattering sensor, the sensor shell protection grade meeting the mine explosion-proof standard; S1-2, initializing the multi-modal sensor array, unifying the time stamp and calibrating the accuracy, and setting the adaptive sampling parameters according to the sensor type; S1-3, starting the synchronous collection mechanism to collect the dust concentration time series data, the dust image data, the sound wave signal data and the environmental parameters at a preset frequency; S1-4, transmitting the collected original data to an edge computing node through a double-redundancy communication channel, and enabling a local caching mechanism to prevent transmission interruption.

3. The coal mine underground dust concentration monitoring method based on multi-modal data fusion according to claim 1, characterized in that: wherein in step S2, the following sub-steps are further included: S2-1, processing the dust concentration time series data, the processing including pre-processing and decomposing the total dust concentration time series data and the respirable dust concentration time series data respectively, removing impulse noise through moving average filtering, and suppressing high-frequency interference through wavelet threshold denoising; S2-2, decomposing the pre-processed dust concentration time series data using variational mode decomposition, setting the decomposition layer number and the penalty coefficient, and solving iteratively through an alternating direction multiplier method to separate the high-frequency component signals and the low-frequency component signals, the high-frequency component signals corresponding to dust concentration fluctuation characteristics, and the low-frequency component signals corresponding to dust concentration trend characteristics; S2-3, pre-processing the dust image data, including median filter denoising, histogram equalization to enhance contrast, and using GrabCut algorithm to segment the dust area; S2-4, inputting the pre-processed dust image data into a convolutional neural network, outputting a dust concentration regression value through full connection layer regression analysis, using a ReLU activation function to optimize nonlinear feature extraction, and outputting the dust concentration regression value as the image dust concentration value; S2-5, performing wavelet packet transform on the acoustic signal data, extracting frequency band features, and mapping output acoustic dust concentration values through a regression model.

4. The coal mine underground dust concentration monitoring method based on multi-modal data fusion according to claim 1, characterized in that: wherein in step S3, the following sub-steps are further included: S3-1, inputting the high-frequency component signal into a long short-term memory neural network, dynamically adjusting the time sequence feature weight through the hidden layer neurons, and outputting the time sequence high-frequency prediction value; S3-2, establishing a gray prediction model GM (1, 1) for the low-frequency component, generating an initial prediction sequence, dividing the prediction residual into a preset number of states, constructing a Markov state transition matrix, correcting the GM (1, 1) prediction value, and outputting the time sequence low-frequency prediction value; S3-3, real-time monitoring the environmental humidity through a humidity sensor, when the humidity > 80%, based on multi-angle light scattering measurement data, calling a humidity-particle size correlation model to correct the dust particle size parameters, and the formula of the humidity-particle size correlation model is represented as: ； wherein D represents the wet-state particle size; wherein D represents the dry-state particle size; and h represents the humidity percentage; wherein D represents the dry-state particle size; and h represents the humidity percentage; and wherein α represents the hygroscopic non-linear coefficient; S3-4, based on the corrected dust particle size parameters, calculating the particle size compensation factor through the Mie scattering theory, combining the humidity correlation factor and the dual-angle scattering light intensity, and performing humidity compensation on the time sequence high-frequency prediction value and the time sequence low-frequency prediction value, and the formula of the humidity compensation is represented as: ； wherein, represents a predicted value to be compensated; represents a particle size compensation factor; represents a humidity correlation factor; represents light intensity received at a 90° scattering angle; represents light intensity received at a 270° scattering angle; and C represents a compensated dust concentration value.

5. The coal mine underground dust concentration monitoring method based on multi-modal data fusion according to claim 1, characterized in that: wherein in step S4, the following sub-steps are further included: S4-1, normalizing the time sequence high-frequency prediction value, the time sequence low-frequency prediction value, the image dust concentration value, and the acoustic dust concentration value, and unifying the dimension to mg / m³; defining the recognition framework Θ = {normal dust concentration, dust concentration over limit}, and setting the over limit threshold to 10 mg / m³; S4-2, constructing a basic probability assignment function, setting the initial weight based on the historical accuracy of each data source, and calculating the conflict coefficient between evidences through the Jousselme distance formula; S4-3, using the improved D-S evidence theory fusion rule to weight and average the conflict evidence and redistribute it, and obtaining the final basic probability assignment through iterative fusion for a preset number of times, and the formula of the improved D-S evidence theory fusion rule is represented as: ； wherein, represents the improved basic probability assignment; (1- )·m(A) represents the part of the original evidence that is preserved with a weight of (1-γ), the larger γ is, the less original information is preserved, and the smaller γ is, the closer it is to the traditional Dempster rule; represents the conflict redistribution part, represents the conflict coefficient, w represents the evidence credibility weight, represents the average support of all evidence to the proposition A; S4-4, calculating the belief function and the likelihood function of the fusion result, and taking the expected value of the belief function as the fusion dust concentration monitoring value.

6. The coal mine underground dust concentration monitoring method based on multi-modal data fusion according to claim 1, characterized in that: wherein in step S5, the following sub-steps are further included: S5-1, compare the fusion dust concentration monitoring value with the preset safety threshold of the heading face, coal face and transportation roadway, and judge the compound condition that the environmental temperature > 40℃ and the humidity > 85%; S5-2, when any condition of the compound condition is met, trigger the sound and light alarm device to emit 110dB of buzzer sound and red warning light in accordance with the mine sound and light alarm standard, and the sound and light alarm lasts for 30 seconds, and the spray dust reduction lasts until the fusion dust concentration monitoring value is lower than the safety threshold; S5-3, send a pre-warning signal to the central control room through the underground industrial Ethernet, and start the spray dust reduction device to perform dust reduction operation according to the adaptive pressure and coverage range; S5-4, record the multi-modal data within one hour before and after the alarm time, generate an event log and upload it to the central control room.

7. The coal mine underground dust concentration monitoring system based on multi-modal data fusion, used to realize the coal mine underground dust concentration monitoring method based on multi-modal data fusion in any one of claims 1-6, characterized in that, It comprises: a multi-modal data acquisition module, a data processing module, a prediction and compensation module, a multi-source fusion module and a pre-warning control module; The multi-modal data acquisition module comprises: a mine multi-modal sensor array, which meets the coal mine underground explosion-proof standard and is arranged at the heading face, coal face and transportation roadway; a synchronous acquisition unit for synchronously acquiring dust concentration time series data, dust image data, sound signal data and environmental parameters at a preset frequency; a data transmission unit for transmitting the acquired data to the data processing module and supporting local caching; The data processing module comprises: a time series data processing unit for decomposing the dust concentration time series data and outputting high-frequency component signals and low-frequency component signals; an image data processing unit for extracting features from the dust image data and outputting image dust concentration values; a sound signal processing unit for extracting features from the sound signal and outputting acoustic dust concentration values; The prediction and compensation module comprises: a multi-modal prediction unit, which is deployed on the GPU acceleration module of the edge computing node, integrates long and short-term memory neural network computing components and gray Markov computing components, and is used for predicting the high-frequency component signals and low-frequency component signals respectively, and outputting time series high-frequency prediction values and time series low-frequency prediction values; a humidity compensation unit for correcting the prediction values based on the humidity-particle size correlation model and the light scattering principle when the environmental humidity exceeds the preset threshold; The multi-source fusion module comprises: a data adaptation unit for normalizing the input multi-source data; a fusion calculation unit for fusing the multi-source data by D-S evidence theory fusion rule and outputting the fusion dust concentration monitoring value; The pre-warning control module comprises: a threshold judgment unit for judging the fusion dust concentration monitoring value and the environmental parameters; an alarm unit for triggering the sound and light alarm when the pre-warning condition is met; a dust reduction control unit for starting the spray dust reduction device when the pre-warning condition is met; a data recording unit for recording the pre-warning related data and generating a log.

Images