Fire monitoring and early warning system for coal mine belt conveyor

Abstract

The invention relates to a fire monitoring and early warning system for a coal mine belt conveyor, and belongs to the technical field of coal mine safety. The system comprises a fire hazard grading early warning system, an acquisition subsystem and a watering subsystem. The acquisition subsystem comprises an AI video subsystem, a voiceprint recognition subsystem, an inspection robot subsystem and a distributed optical fiber temperature measurement subsystem which are respectively used for acquiring the video image characteristics of the belt conveyor, the voiceprint characteristics of the belt conveyor, the concentration of symbolic gas and smoke in a mine and the temperature of the belt conveyor; the fire hazard grading early warning system carries out fusion analysis on symbolic gas concentration, smoke concentration, temperature, video image features and voiceprint features acquired by the acquisition subsystem through a multi-mode information fusion analysis model, and realizes fire hazard and fire hazard grading early warning in a coal mine belt conveyor area. Comprising equipment abnormity monitoring alarm, single index early warning and multi-parameter fusion early warning.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal mine safety, and relates to a fire monitoring and early warning system for coal mine belt conveyors. Background Art

[0002] As a key equipment for underground raw coal transportation, the fire hazards of coal mine belt conveyors have become major risk sources restricting the safe production of coal mines. Fire accidents in the coal mine transportation system account for external fires, and the fires caused by belt friction account for a very high proportion. Such fires have typical triple-coupling disaster-causing characteristics: mechanical friction generates a high-temperature heat source, coal dust deposition provides combustibles, and the ventilation system forms an oxygen-supplying environment. Especially in long-distance (>500m) and large-inclination (>15°) transportation roadways, toxic gases such as CO and H2S generated by the fire can spread along the ventilation network to the mining and excavation working faces in a short time, forming a "chimney effect" and leading to secondary disasters.

[0003] The current fire monitoring system mainly has the following technical bottlenecks: 1) Smoke detectors have a high false alarm rate in the underground high-dust (concentration > 10mg / m 3 ) environment, and there is a response delay due to the influence of ventilation disturbance; 2) The early warning mechanism based on the CO concentration threshold (usually set at 24ppm) of the gas monitoring method is difficult to distinguish normal production emissions from initial fire conditions, and there are discrimination blind spots in actual applications; 3) Point-type temperature sensors are limited by the installation spacing (usually 30 - 50m) and contact measurement characteristics, and cannot capture the transient temperature rise at the dynamic friction point between the roller and the conveyor belt; 4) Although the distributed optical fiber temperature measurement technology can achieve continuous monitoring, it is difficult to monitor the early temperature rise of the fire due to the influence of air flow; 5) The visible light / infrared-based image recognition system has a high misjudgment rate in a dust environment with a visibility < 10m, and cannot penetrate the coal dust accumulation area at the bottom of the conveyor belt for effective monitoring.

[0004] The data acquisition frequency of existing single-sensing technologies (usually below 1Hz) is difficult to capture the transient characteristics of friction sparks, and the data island phenomenon between systems results in an insufficient comprehensive early warning accuracy. Especially in the variable frequency speed regulation (0.5 - 50Hz) working condition, the dynamic friction position of the mechanical transmission system is random (the friction fire source appears at non-fixed monitoring points), and there are serious blind spots in the traditional fixed-point monitoring method. These technical defects have become prominent bottlenecks restricting the upgrade of the coal mine fire prevention and control system, and there is an urgent need to develop a new monitoring technology integrating multi-source information.

[0005] Currently, the main monitoring methods for coal mine belt conveyor fires include smoke detection method, gas monitoring method, point-type temperature detection method, distributed optical fiber temperature measurement method, visible light / infrared image recognition method, etc. These single detection means have disadvantages such as large errors in quantitative acquisition, discrimination, and early warning, and cannot meet the requirements of coal mine belt conveyor fire monitoring and early warning.

[0006] At present, the functions of domestic coal mine fire monitoring systems are not unified. Although separate monitoring of some indicators such as the characteristic gases, temperature, smoke, and flame of belt conveyor fires has been achieved, it is not possible to effectively, comprehensively, and uniformly monitor the factors related to belt conveyor fires in coal mines. False alarms often occur in fire warnings. At the same time, intelligent monitoring technologies and equipment such as AI video images and voiceprints are lacking.

[0007] Current fire monitoring systems often monitor flames, characteristic gases, smoke, etc. after a fire occurs, but do not monitor fire hazards such as equipment abnormalities and personnel violations, and it is impossible to detect fire hazards and give warnings in time to avoid fires.

[0008] Existing belt conveyor fire monitoring only installs or distributes fiber optic temperature measurement at fixed positions such as the head, tail, or transfer points of the belt conveyor. The monitoring range can only cover points and lines, there are monitoring blind spots, and there is a lack of a fire monitoring and warning system that covers the entire belt conveyor in terms of points, lines, surfaces, and volumes. Summary of the Invention

[0009] In view of this, the purpose of the present invention is to provide a coal mine belt conveyor fire monitoring and warning system, which combines fixed monitoring and robot mobile monitoring. Through the monitoring, analysis, and identification of multi-parameter information such as equipment abnormalities, smoke, carbon monoxide concentration, methane concentration, oxygen concentration, temperature, video images, and sounds, a fire monitoring and warning system is formed to achieve real-time monitoring and positioning, hierarchical warning, coordinated linkage, and automatic sprinkler extinguishing of fires in areas such as belt conveyors and power supply lines in terms of points, lines, surfaces, and volumes.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] A coal mine belt conveyor fire monitoring and warning system includes a fire hierarchical warning system, a collection subsystem, and a sprinkler subsystem; the fire hierarchical warning system is respectively connected to an AI video subsystem, a voiceprint recognition subsystem, an inspection robot subsystem, a distributed fiber optic temperature measurement subsystem, and a sprinkler subsystem through the mine Ethernet ring network for data connection.

[0012] The collection subsystem includes an AI video subsystem, a voiceprint recognition subsystem, an inspection robot subsystem, and a distributed fiber optic temperature measurement subsystem; the AI video subsystem is used to obtain the video image features of the belt conveyor; the voiceprint recognition subsystem is used to obtain the voiceprint features of the belt conveyor; the inspection robot subsystem is used to obtain the concentration of characteristic gases and the concentration of smoke in the mine; the distributed fiber optic temperature measurement subsystem is used to obtain the temperature of the belt conveyor.

[0013] The fire classification early warning system performs fusion analysis on the concentrations of landmark gases, smoke concentrations, temperatures, video image features, and voiceprint features obtained by the acquisition subsystem through a multi-modal information fusion analysis model to achieve fire hazard and fire classification early warning in the area of coal mine belt conveyors, including equipment anomaly monitoring and alarm, single-index early warning, and multi-parameter fusion early warning.

[0014] The sprinkler subsystem realizes remote automatic control of sprinkler according to the early warning level analyzed by the fire classification early warning system.

[0015] Preferably, the AI video subsystem includes a camera and a video intelligent analysis system; the video intelligent analysis system is connected to the mine Ethernet ring network and is data-connected to the camera; the camera includes an intrinsically safe camera, an intrinsically safe thermal imaging camera, an intrinsically safe image processing camera, and an intrinsically safe laser emitter, which are installed at fixed positions on the belt conveyor or in the roadway according to the application scenario requirements for obtaining video images; the video intelligent analysis device is installed on the ground for realizing automatic video image streaming, recognition, and judgment.

[0016] Preferably, the voiceprint recognition subsystem includes an intrinsically safe fault diagnosis system host and an intrinsically safe sound sensor; the intrinsically safe fault diagnosis system host is connected to the mine Ethernet ring network, and the intrinsically safe fault diagnosis system host is data-connected to the intrinsically safe sound sensor; the intrinsically safe sound sensor is deployed on the metal surface of the outer shell of the device to be measured by magnetic adsorption, collects the sound signals of the operating device based on the vibration sound pickup principle, and restores the device sound through a built-in processing circuit; when identifying abnormal idlers of the belt conveyor, one intrinsically safe sound sensor is deployed every 10m to 20m along the belt conveyor; when identifying abnormal rotating components such as motors, speed reducers, and drums at the driving part of the belt conveyor, one intrinsically safe sound sensor is deployed at each of the H directions at the driving and non-driving ends of the motor, one at each of the V directions at the driving end of the input shaft of the speed reducer, the non-driving end of the second shaft, and the driving end of the output shaft, and one at each of the H directions at the driving and non-driving ends of the driving drum.

[0017] Preferably, the inspection robot subsystem includes a rail-mounted inspection instrument or a 5G rail-mounted inspection instrument, a flameproof and intrinsically safe controller, an intrinsically safe card reader substation, an intrinsically safe passive identification card, a flameproof variable frequency three-phase asynchronous motor and an underground non-metallic track; the intrinsically safe card reader substation is connected to the mine Ethernet ring network, the intrinsically safe passive identification card is fixed on the surface of the non-metallic track and used for position calibration of the rail-mounted inspection instrument, and the rail-mounted inspection instrument or the 5G rail-mounted inspection instrument is respectively connected to the flameproof The flameproof and intrinsically safe controller is used to control the speed of the flameproof and intrinsically safe three-phase asynchronous motor; a non-metallic track is suspended and installed on the top of the belt conveyor lane, and a flameproof and intrinsically safe three-phase asynchronous motor is deployed at the end of the non-metallic track. The inspection robot subsystem is deployed with one or more rail-mounted inspection instruments or 5G rail-mounted inspection instruments, and a flameproof and intrinsically safe controller and an intrinsically safe card reading substation are deployed at the head of the belt conveyor.

[0018] Preferably, the distributed fiber optic temperature measurement subsystem includes an intrinsically safe fiber optic temperature measurement host or a flameproof and intrinsically safe fiber optic temperature measurement host, and a temperature-sensitive optical cable; the intrinsically safe fiber optic temperature measurement host is connected to the mine Ethernet ring network, the intrinsically safe fiber optic temperature measurement host or the flameproof and intrinsically safe fiber optic temperature measurement host are respectively connected to the temperature-sensitive optical cables, and the temperature-sensitive optical cables are deployed along the belt conveyor frame.

[0019] Preferably, the sprinkler subsystem includes an intrinsically safe repeater, a second flameproof and intrinsically safe controller, a flameproof solenoid valve and a flameproof and intrinsically safe valve electric device to realize remote automatic control of sprinkler; the intrinsically safe repeater is connected to the mine Ethernet ring network, the second flameproof and intrinsically safe controller is data-connected with the flameproof and intrinsically safe valve electric device, and the second flameproof and intrinsically safe controller is connected to the flameproof solenoid valve; the second flameproof and intrinsically safe controller realizes remote start and stop control of sprinkler; the flameproof and intrinsically safe valve electric device or flameproof solenoid valve is installed on the belt conveyor sprinkler pipeline.

[0020] Furthermore, fire monitoring and early warning are divided into the following three levels:

[0021] First stage, initial stage of fire:

[0022] The temperature T1 monitored by the thermal imager is greater than or equal to 1.5 times the warning temperature T1 y1 , but less than 3.5 times the warning temperature T1 y2 ;

[0023] The distributed optical fiber temperature measurement subsystem detects that the temperature T2 of the measurement point along the belt conveyor is greater than or equal to the first threshold warning value T2 of the measurement point y1 , and the temperature rise slope value T3 of the measuring point is less than or equal to the temperature rise slope warning value T3 of the measuring point y At the same time, the regional temperature T4 is greater than or equal to the first threshold warning value T4 of the regiony1 and the regional temperature rise slope T5 is less than or equal to the regional temperature rise slope warning value T5 y ;

[0024] The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentrations monitored at the downwind of the roadway is within the range of Level 1 warning;

[0025] The visible light camera or thermal imager does not detect abnormal smoke alarms;

[0026] Second level, fire development stage:

[0027] The temperature T1 monitored by the thermal imager is greater than or equal to 3.5 times the warning temperature T1 y2 , but less than the ignition point temperature T1 r ; At the same time, the temperatures of the measurement points and the regional temperature along the temperature measurement line of the distributed optical fiber temperature measurement subsystem all rise rapidly. The temperature T2 of the measurement points along the line is greater than or equal to the second threshold warning value T2 of the measurement points y2 , the temperature rise slope T3 of the measurement points is greater than the temperature rise slope warning value T3 of the measurement points y , and the regional temperature T4 is greater than or equal to the second threshold warning value T4 y2 , and the regional temperature rise slope T5 is greater than the regional temperature rise slope warning value T5 y ;

[0028] The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentrations monitored at the downwind of the roadway is within the range of Level 2 warning;

[0029] The visible light camera or thermal imager detects smoke and issues an alarm message;

[0030] Third level, fire combustion stage:

[0031] The temperature T1 monitored by the thermal imager is greater than or equal to the ignition point temperature T1 r ; The temperature T2 of the measurement points along the temperature measurement line of the distributed optical fiber temperature measurement subsystem is greater than or equal to the alarm value T2 of the measurement points b , the temperature rise slope T3 of the measurement points is greater than or equal to the temperature rise slope alarm value T3 of the measurement points b , and the regional temperature T4 is greater than or equal to the regional alarm value T4 b , and the regional temperature rise slope T5 is greater than or equal to the regional temperature rise slope alarm value T5 b ;

[0032] The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentrations monitored at the downwind of the roadway is within the range of Level 3 warning;

[0033] The visible light camera or thermal imager detects obvious smoke and fire and issues an alarm message.

[0034] Furthermore, the single-index early warning algorithm includes an early warning temperature algorithm, a temperature rise slope algorithm, a gas and smoke concentration algorithm, an AI video image recognition algorithm, and a device abnormal sound pattern recognition algorithm.

[0035] Furthermore, the gas concentration algorithm specifically includes the following steps:

[0036] S1: Use the long-term unchanged index, mutation index, and true range average index as single characteristic indicators;

[0037] (1) Long-term unchanged index: Used to assist in verifying the effectiveness of the monitoring value. The change threshold c needs to be determined according to the historical change statistical results of each point; Membership function:

[0038]

[0039] Among them, s1 is the long-term unchanged index score of the monitoring point, c is the change threshold of the monitoring point, and x 1h,v 、x 3h,v 、x 7h,v 、x 15h,v are the 1h change amount, 3h change amount, 7h change amount, and 15h change amount of the monitoring point respectively.

[0040] (2) Mutation index: According to the statistical results of the change rates of carbon monoxide, methane, oxygen, and smoke concentration and the number of alarms, determine the membership function of this early warning index:

[0041]

[0042] Among them, S2 is the mutation index score, and x is the difference between two samplings;

[0043] (3) True range average index: The range directly reflects the change rate of carbon monoxide, methane, oxygen, and smoke concentration over a period of time. The lower the range, the more stable and the higher the safety. According to the mathematical statistics results, the membership function of this index:

[0044]

[0045] In the formula, S3 is the true range average score, and x is the 12h true range average of the monitoring point.

[0046] S2: The comprehensive early warning algorithm for signature gases and smoke is as follows:

[0047] Comprehensively analyze the various characterizations of carbon monoxide, methane, oxygen, and smoke concentration, and comprehensively judge the concentration change trend of the monitoring point. Calculate the comprehensive score:

[0048]

[0049] Where S is the comprehensive score of the monitoring point, and w i is the weight of the i-th index, and s i is the score of the i-th index. The weights of the indexes of carbon monoxide, methane, oxygen, and smoke concentration are determined by the analytic hierarchy process and are 0.4, 0.1, 0.1, and 0.4 in sequence.

[0050] Combined with the determination principle of the membership function of each index, the early warning levels and thresholds of each index are: Level 1 early warning [0, 55) points, Level 2 early warning [55, 75) points, and Level 3 early warning [75, 85) points.

[0051] Furthermore, the specific AI video image recognition algorithm is as follows: The collected video image is used as input data. The input images a and b are converted into a single-channel grayscale image Gray through the geometric mean method. The expression is:

[0052]

[0053] where chn k represents the k-th color channel, K represents the total number of channels used for calculation, and R, G, and B respectively represent the three color channels of the real image; when the input images a and b are at the same moment, the output value in the Decision Network is 1, indicating that the images are the same, otherwise it is 0, indicating that the images are different; at the input end, the mean superposition map of the grayscale images of a and b is added, and the corresponding elements are averaged:

[0054]

[0055] where m and n represent the element values of the m-th column and n-th row of the grayscale image matrix; G avg(m,n) represents the mean of the element values of the m-th column and n-th row of the grayscale image matrix;

[0056] Based on the principle of information entropy to measure uncertainty, the mean superposition map c is used as the input:

[0057]

[0058] where H(x) is the difference representation between images a and b; p(x m,n ) represents the difference in the element values at the same position in the two matrices. The larger the value, the greater the difference in the element values of images a and b at the m-th column and n-th row;

[0059] When images a and b are exactly equal, a = b = c, p(x m,n ) = 0, the information entropy of the input image H(X) = 0. The smaller the information entropy, the smaller the confusion degree between images a and b, and the higher the similarity between a and b; the larger the information entropy, the greater the confusion degree between images a and b, and the lower the similarity between a and b;

[0060] The image a, image b, and the mean superimposed image c are combined into a three-channel image, which is input into the convolutional layer of the network to extract features of color, texture, shape, and the topological structure of the image. The input is a three-channel image composed of 3 grayscale images. Each convolutional layer has 3 feature maps. Each neuron in the convolutional layer is connected to multiple neurons in the area with a close position in the previous layer. Each neuron in the convolutional layer is connected to multiple neurons in the area with a close position in the previous layer:

[0061]

[0062] Among them, Z l+1 is the output of the (l + 1)-th layer convolution, that is, the feature map; Z(i, j) is the feature image pixel, is the Kronecker product of tensor operations, w l+1 is the network parameter value of the (l + 1)-th layer, and g is the bias term of the (l + 1)-th layer parameter; After the three-channel superposition of the image, the complexity of the model will increase. To prevent overfitting of the model, during the convolution process, a Drop Block regularization operation is performed on the convolutional layer, and the pixel values of a region block formed by combining adjacent region units in the feature map are set to 0;

[0063]

[0064] Among them, γ represents the number of Drop Block elements, the γ probability value controls the amount of features masked, feat_size and block_size represent the feature map and region block sizes, and keep_prob is the retention ratio;

[0065] The result Z after the convolution operation is output to the fully connected layer D:

[0066]

[0067] Among them, D i represents the output of the i-th neuron in D, W ij represents the j-th parameter of the i-th neuron, J represents the number of parameters in the neuron, g i is the bias term of the i-th neuron parameter. After Dropout in the Decision Network, the output result f(X) of the activation function is obtained, and the model effect is measured through the loss function:

[0068]

[0069] Among them, Y is the true value of the sample, S is the number of samples, and the squared loss function combines L2 regularization to evaluate the model effect in the training stage and control the training time.

[0070] Furthermore, the device abnormal voiceprint recognition algorithm specifically includes the following steps:

[0071] (1) Obtain the idler sound signal and preprocess it; use wavelet threshold denoising to remove Gaussian white noise, select 3 for the decomposition level, and select db6 for the wavelet basis; pre-emphasize the preprocessed sound signal through a high-pass filter, and for frame processing, the frame length is 1 s, the sampling frequency is 22050 Hz, and each frame of the sound signal is multiplied by the Hamming window function; obtain time-domain feature one, that is, use short-time average energy, short-time average amplitude, short-time zero-crossing rate, peak-to-peak value, and kurtosis to characterize the signal features, and perform normalization processing. After processing, they are E′, M′, Z′, F′, C′ respectively, and then multiply by the weight coefficients a1, a2, a3, a4, a5 respectively; then add them up to obtain the eigenvalue T1 of time-domain feature one, and the expression is as follows:

[0072] F1 = E′a1 + M′a2 + Z′a3 + F′a4 + C′a5

[0073] Among them, the value ranges of a1, a2, a3, a4, a5 are (0, 1);

[0074] Compare the value of time-domain feature one with the preset threshold one. If it is greater than the preset threshold one, issue warning one;

[0075] (2) After performing Fourier transform on the preprocessed signal, calculate the frequency-domain energy; divide the frequency domain into 4 subbands and calculate the subband energy ratio; extract the resonance peaks in the spectral envelope and calculate the sharpness and 1 / 3 octave eigenvalue; the sharpness S is expressed as follows:

[0076]

[0077] Among them, N'(z) is the loudness spectrum on the critical band Z, and the integral of the loudness spectrum over the critical band is the loudness; g(z) is the additional coefficient, Z is the critical band, and d(z) is the critical band differential;

[0078] Normalize the frequency-domain energy, subband energy ratio, resonance peak features, sharpness, and 1 / 3 octave features. Multiply the normalized features by the weight coefficients b1, b2, b3, b4, b5 respectively, and then add them up to obtain the eigenvalue F2 of frequency-domain feature two. Among them, the value ranges of b1, b2, b3, b4, b5 are (0, 1); when the frequency-domain eigenvalue F2 is greater than the preset threshold two, issue warning two;

[0079] (3) When both Warning One and Warning Two occur, generate a voiceprint spectrum, separate the voiceprint spectrum to obtain a harmonic component and a shock wave component. That is, perform a short-time Fourier transform (STFT) on the preprocessed sound signal, and obtain an energy spectrogram after taking the square of the modulus of its value; generate a harmonic component spectrogram and a shock wave component spectrogram from the energy spectrogram through a set of horizontal median filters and a set of vertical median filters respectively; construct a masking matrix by means of binary masking, divide all time-frequency bins in the STFT into the harmonic component or the shock wave component, and divide the harmonic component and the shock wave component from the original STFT result; finally, perform an inverse short-time Fourier transform (iSTFT) on the harmonic component and the shock wave component to obtain the harmonic component and the shock wave component of the original signal;

[0080] Perform MFCC transformation on the shock wave component, with the order taken as 12 - 16, to obtain Voiceprint Feature Three. When the value of Voiceprint Feature Three exceeds the preset threshold, confirm that the device is abnormal and issue an abnormal alarm;

[0081] (4) An equipment anomaly recognition method based on neural network multi-feature fusion, that is, respectively perform singular value decomposition (SVD) on the shock wave component separated by HPSS (Harmonic - Percussive Source Separation) and the MFCC feature matrix to reduce the dimension to a one-dimensional vector, and the number of features is 44 and 10 respectively; then perform normalization to obtain Time - Frequency Feature Three; design a fully connected neural network model with a three-layer network structure, the number of neurons in the hidden layer is 1024, 512, and 256 in sequence, use the relu activation function, and the Adam optimization function to improve the optimization efficiency; use the sigmoid function to activate in the output layer, and use the binary cross-entropy function as the model loss function to perform binary classification discrimination between normal and abnormal; fuse Time - Domain Feature One, Frequency - Domain Feature Two, and Time - Frequency Feature Three into New Feature Four; Time - Domain Feature One includes short-time average energy, short-time average amplitude, short-time zero-crossing rate, peak-to-peak value, and kurtosis; Frequency - Domain Feature Two includes frequency-domain energy, sub-band energy ratio, formant, sharpness, and 1 / 3 octave; Time - Frequency Feature Three includes the separated shock wave component and MFCC; input the new feature vector into the fully connected neural network model and output the final intelligent recognition result; in the training of the model, through the backpropagation algorithm and multiple data iterations, fit the best network parameters to optimize the recognition model;

[0082] When it is recognized as abnormal, output an equipment abnormal alarm signal. When manual review of the abnormal alarm is available, if it is false, manually calibrate the corresponding abnormal sound signal and add it to the built-in voiceprint feature library, effectively reducing false alarms on-site through self-learning and improving robustness; when it is recognized as normal, calculate the similarity between New Feature Four and the built-in features using the Minkowski distance. If the similarity exceeds the set threshold, then the match is successful, and a corresponding normal signal or abnormal alarm signal is issued.

[0083] Furthermore, the multi-modal information fusion analysis model constructs the characteristic gas concentration, smoke concentration, and temperature characteristics into a feature vector x through combination. i , extracts and fits them by using the method of multiple linear regression; the expression of multiple linear regression is y2:

[0084] y2 = β0 + β1x1 + β2x2 + … + β i x i + u

[0085] Among them, β i represents the weight coefficient, and u represents the bias term;

[0086] Fuse the output values y of the residual convolution module and the multiple linear regression module through the fully connected layer, which is expressed as:

[0087] y = w1y1 + w2y2

[0088] Among them, w1 and w2 represent the weight coefficients, which are 0.4 and 0.6 respectively; y1 is the video image feature;

[0089] After fitting the parameters through the fully connected layer, the fire monitoring and early warning recognition result of the belt conveyor is obtained. When the y value is in [0, 0.3), [0.3, 0.5), it is the initial stage of the fire; when it is in [0.5 - 0.7), it is the development stage of the fire; when it is in [0.7 - 1.0), it is the combustion stage of the fire.

[0090] The beneficial effects of the present invention are as follows:

[0091] (1) Aiming at the drawbacks of the existing single belt conveyor fire early warning technology, which cannot meet the requirements of coal mine belt conveyor fire early warning, the present invention is based on multi-modal perception technologies such as signature gas, temperature, smoke, sound, video image, etc., and uses the multi-dimensional information fusion method for on-line analysis and judgment to construct a fire monitoring and early warning system with full coverage of points, lines, surfaces, and volumes, realizing the fire risk control in areas such as belt conveyors and preventing fire accidents.

[0092] (2) The present invention uses new technologies such as AI video image and voiceprint recognition based on machine learning to early detect abnormal noises in the idlers and drive parts of the belt conveyor, abnormal temperatures, open flames, smoke, floating coal accumulation, coal stacking, foreign objects (bolts) in the coal flow, large pieces of materials in the coal flow, coal flow rate, and deviation, early discover fire hazards, and avoid fire accidents.

[0093] (3) Through the mobile inspection robot, the regular inspection of the belt conveyor and the roadway and the fixed monitoring complement each other, filling the monitoring blind spots, and realizing the online monitoring and analysis of potential fire hazards in the man-machine-environment along the belt conveyor, such as smoke, carbon monoxide, oxygen, methane, infrared temperature, distributed optical fiber temperature, voiceprint, visible light, and thermal imaging AI video images. It can identify abnormal equipment, abnormal equipment and environmental temperature, and abnormal roadway gas and smoke concentration. According to the constructed fire warning model, it can timely detect equipment failures, precursors of fires, and fire situations, automatically locate the ignition point / fire point, conduct hierarchical early warnings, timely remind the staff to handle, and the system automatically links to sprinkle water to prevent the expansion of accidents and ensure the safe production of coal mines.

[0094] Other advantages, objectives, and features of the present invention will be described to some extent in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be achieved and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0095] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail with reference to the accompanying drawings, where:

[0096] Figure 1 It is a connection diagram of the composition of the coal mine belt conveyor fire monitoring and early warning system in the embodiment of the present invention;

[0097] Figure 2 It is a fire monitoring hierarchical early warning model diagram of the coal mine belt conveyor in the embodiment of the present invention;

[0098] Figure 3 It is a method for intelligent video detection of the belt conveyor in the embodiment of the present invention;

[0099] Figure 4 It is a flow chart of the method for extracting abnormal characteristics of equipment based on HPSS in the embodiment of the present invention;

[0100] Figure 5 It is a flow chart of the method for identifying abnormal equipment based on multi-feature fusion of neural networks in the embodiment of the present invention;

[0101] Figure 6 It is a processing flow chart of the abnormal equipment identification model based on multi-feature fusion of neural networks in the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0102] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0103] Among them, the drawings are only for illustrative purposes, showing only schematic diagrams, not physical diagrams, and should not be construed as a limitation to the present invention; in order to better illustrate the embodiments of the present invention, some components in the drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the drawings may be omitted.

[0104] Please refer to Figures 1 to 6 , an embodiment of the present invention provides a fire monitoring and early warning system for a coal mine belt conveyor, as Figure 1 shown, the equipment composition is as follows:

[0105] The ground part includes a ground ring network switch, a video intelligent analysis device, a data server, an industrial control computer, and the coal mine belt conveyor fire monitoring and early warning software running thereon. The video intelligent analysis device realizes the automatic extraction, recognition, and judgment of video images, and uploads alarm information and alarm capture pictures to the coal mine belt conveyor fire monitoring and early warning software through protocols such as Ethernet Modbus TCP, Http, and MQTT; the coal mine belt conveyor fire monitoring and early warning software realizes functions such as acquisition data display, storage query, printing, statistical report generation, man-machine dialogue, self-diagnosis, data backup, alarm linkage, and fire early warning.

[0106] The underground part includes an underground ring network switch, an AI video subsystem, a voiceprint recognition subsystem, an inspection robot subsystem, a distributed optical fiber temperature measurement subsystem, and a sprinkler subsystem.

[0107] The AI video subsystem includes an intrinsically safe camera, an intrinsically safe thermal imaging camera, an intrinsically safe image processing camera, and an intrinsically safe laser emitter. The intrinsically safe camera, the intrinsically safe thermal imaging camera, the intrinsically safe image processing camera, and the intrinsically safe laser emitter are installed at fixed positions on the belt conveyor and in the roadway according to the application scenario requirements.

[0108] The voiceprint recognition subsystem includes an intrinsically safe fault diagnosis system host and an intrinsically safe sound sensor. The intrinsically safe fault diagnosis system host is connected to the mine Ethernet ring network, and the intrinsically safe fault diagnosis system host is connected to the intrinsically safe sound sensor for data. The intrinsically safe sound sensor is deployed on the metal surface of the outer shell of the device to be measured by a magnetic adsorption method, collects the sound signal of the operating device based on the vibration sound pickup principle, and has a built-in processing circuit to restore the device sound; when identifying the abnormality of the belt conveyor idler, 1 intrinsically safe sound sensor is deployed every 10m - 20m along the belt conveyor; when identifying the abnormality of rotating components such as the motor, speed reducer, and drum of the drive part of the belt conveyor, 1 intrinsically safe sound sensor is deployed at each of the H directions at the driving and non-driving ends of the motor, 1 at each of the driving ends of the input shaft, second shaft non-driving end, and output shaft driving end of the speed reducer in the V direction, and 1 at each of the driving and non-driving ends of the driving drum in the H direction.

[0109] The inspection robot subsystem includes a hanging rail type inspection instrument, a 5G hanging rail type inspection instrument, an explosion-proof and intrinsically safe controller I, an intrinsically safe card reading substation, an intrinsically safe passive identification card, an explosion-proof variable frequency three-phase asynchronous motor, and an underground non-metallic track. The intrinsically safe card reading substation is connected to the mine Ethernet ring network; the intrinsically safe passive identification card is fixed on the surface of the non-metallic track for position calibration of the hanging rail type inspection instrument, and the hanging rail type inspection instrument and the 5G hanging rail type inspection instrument are respectively mechanically connected to the explosion-proof variable frequency three-phase asynchronous motor; the explosion-proof and intrinsically safe controller I is used to control the speed of the explosion-proof variable frequency three-phase asynchronous motor. The non-metallic track is hung and installed at the top of the belt conveyor roadway, 1 explosion-proof variable frequency three-phase asynchronous motor is deployed at the end of the non-metallic track, 1 or more hanging rail type inspection instruments or 5G hanging rail type inspection instruments are deployed in the inspection robot subsystem, and 1 explosion-proof and intrinsically safe controller I and 1 intrinsically safe card reading substation are deployed at the head of the belt conveyor.

[0110] The distributed optical fiber temperature measurement subsystem includes an intrinsically safe optical fiber temperature measurement host, an explosion-proof and intrinsically safe optical fiber temperature measurement host, and a temperature sensing optical cable. The intrinsically safe optical fiber temperature measurement host is connected to the mine Ethernet ring network, the intrinsically safe optical fiber temperature measurement host and the explosion-proof and intrinsically safe optical fiber temperature measurement host are respectively connected to the temperature sensing optical cable, and the temperature sensing optical cable is deployed along the belt conveyor frame.

[0111] The sprinkler subsystem includes an intrinsically safe repeater, an explosion-proof and intrinsically safe controller II, an explosion-proof solenoid valve, and an explosion-proof and intrinsically safe valve electric device. The intrinsically safe repeater is connected to the mine Ethernet ring network, the explosion-proof and intrinsically safe controller II is connected to the explosion-proof and intrinsically safe valve electric device for data, and the explosion-proof and intrinsically safe controller II is connected to the explosion-proof solenoid valve. The explosion-proof and intrinsically safe controller II realizes remote start and stop control of sprinkler; the explosion-proof and intrinsically safe valve electric device or the explosion-proof solenoid valve is installed on the sprinkler pipeline of the belt conveyor.

[0112] Such as Figure 2As shown, the coal mine belt conveyor fire monitoring and early warning indicators cover three aspects: man, machine, and environment, including: characteristic gas concentration, smoke concentration, temperature, video image features, and voiceprint features. The system realizes the fire hazards and fire graded early warning in the coal mine belt conveyor area through multimodal information fusion analysis. The graded early warning indicators include equipment abnormality monitoring alarm, single indicator early warning, and multi-parameter fusion early warning. Single indicator early warning, including characteristic gas concentration, smoke concentration, temperature, video image features, voiceprint features, each single indicator reaches the early warning threshold. The multi-parameter fusion early warning is a multimodal information fusion analysis model established based on each single indicator. It comprehensively analyzes the various representations of fire hazards in man, machine, and environment, comprehensively judges the fire situation, and comprehensively scores it, and divides it into three early warning levels, namely the initial stage of fire, the development stage of fire, and the combustion stage of fire. The details are as follows:

[0113] 1. Equipment abnormality monitoring and alarm, including equipment abnormality, environmental abnormality, personnel violation, alarm reminder, and timely processing. Features include:

[0114] Equipment abnormalities include abnormalities in voiceprint recognition rollers, motors, reduction gearboxes, rollers, etc.;

[0115] Environmental anomalies include floating coal accumulation, coal piles, foreign objects / anchors, large coal lumps, deviations, etc. identified by AI video images;

[0116] Personnel violations include AI video image recognition personnel failing to conduct regular inspections.

[0117] 2. Fire monitoring and graded warning: Based on the characteristics of characteristic gases, smoke concentration, temperature and video images, a three-level graded warning system for fire monitoring is implemented.

[0118] 1) First stage, initial stage of fire

[0119] During this stage, the conveyor belt becomes hot due to roller failure and friction of floating coal. The motor, roller, reducer, cable, etc. become hot due to heavy load and mechanical damage. The infrared thermal imager monitors that the temperature of the equipment along the belt conveyor and in the area is still low, without smoke or fire. The characteristics include:

[0120] The temperature T1 monitored by the thermal imager is greater than or equal to 1.5 times the warning temperature T1 y1 , but less than 3.5 times the warning temperature T1 y2 ;

[0121] The distributed optical fiber temperature measurement subsystem detects that the temperature T2 of the measurement point along the belt conveyor is greater than or equal to the first threshold warning value T2 of the measurement point y1 , and the temperature rise slope value T3 of the measuring point is less than or equal to the temperature rise slope warning value T3 of the measuring point y At the same time, the regional temperature T4 is greater than or equal to the first threshold warning value T4 of the region y1and the regional temperature rise slope T5 is less than or equal to the regional temperature rise slope warning value T5 y ;

[0122] The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentration monitored at the downwind of the roadway is within the range of level 1 warning;

[0123] The visible light camera or thermal imager does not detect abnormal smoke and give an alarm.

[0124] 2) The second level, the fire development stage

[0125] The temperature rises rapidly in this stage, manifested as the obvious and rapid increase in the temperature of rotating parts such as idlers, motors, drums, and speed reducers or cables. The characteristics include:

[0126] The temperature T1 monitored by the thermal imager is greater than or equal to 3.5 times the warning temperature T1 y2 , but less than the ignition temperature T1 r ; At the same time, the temperature of the measurement points and the regional temperature along the measurement line of the distributed optical fiber temperature measurement subsystem both rise rapidly. The temperature T2 of the measurement points along the line is greater than or equal to the second threshold warning value T2 of the measurement points y2 , the temperature rise slope T3 of the measurement points is greater than the warning value T3 of the temperature rise slope of the measurement points y , and the regional temperature T4 is greater than or equal to the second threshold warning value T4 y2 , and the regional temperature rise slope T5 is greater than the regional temperature rise slope warning value T5 y ;

[0127] The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentration monitored at the downwind of the roadway is within the range of level 2 warning;

[0128] The visible light camera or thermal imager detects smoke and sends out an alarm message.

[0129] 3) The third level, the fire combustion stage

[0130] Open flames appear in this stage, the temperature suddenly rises steeply and expands rapidly, and at the same time a large amount of smoke appears accompanied by a burnt smell. The characteristics include:

[0131] The temperature T1 monitored by the thermal imager is greater than or equal to the ignition temperature T1 r ; The temperature T2 of the measurement points along the measurement line of the distributed optical fiber temperature measurement subsystem is greater than or equal to the alarm value T2 of the measurement points b , the temperature rise slope T3 of the measurement points is greater than or equal to the alarm value T3 of the temperature rise slope of the measurement points b , and the regional temperature T4 is greater than or equal to the regional alarm value T4 b , and the regional temperature rise slope T5 is greater than or equal to the regional temperature rise slope alarm value T5 b ;

[0132] The comprehensive score S of carbon monoxide, methane, oxygen, and smoke concentration monitored at the downwind of the roadway is within the range of level 3 warning;

[0133] Visible light cameras or thermal imagers detect obvious fire and smoke and send out alarm messages.

[0134] 3. Single-index warning algorithm

[0135] 1) Warning temperature algorithm:

[0136] Let the measured temperature be x(t), the ambient temperature at the measurement point be h(t), the comparison signal be y(t), and the transformation function be T. Then we have:

[0137] y(t) = T(x(t) - h(t)) + x(t)

[0138]

[0139] In the formula: D(y(t)) = 1 indicates that the temperature rise exceeds the limit; s is divided into 3 levels. s1 = 50 is set as the first-level warning line, s2 = 70 is the second-level warning line, and s3 = 90 is the third-level warning line.

[0140] 2) Temperature rise slope algorithm:

[0141] The slope ΔX(t) / ΔT = (X(t2) - X(t1)) / (t2 - t1), where t2 is the current moment and t1 is the previous moment. To improve the reliability and anti-interference ability of the temperature rise slope algorithm at the measurement point, the temperature data at the measurement point is averaged and delayed. The slope threshold s of the measurement point is divided into two levels, s1 and s2, with values of s1 = 0.11 °C / min and s2 = 0.1755 °C / min respectively. Then the warning temperature of the temperature rise slope at the measurement point is:

[0142]

[0143] 3) Gas concentration algorithm, including carbon monoxide, methane, oxygen, and smoke concentration algorithms:

[0144] (1) Use the long-term invariant index, mutation index, and mean index of the true fluctuation range as single characteristic indicators.

[0145] ① Long-term invariant index: Used to assist in verifying the effectiveness of the monitored value. The change threshold c needs to be determined according to the historical change statistical results of each point. Membership function:

[0146]

[0147] Among them, S1 is the long-term invariant index score of the monitoring point, c is the change threshold of the monitoring point, x 1h,v , x 3h,v , x7h,v , x 15h,v are the change amounts at the monitoring point for 1h, 3h, 7h, and 15h respectively.

[0148] ②Mutation index: According to the statistical results of the change rates of carbon monoxide, methane, oxygen, and smoke concentration and the number of alarm times, determine the membership function of this early warning index:

[0149]

[0150] Among them, s2 is the mutation index score, and x is the difference between two samplings.

[0151] ③True fluctuation amplitude mean index: The fluctuation amplitude directly reflects the change rate of carbon monoxide, methane, oxygen, and smoke concentration over a period of time. The lower the fluctuation amplitude, the more stable and the higher the safety. According to the statistical results, the membership function of this index:

[0152]

[0153] In the formula, S3 is the true fluctuation amplitude mean score, and x is the true fluctuation amplitude mean at the monitoring point for 12h.

[0154] (2) The comprehensive early warning algorithm for landmark gases and smoke is as follows:

[0155] Comprehensively analyze the various characteristics of carbon monoxide, methane, oxygen, and smoke concentration, and conduct a comprehensive identification of the concentration change trend at the monitoring point. The comprehensive score calculation:

[0156]

[0157] In the formula, S is the comprehensive score of the monitoring point, w i is the weight of the i-th index, s i is the score of the i-th index. The weights of carbon monoxide, methane, oxygen, and smoke concentration indicators are determined by the analytic hierarchy process and are 0.4, 0.1, 0.1, and 0.4 in turn.

[0158] Combined with the determination principle of the membership function of each index, the early warning levels and thresholds of each index are: Level 1 early warning [0, 55) points, Level 2 early warning [55, 75) points, Level 3 early warning [75, 85) points.

[0159] 4) AI video image recognition algorithm, as Figure 3 shown.

[0160] Taking the collected video images as input data, convert the input images a and b into a single-channel grayscale image Gray through the geometric mean method. The expression is:

[0161]

[0162] Among them, chn k represents the k-th color channel, K represents the total number of channels used for calculation, and R, G, and B respectively represent the three color channels of the real image; when the input images a and b are at the same moment, if the output value in the Decision Network is 1, it means the images are the same, otherwise 0, indicating the images are different; at the input end, the mean superposition map of the grayscale images of a and b is added, and the corresponding elements are averaged:

[0163]

[0164] Among them, m and n represent the element values of the m-th column and n-th row of the grayscale image matrix.

[0165] Based on the principle of information entropy to measure uncertainty, the mean superposition map c is used as the input:

[0166]

[0167] Among them, H(x) is the representation of the difference between images a and b; p(x m,n ) represents the difference in the element values at the same position in the two matrices. The larger the value, the greater the difference in the element values of images a and b at the m-th column and n-th row.

[0168] When images a and b are completely equal, a = b = c, p(x m,n ) = 0, the input image information entropy H(X) = 0. The smaller the information entropy, the smaller the degree of confusion between images a and b, and the higher the similarity between a and b; the larger the information entropy, the greater the degree of confusion between images a and b, and the lower the similarity between a and b.

[0169] Images a, b, and the mean superposition map c are combined into a three-channel image and input into the network convolutional layer for feature extraction of color, texture, shape, and the topological structure of the image; the input is a three-channel image composed of 3 grayscale images. Each convolutional layer has 3 feature maps. Each neuron in the convolutional layer is connected to multiple neurons in the region with a close position in the previous layer. Each neuron in the convolutional layer is connected to multiple neurons in the region with a close position in the previous layer:

[0170]

[0171] Among them, Z l+1 is the output of the (l + 1)-th layer convolution, that is, the feature map; Z(i, j) is the feature image pixel, is the Kronecker product of tensor operations, w l+1is the network parameter value of the (l + 1)-th layer, and g is the bias term of the parameters of the (l + 1)-th layer; after the image is superimposed in three channels, the complexity of the model will increase. To prevent the model from overfitting, during the convolution process, Drop Block regularization operation is performed on the convolutional layer, and the pixel values of a regional block formed by combining adjacent regional units in the feature map are set to 0;

[0172]

[0173] where γ represents the number of Drop Block elements, the γ probability value controls the amount of features to be masked, feat_size and block_size represent the feature map and regional block sizes respectively, keep_prob is the retention ratio, and the Drop Block operation is implemented in each convolutional layer through the above formula.

[0174] The result Z after the convolution operation is output to the fully connected layer D:

[0175]

[0176] where D i represents the output of the i-th neuron in D, W ij represents the j-th parameter of the i-th neuron, g i is the bias term of the parameters of the i-th neuron. After Dropout in the Decision Network, the output result f(X) of the activation function is obtained, and the model effect is measured through the loss function:

[0177]

[0178] where Y is the true value of the sample, S is the number of samples, the squared loss function combines L2 regularization, and the model effect is evaluated in the training stage to control the training time.

[0179] 5) The device abnormal voiceprint recognition algorithm, such as Figure 4 shown.

[0180] (1) Obtain the idler sound signal and preprocess it; use wavelet threshold denoising to remove Gaussian white noise, the decomposition layer number is selected as 3, and the wavelet basis is selected as db6. The preprocessed sound signal is pre-emphasized through a high-pass filter, and the frame processing adopts a frame length of 1 s, a sampling frequency of 22050 Hz, and each frame of the sound signal is multiplied by the Hamming window function. Obtain the time-domain feature one, that is, use short-time average energy, short-time average amplitude, short-time zero-crossing rate, peak-to-peak value, and kurtosis to characterize the signal features, and perform normalization processing. After processing, they are E′, M′, Z′, F′, C′ respectively, and then multiplied by the weight coefficients a1, a2, a3, a4, a5 respectively; then add them up to obtain the eigenvalue T1 of the time-domain feature one, and the expression is as follows:

[0181] T1 = E′a1 + M′a2 + Z′a3 + F′a4 + C′a5

[0182] Among them, the value ranges of a1, a2, a3, a4, and a5 are (0, 1);

[0183] Compare the value of the time-domain feature one with the preset threshold one. If it is greater than the preset threshold one, issue warning one.

[0184] (2) After performing Fourier transform on the preprocessed signal, calculate the frequency-domain energy; divide the frequency domain into 4 sub-bands and calculate the sub-band energy ratio; extract the resonance peaks in the spectral envelope and calculate the sharpness and 1 / 3 octave characteristic values; the sharpness S is expressed as follows:

[0185]

[0186] Among them, N'(z) is the loudness spectrum on the critical band Z, and the integral of the loudness spectrum over the critical band is the loudness; g(z) is the additional coefficient, Z is the critical band, and d(z) is the derivative of the critical band;

[0187] Normalize the frequency-domain energy, sub-band energy ratio, resonance peak characteristics, sharpness, and 1 / 3 octave characteristic values. Multiply the normalized characteristics by the weight coefficients b1, b2, b3, b4, and b5 respectively, and then add them to obtain the eigenvalue F2 of the frequency-domain feature two. Among them, the value ranges of b1, b2, b3, b4, and b5 are (0, 1); when the frequency-domain eigenvalue F2 is greater than the preset threshold two, issue warning two.

[0188] (3) When both warning one and warning two occur, generate a voiceprint spectrum, separate the voiceprint spectrum to obtain the harmonic component and the shock wave component, that is, perform short-time Fourier transform STFT on the preprocessed sound signal, and obtain the energy spectrogram after taking the square of its modulus value; generate the harmonic component spectrogram and the shock wave component spectrogram through a group of horizontal median filters and a group of vertical median filters respectively for the energy spectrogram; construct a masking matrix by means of binary masking, divide all the time-domain-frequency-domain bins in the STFT into the harmonic component or the shock wave component, and divide the harmonic component and the shock wave component from the original STFT result; finally, perform inverse short-time Fourier transform iSTFT on the harmonic component and the shock wave component to obtain the harmonic component and the shock wave component of the original signal.

[0189] Perform MFCC transformation on the shock wave component, and take the order as 12 - 16 to obtain the voiceprint feature three. When the value of the voiceprint feature three exceeds the preset threshold, confirm that the device is abnormal and issue an abnormal alarm.

[0190] (4) The device anomaly recognition method based on neural network multi-feature fusion, such as Figure 5 、 6As shown in the figure, the HPSS shock wave component and the MFCC feature matrix are respectively reduced to one-dimensional vectors through singular value decomposition (SVD), and the number of features is 44 and 10 respectively; then normalization is performed to obtain time-frequency feature three. A fully connected neural network model with a three-layer network structure is designed, and the number of neurons in the hidden layer is 1024, 512, and 256 in sequence. The relu activation function and the Adam optimization function are used to improve the optimization efficiency; the sigmoid function is used for activation in the output layer, and the binary cross-entropy function is used as the model loss function for normal and abnormal binary classification discrimination; the time-domain feature one, the frequency-domain feature two, and the time-frequency feature three are fused into a new feature four; the time-domain feature one includes short-time average energy, short-time average amplitude, short-time zero-crossing rate, peak-to-peak value, and kurtosis; the frequency-domain feature two includes frequency-domain energy, sub-band energy ratio, formant, sharpness, and 1 / 3 octave; the time-frequency feature three includes the separated shock wave component and MFCC; the new feature vector is input into the fully connected neural network model, and the final intelligent recognition result is output; in the training of the model, the best network parameters are fitted through the backpropagation algorithm and multiple data iterations to optimize the recognition model.

[0191] When an abnormality is recognized, the output device outputs an abnormal alarm signal. When manually reviewing the abnormal alarm, if it is false, the corresponding abnormal sound signal is manually calibrated and added to the built-in voiceprint feature library, which effectively reduces false alarms on-site and improves robustness through self-learning; when a normal situation is recognized, the Minkowski distance is used to calculate the similarity between the new feature four and the built-in features. If the similarity exceeds the set threshold, the matching is successful, and the corresponding normal signal or abnormal alarm signal is sent.

[0192] 4. An algorithm for fire monitoring and early warning of coal mine belt conveyors based on multi-modal fusion.

[0193] The graph features obtained through the AI video image recognition algorithm are regarded as y1;

[0194] The characteristic gas concentration, smoke concentration, temperature characteristics, etc. are combined to construct a feature vector x i , which is extracted and fitted by using the method of multiple linear regression; the expression of multiple linear regression is y2:

[0195] y2 = β0 + β1x1 + β2x2 + … + β i x i + u

[0196] Among them, β i represents the weight coefficient, and u represents the bias term.

[0197] The output values y of the residual convolution module and the multiple linear regression module are fused through the fully connected layer, which is expressed as:

[0198] y = w1y1 + w2y2

[0199] Among them, w1 and w2 represent weight coefficients, which are 0.4 and 0.6 respectively.

[0200] After fitting the parameters through the fully connected layer, the fire monitoring and early warning recognition result of the belt conveyor is obtained. When the y value is in the range of [0, 0.3), [0.3, 0.5) is the initial stage of the fire, [0.5 - 0.7) is the development stage of the fire, and [0.7 - 1.0) is the combustion stage of the fire.

[0201] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the purpose and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A fire monitoring and early warning system for a coal mine belt conveyor, characterized in that, The system includes a fire classification early warning system, a collection subsystem, and a sprinkler subsystem; the fire classification early warning system is connected to the collection subsystem and the sprinkler subsystem through the mine Ethernet ring network for data connection; The collection subsystem includes an AI video subsystem, a voiceprint recognition subsystem, an inspection robot subsystem, and a distributed optical fiber temperature measurement subsystem; the AI video subsystem is used to obtain the video image features of the belt conveyor; the voiceprint recognition subsystem is used to obtain the voiceprint features of the belt conveyor; the inspection robot subsystem is used to obtain the concentration of landmark gases and smoke concentration in the mine; the distributed optical fiber temperature measurement subsystem is used to obtain the temperature of the belt conveyor; The fire classification early warning system performs fusion analysis on the landmark gas concentration, smoke concentration, temperature, video image features, and voiceprint features obtained by the collection subsystem through a multi-modal information fusion analysis model to realize fire hazard and fire monitoring and early warning in the area of the coal mine belt conveyor, including equipment anomaly monitoring and alarm, single-index early warning, and multi-parameter fusion early warning; the single-index early warning algorithms include an early warning temperature algorithm, a temperature rise slope algorithm, a gas and smoke concentration algorithm, an AI video image recognition algorithm, and an equipment anomaly voiceprint recognition algorithm; The sprinkler subsystem realizes remote automatic control of sprinkler according to the early warning level analyzed by the fire classification early warning system.

2. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, characterized in that The AI video subsystem includes a camera and a video intelligent analysis system; the video intelligent analysis system is connected to the mine Ethernet ring network and is connected to the camera for data connection; the camera includes an intrinsically safe camera, an intrinsically safe thermal imaging camera, an intrinsically safe image processing camera, and an intrinsically safe laser emitter, which are installed at fixed positions on the belt conveyor or in the roadway according to the application scenario requirements for obtaining video images; the video intelligent analysis device is installed on the ground for realizing automatic video image streaming, recognition, and judgment.

3. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, characterized in that The voiceprint recognition subsystem includes an intrinsically safe fault diagnosis system host and an intrinsically safe sound sensor; the intrinsically safe fault diagnosis system host is connected to the mine Ethernet ring network, and the intrinsically safe fault diagnosis system host is connected to the intrinsically safe sound sensor for data connection; the intrinsically safe sound sensor is deployed on the metal surface of the outer shell of the measured equipment, and based on the vibration sound pickup principle, it collects the sound signals of the operating equipment and restores the equipment sound through the built-in processing circuit; when identifying the abnormality of the belt conveyor idler, one intrinsically safe sound sensor is deployed every 10m to 20m along the belt conveyor; when identifying the abnormality of the motor, reducer, and drum of the drive part of the belt conveyor, one intrinsically safe sound sensor is deployed at each of the H directions at the drive and non-drive ends of the motor, one at each of the V directions at the drive end of the input shaft of the reducer, the non-drive end of the second shaft, and the drive end of the output shaft, and one at each of the H directions at the drive and non-drive ends of the drive drum.

4. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, wherein, The inspection robot subsystem includes a rail-mounted inspection instrument or a 5G rail-mounted inspection instrument, a flameproof and intrinsically safe controller, an intrinsically safe card reader substation, an intrinsically safe passive identification card, a flameproof variable frequency three-phase asynchronous motor and an underground non-metallic track; the intrinsically safe card reader substation is connected to the mine Ethernet ring network, the intrinsically safe passive identification card is fixed on the surface of the non-metallic track, and is used for position calibration of the rail-mounted inspection instrument. The rail-mounted inspection instrument or the 5G rail-mounted inspection instrument is respectively connected to the flameproof variable frequency three-phase asynchronous motor and the underground non-metallic track. The three-phase asynchronous motor is mechanically connected, and the flameproof and intrinsically safe controller is used to control the speed of the flameproof variable frequency three-phase asynchronous motor; a non-metallic track is suspended and installed on the top of the belt conveyor tunnel, and a flameproof variable frequency three-phase asynchronous motor is deployed at the end of the non-metallic track. The inspection robot subsystem is deployed with one or more rail-mounted inspection instruments or 5G rail-mounted inspection instruments, and the belt conveyor head is deployed with one flameproof and intrinsically safe controller and one intrinsically safe card reading substation.

5. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, wherein, Fire monitoring and early warning are divided into the following three levels: First stage, initial stage of fire: The temperature T1 monitored by the thermal imager is greater than or equal to 1.5 times the warning temperature T1 y1 , but less than 3.5 times the warning temperature T1 y2 ; The distributed optical fiber temperature measurement subsystem monitors that the temperature T2 at the measurement points along the belt conveyor is greater than or equal to the first threshold warning value T2 of the measurement point y1 , and the temperature rise slope value T3 of the measurement point is less than or equal to the temperature rise slope warning value T3 of the measurement point y , and at the same time the regional temperature T4 is greater than or equal to the first threshold warning value T4 of the region y1 , and the regional temperature rise slope T5 is less than or equal to the regional temperature rise slope warning value T5 y ; The combined score S of carbon monoxide, methane, oxygen and smoke concentration monitored at the downwind outlet of the tunnel is within the level 1 warning range; The visible light camera or thermal imager did not detect any abnormal smoke alarm; Second stage, fire development stage: The temperature T1 monitored by the thermal imager is greater than or equal to 3.5 times the warning temperature T1 y2 , but less than the ignition temperature T1 r ; at the same time, the temperatures of the measurement points and regions along the line measured by the distributed optical fiber temperature measurement subsystem all rise rapidly. The temperature T2 of the measurement points along the line is greater than or equal to the second threshold warning value T2 of the measurement points y2 , the temperature rise slope T3 of the measurement points is greater than the warning value T3 of the temperature rise slope of the measurement points y , and the regional temperature T4 is greater than or equal to the second threshold warning value T4 y2 , the regional temperature rise slope T5 is greater than the warning value T5 of the regional temperature rise slope y ; The combined score S of carbon monoxide, methane, oxygen and smoke concentration monitored at the downwind outlet of the tunnel is within the level 2 warning range; Visible light cameras or thermal imagers detect smoke and send out alarm messages; The third stage, fire burning stage: The temperature T1 monitored by thermal imaging is greater than or equal to the ignition temperature T1 r ; The temperature T2 at the measurement points along the line measured by the distributed optical fiber temperature measurement subsystem is greater than or equal to the alarm value T2 of the measurement point b and the temperature rise slope T3 of the measurement point is greater than or equal to the alarm value T3 of the temperature rise slope of the measurement point b , and the regional temperature T4 is greater than or equal to the regional alarm value T4 b and the regional temperature rise slope T5 is greater than or equal to the alarm value T5 of the regional temperature rise slope b ; The comprehensive score S of carbon monoxide, methane, oxygen and smoke concentration monitored at the downwind outlet of the tunnel is within the level 3 warning range; Visible light cameras or thermal imagers detect obvious fireworks and issue an alarm message.

6. The coal mine belt conveyor fire monitoring and early warning system according to claim 1, wherein, The watering subsystem comprises an intrinsically safe repeater, a flameproof and intrinsically safe controller 2, a flameproof solenoid valve and a flameproof and intrinsically safe valve electric device, and is used to realize remote automatic control of watering.

7. The coal mine belt conveyor fire monitoring and early warning system according to claim 1, characterized in that, The gas and smoke concentration algorithm specifically includes the following steps: S1: Long-term constant index, mutation index and true fluctuation range mean index are used as single characteristic indicators; (1) Long-term invariance index: used to assist in verifying the validity of the monitoring value. The change threshold c needs to be determined based on the historical change statistics of each point; (2) Mutation index: Based on the statistical results of the change rate of carbon monoxide, methane, oxygen, and smoke concentrations and the number of alarms, the membership function of the early warning index is determined: S2: The algorithm for comprehensive warning of iconic gas and smoke is as follows: Comprehensively analyze the various characteristics of carbon monoxide, methane, oxygen and smoke concentrations, and make a comprehensive judgment on the concentration change trend of the monitoring point; Combined with the principle of determining the membership function of each indicator, the warning level and threshold of each indicator are: Level 1 warning [0, 55) points, Level 2 warning [55, 75) points, Level 3 warning [75, 85) points.

8. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, characterized in that, The AI ​​video image recognition algorithm is specifically as follows: the collected video image is used as input data, and the input image a and image b are converted into a single-channel grayscale image Gray by the geometric mean method, and the expression is: where chn k represents the k-th color channel, K represents the total number of channels used for calculation, and R, G, and B respectively represent the three color channels of the real image; when the input images a and b are at the same moment, an output value of 1 in the Decision Network indicates that the images are the same, otherwise 0 indicates that the images are different; at the input end, the mean superposition map of the grayscale images of images a and b is added, and the mean value of the corresponding elements is calculated: where m and n represent the element value of the m-th column and n-th row of the grayscale image matrix; G avg(m,n) represents the mean value of the element in the m-th column and n-th row of the grayscale image matrix; Based on the principle of information entropy measurement uncertainty, the mean overlay map c is used as input: Among them, H(x) is the difference representation between image a and image b; p(x m,n ) represents the difference in the element values at the same position in the two matrices. The larger the value, the greater the difference in the element values of image a and image b at the m-th column and n-th row; When image a and image b are exactly equal, a = b = c, p(x m,n ) = 0, the information entropy H(X) of the input image is 0. The smaller the information entropy, the smaller the degree of confusion between image a and b, and the higher the similarity between a and b; the larger the information entropy, the greater the degree of confusion between image a and b, and the lower the similarity between a and b; The images a, b, and the mean - superimposed image c are synthesized into a three - channel image, which is input into the convolutional layer of the network to extract features of color, texture, shape, and the topological structure of the image. The input is a three - channel image composed of 3 grayscale images. Each convolutional layer has 3 feature maps. Each neuron in the convolutional layer is connected to multiple neurons in the region with a close position in the previous layer. Each neuron in the convolutional layer is connected to multiple neurons in the region with a close position in the previous layer: Among them, Z l+1 is the output of the (l + 1)-th layer of convolution, that is, the feature map; Z(i, j) is the feature map pixel, is the Kronecker product of tensor operations, w l+1 is the network parameter value of the (l + 1)-th layer, and g is the bias term of the (l + 1)-th layer parameters; during the convolution process, a Drop Block regularization operation is performed on the convolutional layer, and the pixel values of a region block formed by combining adjacent region units in the feature map are set to 0; Among them, γ represents the number of Drop Block elements, the γ probability value controls the amount of features masked, feat_size and block_size represent the feature map and region block sizes, and keep_prob is the retention ratio; The result Z after the convolution operation is output to the fully - connected layer D: Among them, D i represents the output of the i-th neuron in D, and W ij represents the j-th parameter of the i-th neuron. J represents the number of parameters in the neuron. g i is the bias term of the i-th neuron parameter. After performing Dropout in the Decision Network, the output result f(X) of the activation function is obtained, and the model effect is measured through the loss function: Among them, Y is the true value of the sample, S is the number of samples. The squared - loss function combines L2 regularization to evaluate the model effect in the training stage and control the training time.

9. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, characterized in that, The device abnormal voiceprint recognition algorithm specifically includes the following steps: (1) Obtain the idler sound signal and pre - process it. Use wavelet threshold denoising to remove Gaussian white noise. Pre - emphasize the pre - processed sound signal through a high - pass filter. Obtain the time - domain feature one, that is, use short - time average energy, short - time average amplitude, short - time zero - crossing rate, peak - to - peak value, and kurtosis to characterize the signal features, and perform normalization processing. After processing, they are respectively E′, M′, Z′, F′, C′, and then multiply them by the weight coefficients a1, a2, a3, a4, a5 respectively. Then add them up to get the eigenvalue T1 of the time - domain feature one. The expression is as follows: T1 = E′a1 + M′a2 + Z′a3 + F′a4 + C′a5 Among them, the value ranges of a1, a2, a3, a4, a5 are (0, 1); Compare the value of the time - domain feature one with the preset threshold one. If it is greater than the preset threshold one, issue warning one; (2) After performing Fourier transform on the pre - processed signal, calculate the frequency - domain energy. Divide the frequency - domain into 4 sub - bands and calculate the sub - band energy ratio. Extract the resonance peaks in the spectral envelope and calculate the sharpness and 1 / 3 octave eigenvalue. The sharpness S is expressed as follows: Among them, N'(z) is the loudness spectrum on the critical band Z, and the integral of the loudness spectrum over the critical band is the loudness; g(z) is the additional coefficient, Z is the critical band, and d(z) is the differential of the critical band; Perform normalization processing on the frequency - domain energy, sub - band energy ratio, resonance - peak features, sharpness, and 1 / 3 octave features. Multiply the normalized features by the weight coefficients b1, b2, b3, b4, b5 respectively, and then add them up to get the eigenvalue F2 of the frequency - domain feature two. Among them, the value ranges of b1, b2, b3, b4, b5 are (0, 1); When the frequency - domain eigenvalue F2 is greater than the preset threshold two, issue warning two; (3) When both Warning 1 and Warning 2 occur, generate a voiceprint spectrum, separate the voiceprint spectrum to obtain a harmonic component and a shock wave component, that is, perform a short-time Fourier transform STFT on the preprocessed sound signal, and obtain an energy spectrogram after taking the square of the modulus of its value; generate a harmonic component spectrogram and a shock wave component spectrogram through a group of horizontal median filters and a group of vertical median filters respectively for the energy spectrogram; construct a masking matrix with the help of binary masking, divide all time-domain frequency-domain bins in the STFT into either the harmonic component or the shock wave component, and divide the harmonic component and the shock wave component from the original STFT result; finally, perform an inverse short-time Fourier transform iSTFT on the harmonic component and the shock wave component to obtain the harmonic component and the shock wave component of the original signal; Perform MFCC transformation on the shock wave component to obtain Voiceprint Feature 3. When the value of Voiceprint Feature 3 exceeds the preset threshold, confirm that the device is abnormal and issue an abnormal alarm; (4) A device abnormality recognition method based on neural network multi-feature fusion, that is, perform singular value decomposition SVD on the shock wave component separated by HPSS and the MFCC feature matrix respectively to reduce the dimension to a one-dimensional vector, where HPSS represents harmonic-shock wave source separation; then perform normalization to obtain Time-Frequency Feature 3; design a fully connected neural network model with a three-layer network structure, the number of neurons in the hidden layer is 1024, 512, and 256 in sequence, use the relu activation function, and the Adam optimization function to improve the optimization efficiency; use the sigmoid function to activate in the output layer, and use the binary cross-entropy function as the model loss function to perform normal and abnormal binary classification discrimination; fuse Time-Domain Feature 1, Frequency-Domain Feature 2, and Time-Frequency Feature 3 into a new feature 4; Time-Domain Feature 1 includes short-time average energy, short-time average amplitude, short-time zero-crossing rate, peak-to-peak value, and kurtosis; Frequency-Domain Feature 2 includes frequency-domain energy, sub-band energy ratio, formant, sharpness, and 1 / 3 octave; Time-Frequency Feature 3 includes the separated shock wave component and MFCC; input the new feature vector into the fully connected neural network model and output the final intelligent recognition result; in the training of the model, through the backpropagation algorithm and multiple data iterations, fit the best network parameters to optimize the recognition model; When it is recognized as abnormal, output a device abnormal alarm signal. When manually reviewing the abnormal alarm, if it is false, manually calibrate the corresponding abnormal sound signal and add it to the built-in voiceprint feature library; when it is recognized as normal, calculate the similarity between the new feature 4 and the built-in feature using the Minkowski distance. If the similarity exceeds the set threshold, then the match is successful, and a corresponding normal signal or abnormal alarm signal is issued.

10. The belt conveyor fire monitoring and early warning system for coal mines according to claim 1, characterized in that, The multi-modal information fusion analysis model constructs the characteristic gas concentration, smoke concentration, and temperature characteristics into a feature vector x through combination i , extracts them by means of multiple linear regression, and fits them; the expression of multiple linear regression is y2: y2 = β0 + β1x1 + β2x2 + … + β i x i + u Among them, β i represents the weight coefficient, and u represents the bias term; Fuse the output values y of the residual convolution module and the multiple linear regression module through a fully connected layer, expressed as: y = w1y1 + w2y2 where, w1 and w2 represent weight coefficients; y1 is the video image feature; After fitting the parameters through the fully connected layer, obtain the recognition result of the belt conveyor fire monitoring and early warning. When the y value is in [0, 0.3), [0.3, 0.5), it is the initial stage of the fire. When it is in [0.5 - 0.7), it is the development stage of the fire. When it is in [0.7 - 1.0), it is the combustion stage of the fire.

Images