A management system for coal mine monitoring

Abstract

The present invention proposes a management system for coal mine monitoring, including: a monitoring module, an intelligent center, a memory database and a display module; the monitoring module includes a variety of audio and video sensor devices fixed in various production areas of the coal mine, and the monitoring module is used to collect or send audio and video data in real time; the intelligent center retrieves the memory database and performs intelligent analysis on the audio and video data, and sends the monitoring data to the display module; the memory database is used to store various types of user information and production information; the display module is used to display the monitoring data; when there is an alarm content in the monitoring data, the display module sends an alarm instruction to the monitoring module and the user terminal through a wireless network. The present invention uses the intelligent center to perform intelligent analysis on the audio and video data, and conducts real-time monitoring and intelligent analysis on the operation of personnel and equipment in complex environments such as coal mine plants, thereby helping supervisors to discover potential risks in a timely manner to improve safety in the production process.

Description

Technical Field

[0001] The invention belongs to the technical field of intelligent monitoring, and in particular relates to a management system for coal mine monitoring. Background Art

[0002] In the coal mine production process, once a safety accident occurs, it is difficult for production personnel to respond and escape the accident area in the first time. The current supervision is only to view the production area through video cameras, and the illegal operations of production personnel need to be manually identified. This leads to the current monitoring being extremely dependent on high-intensity observation by personnel, and there are certain blind spots in some production sites, which are difficult for supervisors to take into account. In addition, equipment failures are usually discovered by experienced production personnel through active observation, which poses a huge safety hazard. Summary of the invention

[0003] In order to solve the problem that the traditional methods and technologies described in the background technology have defects in production safety supervision, the present invention proposes the following technical solutions:

[0004] A management system for coal mine monitoring, characterized in that it includes: a monitoring module, an intelligent center, a memory database and a display module; the monitoring module includes a variety of audio and video sensor devices fixed in various production areas of the coal mine, and the monitoring module is used to collect or send audio and video data in real time; the intelligent center retrieves the memory database and then performs intelligent analysis on the audio and video data, and sends monitoring data to the display module; the memory database is used to store various types of user information and production information; the display module is used to display the monitoring data; when there is alarm content in the monitoring data, the display module sends an alarm instruction to the monitoring module and the user terminal via a wireless network.

[0005] Among them, after obtaining the audio and video data, the intelligent center needs to pre-process the audio and video data to generate sample data to build a self-learning model; in the process of receiving the audio and video data, the intelligent center generates monitoring data through the self-learning model, and updates and iterates the self-learning model according to the production information in the memory database.

[0006] Furthermore, when performing preprocessing, the intelligent center needs to downsample the audio and video data, reduce the video frame rate, perform noise reduction optimization, and perform frequency domain conversion to form the sample data.

[0007] Furthermore, the intelligent center trains the production information and the sample data in the memory database through a regression model, thereby generating the self-learning model.

[0008] Furthermore, the self-learning model performs the feature extraction on the sample data through a residual time-shift network to obtain a feature vector, and then performs similarity calculation on the feature vector through a similarity algorithm to complete action recognition and generate the monitoring data.

[0009] Furthermore, the user information includes: employee information, risky operation license, violation record and operation rating; the employee information and the risky operation license are uploaded by the user terminal, and the violation record is generated by the intelligent center; the operation rating is generated by the user terminal according to the rating instruction feedback from the display module.

[0010] Furthermore, when displaying the monitoring data, the display module simultaneously calls the internal database and the external database to construct a production area model diagram containing the interface of each audio and video sensor device; when the monitoring data contains alarm content, the display module calls the audio and video sensor device to display the audio and video data and sends the alarm instruction to the corresponding user terminal.

[0011] Beneficial effects: The present invention uses an intelligent center to perform intelligent analysis of audio and video data, and conducts real-time monitoring and intelligent analysis of personnel operations and equipment operation in complex environments such as coal mines, thereby helping supervisors to promptly identify potential risks and improve safety in the production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 The figure is a schematic diagram of the structure of a management system for coal mine monitoring according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solution and advantages of the present application clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0014] It should be understood that the terms "center", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the drawings, and are only for the convenience of describing this patent and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on this patent.

[0015] Figure 1 The figure is a schematic diagram of the structure of a management system for coal mine monitoring according to an embodiment of the present invention.

[0016] Reference Figure 1According to an embodiment of the present invention, a management system for coal mine monitoring includes: a monitoring module 1, an intelligent hub 2, a memory database 3 and a display module 4. The monitoring module 1 includes a variety of audio and video sensor devices fixed in each production area of ​​the coal mine, and the monitoring module 1 is used to collect or send audio and video data in real time. After the intelligent hub 2 retrieves the memory database 3, it performs intelligent analysis on the audio and video data and sends the monitoring data to the display module 4. The memory database 3 is used to store various types of user information and production information. The display module 4 is used to display the monitoring data. When there is alarm content in the monitoring data, the display module 4 sends an alarm instruction to the monitoring module 1 and the user terminal via the wireless network.

[0017] Specifically, the monitoring module 1 is composed of a plurality of audio and video sensor devices connected by a router, and each audio and video sensor device is fixedly arranged in each production area in the coal mine. The display module 4 is provided with an audio and video sensor device interface matching each audio and video sensor device, and the display module 4 can obtain the image and audio information of the corresponding audio and video sensor device in each current area through the audio and video sensor device interface. When the monitoring data contains alarm data content, the display device sends voice data to the corresponding audio and video sensor device through the audio and video sensor device interface after passing through the wireless network, thereby sounding an alarm in the corresponding area. Among them, the alarm sounded by the audio and video sensor device can be a real-time input or system-predetermined voice data type such as fault voice broadcast and manual voice notification.

[0018] Preferably, the display module 4 calls the external database 5 and the internal database at the same time during operation, so as to construct a production area model diagram including the interface of each audio and video sensor device. When the monitoring data contains alarm content, the display module 4 will call the corresponding audio and video sensor device to display the real-time audio and video data and send an alarm instruction to the corresponding user terminal.

[0019] Specifically, the intelligent hub 2 first needs to obtain audio and video data through the monitoring module 1, and the sources of these audio and video data can be diverse. For example, in a coal mine, real-time audio and video data (including video and audio) can be collected through audio and video sensing devices such as cameras and microphones installed at key locations in the production workshop. The camera will capture the operating status of the equipment on the production line, the operation of the workers, and other images, and the microphone will collect the sound generated by the operation of the machine, the speech of the personnel, and other audio content. These original audio and video data are not in a uniform format and contain noise, and need to be further processed before they can be used.

[0020] Smart Hub 2 must first convert these data in different formats into sample data in a standard format to ensure that the subsequent analysis process will not have compatibility issues due to format differences. After obtaining the audio and video data, Smart Hub 2 removes the noise in the sample data (i.e., noise reduction optimization). For video data, there may be interference factors such as snowflakes and flickering light and shadow in the picture, which can be repaired and optimized through filtering algorithms, image enhancement technology, etc., to make the picture clearer and more stable. For audio data, such as background noise, current noise, etc., an audio noise reduction algorithm is used to separate and remove it from the original audio according to the spectral characteristics of the noise, thereby improving the purity of the audio.

[0021] Preferably, since the amount of original audio and video data is very large, in order to facilitate the training of subsequent models and the rational use of computing resources, the intelligent hub 2 will sample the data at certain time intervals or key event nodes (i.e., downsampling processing) and extract representative segments (i.e., reduce the video frame rate). For example, in video data, a clear video segment containing key production operations is captured every few minutes; for audio data, it is segmented according to the semantic paragraphs or periodic laws of the sound to form relatively independent and meaningful audio segments.

[0022] The intelligent center 2 extracts key feature information from the cleaned and segmented audio and video data. These features can reflect the substantive content behind the data. In terms of video, it is possible to extract the texture features of the image, the shape features of the object, the motion trajectory features, etc.; in terms of audio, the frequency features, amplitude features, timbre features, etc. of the sound are extracted. And according to the actual application scenario, these feature data are annotated. For example, in an industrial production scenario, a certain video clip is marked as corresponding to the normal production state or the equipment failure state, and a certain audio feature corresponds to the normal operation sound of the equipment or the abnormal friction sound. Through such annotation, the processed audio and video data is converted into sample data that can be used to build a self-learning model 21 through frequency domain conversion.

[0023] After acquiring the sample data, the intelligent hub 2 constructs a self-learning model 21 based on the sample data using a regression model as a framework. In this embodiment, the regression model uses a convolutional neural network (CNN)-related architecture. In other embodiments, the regression model can also be constructed using, but not limited to, a recurrent neural network (RNN) and its variants such as a long short-term memory network (LSTM), a gated recurrent unit (GRU), or a hybrid architecture model.

[0024] During the construction of the self-learning model 21, the intelligent hub 2 will include the labeled sample data and divide it into a training set, a validation set, and a test set according to a certain ratio. The training set data will be input into the self-learning model 21 to allow the model to learn the mapping relationship between the features in the sample and the labeled results. The model parameters will be continuously adjusted using a suitable optimization algorithm (such as gradient descent, Adam optimizer, etc.) to optimize the performance of the self-learning model 21 on the validation set. Finally, the test set will be used to evaluate the generalization ability of the model to ensure that the model can accurately analyze and judge new and unseen audio and video data. After the self-learning model 21 is constructed, in the process of receiving new audio and video data, the intelligent hub 2 will input these real-time data into the self-learning model 21 that has been constructed. The self-learning model 21 will analyze the input audio and video data based on its learned knowledge and pattern recognition capabilities, and output the corresponding monitoring data. For example, during the production process in a coal mine, the model will determine whether the current equipment is operating normally, whether the workers' operations are in compliance with specifications, etc., and output specific monitoring results in the form of structured data. If equipment failures, operational violations, etc. occur, the monitoring data sent by the self-learning model 21 will contain corresponding alarm content.

[0025] Furthermore, the memory database 3 stores a wealth of production information, which covers detailed data such as past production records, changes in equipment parameters, and operating status under different working conditions (in traffic scenarios, historical traffic flow, accident records, and other information). The intelligent hub 2 will correlate the newly generated monitoring data with this information in the memory database 3 to compare whether there is a deviation between the output results of the current model and the actual production situation. If a deviation is found, it means that the model may need to be further optimized to adapt to the new situation. At this time, the newly acquired audio and video data and the corresponding accurate production information will be re-incorporated into the sample data, and the model will be trained again, and the model parameters will be adjusted so that the model can continuously learn new patterns and features, update and iterate itself, thereby improving the model's adaptability to ever-changing actual scenarios and monitoring accuracy, and continuously improving its performance.

[0026] Specifically, in the process of generating monitoring data through the self-learning network, the action needs to be identified through the residual time-shift network and the similarity algorithm. When the acquired sample data is input into the residual time-shift network, the network will process the data step by step according to its established hierarchical structure. For sample data containing audio and video content, such as video samples, each frame of the video is first regarded as a data point at a time step. The network will process these frame images in sequence along the time dimension, and extract spatial features through convolution operations within each frame. Taking industrial production monitoring videos as an example, the network will capture the appearance of production equipment at different times, the position relationship of components, and other features, as well as the changes of these features over time, such as the movement trajectory of a certain component of the equipment, the change of vibration amplitude, etc. For audio sample data, the frequency, amplitude and other features of the sound are also extracted along the time axis at different time points, such as the pitch change of the sound of the machine running, the fluctuation of sound intensity and other features. Through this multi-dimensional, cross-temporal feature extraction method, the residual time-shifting network ultimately converts the sample data into a feature vector that can comprehensively reflect its essential characteristics. This feature vector covers the key information of the data in space, time, and different modalities (video, audio, etc.), and becomes an important basis for subsequent similarity calculation and action recognition.

[0027] After the feature vector is obtained, it will be similar to the pre-stored standard action feature vector library. This standard action feature vector library is constructed by performing the above feature extraction on a large number of sample data of known actions. Each standard action corresponds to a set of feature vectors, which represents the typical feature performance of the action in different situations. For example, in the coal mine production scene, the feature vector set corresponding to the standard action "opening the machine valve" of a worker operating a certain equipment contains vector representations of hand movement trajectory, body posture changes, and corresponding sound features. When the feature vector extracted in real time is similar to the feature vectors of these standard actions, if the calculated similarity value exceeds the set threshold (this threshold is determined by continuous verification and adjustment during the training phase), it can be determined that the action corresponding to the current sample data matches the standard action, thereby realizing action recognition. For example, if the similarity between the newly input feature vector and the standard feature vector of "opening the machine valve" reaches 0.8 (assuming the threshold is set to 0.7), it can be determined that the currently monitored action is the operation of "opening the machine valve".

[0028] Specifically, a large amount of user information such as employee information, risk operation permits, violation records, and operation ratings are also stored in the internal memory. Among them, employee information and risk operation permits are newly uploaded by the user, and violation records are generated by the self-learning model 21 and recorded in the corresponding employee information. The risk operation permit matches the employee information. After the monitoring personnel records the violation records in the corresponding employee information according to the alarm content, the monitoring personnel regularly feedback the rating instructions through the display module 4 according to the violation records, thereby generating the corresponding operation rating. When the employee's operation rating is unqualified, the risk permit is cancelled, and the employee cannot continue to operate the corresponding equipment. After the self-learning model 21 determines that the employee operates the corresponding equipment, it will issue an alarm content.

[0029] To summarize, the present invention uses an intelligent center to perform intelligent analysis of audio and video data, and conducts real-time monitoring and intelligent analysis of personnel operations and equipment operation in complex environments such as coal mines, thereby helping supervisors to promptly identify potential risks and improve safety in the production process.

[0030] The foregoing describes certain embodiments of the invention. Other embodiments are within the scope of the following claims.

[0031] The terms "exemplary," "example," and the like used throughout this specification mean "used as an example, instance, or illustration" and do not mean "preferred" or "advantageous" over other embodiments. The detailed description includes specific details for the purpose of providing an understanding of the described techniques. However, these techniques can be implemented without these specific details. In some instances, in order to avoid obscuring the concepts of the described embodiments, well-known structures and devices are shown in block diagram form.

[0032] The optional implementation modes of the embodiments of the present invention are described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details in the above implementation modes. Within the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all belong to the protection scope of the embodiments of the present invention.

[0033] The above description of the contents of this specification is provided to enable any person of ordinary skill in the art to implement or use the contents of this specification. Various modifications to the contents of this specification will be apparent to those of ordinary skill in the art, and the general principles defined herein may be applied to other variations without departing from the scope of protection of the contents of this specification. Therefore, the contents of this specification are not limited to the examples and designs described herein, but are consistent with the widest range of principles and novel features disclosed herein.

Claims

1. A management system for coal mine monitoring, characterized in that: include: A monitoring module (1), an intelligent hub (2), a memory database (3) and a display module (4); the monitoring module (1) comprises a plurality of audio and video sensor devices fixed in each production area of ​​a coal mine, and the monitoring module (1) is used to collect or send audio and video data in real time; the intelligent hub (2) retrieves the memory database (3), performs intelligent analysis on the audio and video data, and sends monitoring data to the display module (4); The memory database (3) is used to store various types of user information and production information; the display module (4) is used to display the monitoring data; when there is alarm content in the monitoring data, the display module (4) sends an alarm instruction to the monitoring module (1) and the user terminal via a wireless network.

2. A management system for coal mine monitoring according to claim 1, characterized in that: After acquiring the audio and video data, the intelligent center (2) needs to pre-process the audio and video data to generate sample data to construct a self-learning model (21); in the process of receiving the audio and video data, the intelligent center (2) generates monitoring data through the self-learning model (21), and updates and iterates the self-learning model (21) according to the production information in the memory database (3).

3. A management system for coal mine monitoring according to claim 2, characterized in that: When performing preprocessing, the intelligent center (2) needs to downsample the audio and video data, reduce the video frame rate, optimize noise reduction, and perform frequency domain conversion to form the sample data.

4. A management system for coal mine monitoring according to claim 3, characterized in that: The intelligent center (2) trains the production information and the sample data in the memory database (3) through a regression model, thereby generating the self-learning model (21).

5. A management system for coal mine monitoring according to claim 4, characterized in that: The self-learning model (21) extracts the features of the sample data through a residual time-shift network to obtain a feature vector, and then calculates the similarity of the feature vector through a similarity algorithm to complete action recognition and generate the monitoring data.

6. A management system for coal mine monitoring according to claim 5, characterized in that: The user information includes: employee information, risk operation license, violation record and operation rating; the employee information and the risk operation license are uploaded by the user terminal, and the violation record is generated by the intelligent center (2); the operation rating is generated by the user terminal according to the rating instruction fed back by the display module (4).

7. A management system for coal mine monitoring according to claim 5, characterized in that: When displaying the monitoring data, the display module (4) simultaneously calls the internal database and the external database (5) to construct a production area model diagram including the interface of each audio and video sensor device; when the monitoring data contains alarm content, the display module (4) calls the audio and video sensor device to display the audio and video data and sends the alarm instruction to the corresponding user terminal.

Images