Coal mine water disaster symptom identification method based on video monitoring

Abstract

The invention relates to the technical field of coal mine underground water inrush identification, in particular to a coal mine water disaster symptom identification method based on video monitoring. The method comprises the following steps: arranging a video acquisition device in a water disaster susceptible area, acquiring a mine water disaster monitoring video, and preprocessing the mine water disaster monitoring video to obtain a video frame sequence; constructing a feature labeling model, an image segmentation model and a video tracking model; training the feature labeling model, the image segmentation model and the video tracking model by using a video frame sequence; based on the video frame sequence corresponding to the current moment and the trained feature labeling model, the image segmentation model and the video tracking model, obtaining a water disaster segmentation image at the current moment; the water disaster segmentation image comprises a water spraying area and other areas; and obtaining a water disaster early warning index based on the water spraying area in the water disaster segmentation image, and performing water disaster early warning according to the water disaster early warning index. According to the invention, the accuracy and efficiency of coal mine underground water inrush symptom identification can be improved.

Description

Technical Field

[0001] The invention relates to the technical field of water inrush identification in underground coal mines, and in particular to a method for identifying coal mine water hazard signs based on video monitoring. Background Art

[0002] Mine water disaster is one of the sources of serious and major accidents in coal mines. In order to effectively prevent the occurrence of water disaster accidents, groundwater changes, water level monitoring, water inflow prediction and other methods are usually used for prediction and early warning. Before the occurrence of water inrush, some abnormal phenomena usually appear in the working face and its surroundings. These phenomena are collectively referred to as "water inrush signs". These signs remind us of the impending sudden water disaster. Dynamic real-time monitoring of water inrush accidents is carried out to take effective prevention and control measures. It has important practical significance for ensuring the safe production of coal mines, ensuring the sustainable and stable development of my country's coal industry, and reducing casualties.

[0003] At present, the discovery of water inrush signs mainly relies on manual judgment. However, due to the limited visual range and dim environment in the underground space, it is difficult to detect water inrush signs in time. Therefore, image recognition of coal mine water inrush signs has been applied. Traditional image processing methods have the problems of being easily interfered by underground lighting equipment, poor grayscale feature stability, long recognition time, low recognition rate, and low recognition stability. In the similar field of tunnel leakage recognition, the image segmentation and recognition method based on deep learning has a certain reference significance for the recognition of underground water hazards in coal mines. However, compared with the tunnel environment, the underground environment of coal mines is harsh, with more floating dust, resulting in poor imaging quality. The amount of image data available in coal mines is small, the data quality is poor, and these algorithms are all supervised deep learning models, which require a large amount of high-quality image data for training. In summary, the various problems make it difficult for the above methods to be effectively applied in the actual mine water hazard recognition. Summary of the invention

[0004] In order to solve the above technical problems, the purpose of the present invention is to provide a method for identifying coal mine water hazard signs based on video monitoring. The technical solution adopted is as follows:

[0005] An embodiment of the present invention provides a method for identifying coal mine water hazard signs based on video monitoring, the method comprising:

[0006] A video acquisition device is set up in a flood-prone area to collect mine flood monitoring videos, and the mine flood monitoring videos are pre-processed to obtain a video frame sequence;

[0007] Constructing a feature annotation model, an image segmentation model and a video tracking model; and training the feature annotation model, the image segmentation model and the video tracking model using a video frame sequence;

[0008] Based on the video frame sequence corresponding to the current moment and the trained feature annotation model, image segmentation model and video tracking model, a water damage segmentation image at the current moment is obtained; the water damage segmentation image includes the water-sprinkling area and other areas;

[0009] A water damage warning index is obtained based on the water-sprinkling area in the water damage segmentation image, and a water damage warning is performed according to the water damage warning index.

[0010] Preferably, the mine water hazard monitoring video is preprocessed to obtain a video frame sequence, including:

[0011] The video brightness, frame rate, resolution and aspect ratio of the mine water hazard monitoring video are adjusted, and time information and position information are added to the mine water hazard detection video to obtain a video frame sequence, wherein each frame image in the video frame sequence is a water hazard monitoring image.

[0012] Preferably, constructing a feature annotation model, an image segmentation model and a video tracking model includes:

[0013] The feature annotation model is the DINO self-supervised recognition annotation model; the image segmentation model is the SAM model; and the video tracking model is the XMem model.

[0014] Preferably, the feature annotation model, the image segmentation model and the video tracking model are trained using a video frame sequence, including:

[0015] Obtaining a training set of a feature annotation model, wherein the training set consists of a video frame sequence, and using the training set of the feature annotation model to train the feature annotation model;

[0016] A training set of an image segmentation model is obtained according to the trained feature annotation model and the training set of the feature annotation model, and the image segmentation model is trained using the training set of the image segmentation model;

[0017] A training set of a video tracking model is obtained according to the trained feature annotation model, the training set of the feature annotation model, the image segmentation model and the training set of the image segmentation model, and the video tracking model is trained using the training set of the video tracking model.

[0018] Preferably, obtaining a training set of an image segmentation model according to the trained feature annotation model and the training set of the feature annotation model includes:

[0019] The training set of the feature annotation model is input into the trained feature annotation model to obtain the annotated water hazard image corresponding to each frame of water hazard monitoring image. Each frame of water hazard monitoring image and the corresponding annotated water hazard image constitute the training set of the image segmentation model.

[0020] Preferably, the training set of the video tracking model is obtained according to the trained feature annotation model, the training set of the feature annotation model, the image segmentation model and the training set of the image segmentation model, including:

[0021] The training set of the feature labeling model is input into the feature labeling model to obtain the labeled water damage images corresponding to each video frame sequence to form a labeled video frame sequence; the training set of the image segmentation model is input into the image segmentation model to obtain the first segmented image corresponding to each frame of the labeled water damage image. The labeled video frame sequence and the first segmented image corresponding to the first frame of the labeled water damage image in the labeled video frame sequence are a sample in the training set of the video tracking model.

[0022] Preferably, obtaining a water damage segmentation image at the current moment based on a video frame sequence corresponding to the current moment and a trained feature annotation model, an image segmentation model, and a video tracking model includes:

[0023] Obtain a video frame sequence corresponding to the current moment; input each frame of water hazard monitoring image in the video frame sequence corresponding to the current moment into a trained feature annotation model to obtain an annotated water hazard image corresponding to each frame of water hazard monitoring image, and the annotated water hazard images form an annotated video frame sequence in a time sequence;

[0024] Input the annotated water damage image of each frame in the annotated video frame sequence at the current moment and its corresponding water damage monitoring image into the image segmentation model to obtain the first segmented image respectively; input the first segmented image of the first frame and the annotated video frame sequence at the current moment into the video tracking model to obtain the second segmented image at the current moment;

[0025] A water damage segmentation image at the current moment is obtained according to the first segmentation image and the second segmentation image at the current moment.

[0026] Preferably, obtaining the water damage segmentation image at the current moment according to the first segmentation image and the second segmentation image at the current moment includes:

[0027] The pixel values ​​of two pixels at the same position in the first segmented image and the second segmented image at the current moment are weighted averaged to obtain the water damage segmented image at the current moment.

[0028] Preferably, obtaining a water hazard warning indicator based on the water-sprinkling area in the water hazard segmentation image includes:

[0029] Binarize the water damage segmentation image at the current moment to obtain a binary image, and obtain the number of pixels in the water area in the binary image; obtain the absolute value of the difference in the number of pixels in the water area corresponding to the current moment, and the absolute value of the difference in the number of pixels in the water area corresponding to the current moment is the absolute value of the difference between the number of pixels in the water area corresponding to the current moment and the number of pixels in the water area corresponding to the previous moment; calculate the pixel change rate at the current moment, and the calculation formula is:

[0030]

[0031] Among them, B represents the pixel change rate at the current moment; Δ Q Indicates the absolute value of the difference in the number of pixels in the watering area corresponding to the current moment; Represents the mean of the absolute values ​​of the differences in the number of pixels in the watering area corresponding to each moment;

[0032] The water hazard warning index at the current moment is obtained by taking a weighted sum of the pixel change rate at the current moment and the number of pixels in the watering area.

[0033] Preferably, flood warning is performed according to the flood warning indicators, including:

[0034] A first threshold and a second threshold are set; when the value of the water hazard warning indicator is greater than the first threshold and less than the second threshold, a low-level warning is triggered; when the value of the water hazard warning indicator is greater than or equal to the second threshold, a high-level warning is triggered.

[0035] The embodiments of the present invention have at least the following beneficial effects: the present invention sets a video acquisition device in a water-prone area, collects mine water monitoring videos, and performs preprocessing to obtain a video frame sequence, which can improve the quality of data and make subsequent analysis more accurate; at the same time, the present invention constructs a feature annotation model, an image segmentation model and a video tracking model; the feature annotation model, the image segmentation model and the video tracking model are trained using a video frame sequence, and the use of the feature annotation model can solve the influence of the small amount of available samples on the deep learning model, and the recognition effect is better than that of the traditional algorithm. The combination of the image segmentation model and the video tracking model can improve the efficiency of segmentation and achieve the purpose of real-time monitoring of mine water hazards; finally, a water hazard warning indicator is obtained based on the water-sprinkling area in the water hazard segmentation image, and a water hazard warning is performed according to the water hazard warning indicator to detect and identify signs that may indicate water inrush, thereby achieving the purpose of warning of water hazards in mines and ensuring the safety of mines. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A method flow chart of a method for identifying coal mine water hazard signs based on video monitoring provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the specific implementation method, structure, characteristics and effects of a method for identifying coal mine water hazard signs based on video monitoring proposed by the present invention in combination with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics in one or more embodiments may be combined in any suitable form.

[0039] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0040] The following is a detailed description of a method for identifying coal mine water hazard signs based on video monitoring provided by the present invention in conjunction with the accompanying drawings.

[0041] Example:

[0042] The main application scenarios of the present invention are: Signs of water inrush in coal mines refer to various signals that may indicate sudden floods in coal mines. The identification of signs of water inrush faces many challenges, including the difficulty in obtaining available data, high data annotation costs, difficulty in extracting sign features, insufficient recognition accuracy, and poor timeliness. Increased water dripping is the easiest sign to identify among the signs of water inrush in the roof. This article conducts identification and analysis around the changes in water dripping. Video recognition of signs of water inrush is a technical means for video monitoring of the interior of a coal mine and processing and analyzing the monitoring video in order to timely identify and obtain signs of water inrush, thereby issuing early warning information in a timely manner and taking relevant emergency measures. This technology uses computer vision technology to perform real-time analysis and processing of videos taken by underground coal mine surveillance cameras to detect and identify signs that may indicate water inrush, and to promptly issue alarms to mine managers.

[0043] See also Figure 1 , which shows a method flow chart of a method for identifying coal mine water hazard signs based on video monitoring provided by an embodiment of the present invention, the method comprising the following steps:

[0044] Step S1, setting a video acquisition device in a flood-prone area, collecting mine flood monitoring videos, and preprocessing the mine flood monitoring videos to obtain a video frame sequence.

[0045] Video recognition of water inrush signs is a video monitoring method used to automatically monitor signs of water inrush in coal mines. It usually includes key steps such as monitoring video acquisition, preprocessing, feature extraction, classification and decision-making. It requires real-time video acquisition of key areas prone to water disasters. Areas prone to water disasters include working faces, tunnels and pump rooms, which are recorded as water-prone areas.

[0046] The video acquisition camera is installed in the flood-prone area to collect the mine flood monitoring video in real time, and transmit the collected mine flood monitoring video to the data processing center through the video transmission equipment for further processing. It should be noted that the video data collected by the mine flood monitoring video is the video data from time 0 to the current time.

[0047] Furthermore, it is necessary to pre-process the mine water hazard monitoring video, including adjusting the video brightness, frame rate, resolution and aspect ratio of the mine water hazard monitoring video, and adding time information and location information to the mine water hazard detection video to obtain a video frame sequence, wherein the video frame sequence includes multiple frames of water hazard monitoring images. After pre-processing the mine water hazard monitoring video, a video frame sequence is obtained, which can improve the quality of each frame image in the video frame sequence.

[0048] Step S2, constructing a feature annotation model, an image segmentation model and a video tracking model; and training the feature annotation model, the image segmentation model and the video tracking model using a video frame sequence.

[0049] At present, there are few studies on coal mine water hazard image recognition. Among the existing methods, one method realizes the recognition and early warning of floods by analyzing the grayscale changes between multiple frames of video images. On this basis, a flood image recognition model is established. When the segmented image presents flood texture features, the flood alarm is realized; another method realizes the recognition of floods by maximizing the sum of the minimum distances from the sample image feature vector to the nonlinear regression equation; and the recognition of sudden water is realized by performing Gabor decomposition on the sudden water image in different receptive fields and different directions, and then comparing the similarity between the learned feature vector and the feature vector to be tested. However, these methods are greatly affected by the image quality. In the mine, the image quality is easily disturbed by the underground lighting, resulting in poor image quality and a small number of available samples, which in turn affects the accuracy and efficiency of the recognition results.

[0050] At the same time, due to the harsh environment underground, it is very difficult to obtain high-quality video surveillance data. If traditional supervised deep learning models are used for further analysis, insufficient data will lead to difficulties in converging the deep learning models, which will in turn affect the accuracy of subsequent output results.

[0051] When using deep learning algorithms for subsequent analysis, due to the small amount of qualified data and the difficulty in obtaining a large amount of labeled image data in coal mines, the present invention selects a self-supervised learning method to learn effective recognition features from unlabeled water hazard image data in mines. Construct a feature annotation model, the feature description model is a DINO self-supervised recognition annotation model, DINO is a self-supervised learning model, traditional computer vision models usually need to be trained under manual supervision, which means that labels must be created manually for data training. In contrast, self-supervised learning is a model that can train a model without labeled data.

[0052] The core idea of ​​DINO is to use a teacher network and a student network to learn from each other. The feature annotation model needs to be trained. The training set of the feature annotation model is the water hazard monitoring images of each frame in the video frame sequence. The loss function is the cross entropy loss function. The feature annotation model is trained until the loss function converges and the training is completed. It should be noted that the training set does not only include one video frame sequence, but multiple video frame sequences constitute the training set.

[0053] After the training is completed, the unlabeled flood monitoring image is input, and the model generates multiple enhanced views from the unlabeled flood monitoring image and inputs them into the model for feature extraction. After contrast learning and knowledge distillation, the student model is optimized by the teacher model to output labeled high-quality labeled flood images. One frame of the unlabeled flood monitoring image can get a labeled labeled flood image.

[0054] Furthermore, an image segmentation model is constructed. The images in the mine have small grayscale differences and many noise points. Ordinary image segmentation algorithms are easily affected by this and have poor segmentation effects. The present invention uses the SAM model to segment the images of the watering area. In addition, the model has zero-sample segmentation capabilities, which can also solve the problem of small underground data sets. SAM (Segmentanything model) is the first general image segmentation visual model in the field of computer vision. SAM achieves results comparable to supervised learning by training with ultra-large-scale data sets (billions of levels) and has zero-sample segmentation capabilities. SAM's pre-trained model is universal and can be widely used in various tasks. It also provides it with the ability of zero-sample segmentation, enabling it to accurately identify objects, boundaries and other targets in images, and perform pixel-level segmentation.

[0055] The SAM model is different from traditional segmentation models. Traditional segmentation models usually can only accept the original image as input and output fixed segmentation results. SAM can accept the original image and specific prompts (such as points, boxes, shadows, text) as input at the same time, and generate different segmentation result images according to different prompts. The SAM model consists of three parts: image encoder module, prompt encoder module, and segmentation mask decoder module.

[0056] The SAM model mainly uses a dynamically scaled cross entropy loss function focal loss L F To supervise mask prediction, we adjust the factor (1-p t )γ, which can dynamically reduce the weights of samples that are easy to distinguish during training. The formula is:

[0057]

[0058] Where y is the representative value of foreground or background, with values ​​of 1 and -1, representing foreground and background respectively; p is the probability of the model predicting that it belongs to the foreground, with a value range of 0 to 1; p t The model predicts the probability of a certain category; γ ≥ 0 is an adjustable focusing parameter. The larger the γ value, the more the model focuses on samples that are difficult to classify.

[0059] It should be noted that, although the pre-trained model of the image segmentation model is universal and can adapt to various tasks, in order to better capture the information of the water hazard image under the mine, the present invention also needs to train the image segmentation model. The training set of the image segmentation model consists of the annotated water hazard images obtained by the feature annotation model and the corresponding unlabeled water hazard monitoring images, that is, the training set of the image segmentation model is obtained according to the trained feature annotation model and the training set of the feature annotation model;

[0060] Specifically, the training set of the feature annotation model is input into the trained feature annotation model to obtain the annotated water damage image corresponding to each frame of the water damage monitoring image. Each frame of the water damage monitoring image and the corresponding annotated water damage image constitute the training set of the image segmentation model. The input of the image segmentation model is a frame of the water damage monitoring image and its corresponding annotated water damage image, wherein the annotated water damage image provides segmentation hints for the SAM model. The output of the water damage image segmentation model is the first segmented image.

[0061] Finally, a video tracking model needs to be built. Common video object segmentation methods include the fully convolutional network (FCN) based on deep learning, the spatiotemporal 3D convolutional network, and the long short-term memory network (LSTM). In order to achieve real-time dynamic tracking of the video, the XMem model was selected as the video tracking model. Compared with other video segmentation methods, the XMem model has a faster reading and writing speed and is more suitable for real-time analysis and processing of data. It has the advantages of real-time, accuracy, and wide applicability. XMem is a long video target segmentation model based on a multiple memory model. Its working principle is that by first giving a picture and a target mask, XMem tracks the target and generates the corresponding mask map. Specifically, different feature storage pools are first initialized with input, including long-term memory, working memory, and sensitive memory. Subsequently, for each frame of the image, features are read from these feature storage pools and used to generate the segmentation mask of the target. At the same time, each feature storage pool is updated at different frequencies.

[0062] XMem consists of three end-to-end trainable convolutional networks: a query encoder that extracts query-specific image features, a decoder that takes the output of the memory read step to produce an object mask, and a value encoder that combines the image with the object mask to extract new memory features.

[0063] Among them, the XMem model also needs to be trained. The training set of the video tracking model is obtained according to the trained feature annotation model, the training set of the feature annotation model, the image segmentation model and the training set of the image segmentation model. The video tracking model is trained using the training set of the video tracking model. Specifically, the training set of the feature annotation model is input into the feature annotation model to obtain the annotated water damage images corresponding to each video frame sequence to form an annotated video frame sequence; the training set of the image segmentation model is input into the image segmentation model to obtain the first segmented image corresponding to each frame of the water damage monitoring image, that is, the first segmented image corresponding to each frame of the annotated water damage image. The annotated video frame sequence and the first segmented image corresponding to the first frame of the annotated water damage image in the annotated video frame sequence are a sample in the training set of the video tracking model; after obtaining the training set, the video tracking model is trained. The input of the XMem model is the annotated video frame sequence and the first segmented image corresponding to the first frame of the annotated water damage image in the annotated video frame sequence.

[0064] This completes the training of the feature annotation model, image segmentation model and video tracking model.

[0065] Step S3, obtaining the water damage segmentation image at the current moment based on the video frame sequence corresponding to the current moment and the trained feature annotation model, image segmentation model and video tracking model.

[0066] The video frame sequence corresponding to the current moment is further collected. It should be noted that the video frame sequence here has been preprocessed.

[0067] Input each frame of water hazard monitoring image in the video frame sequence corresponding to the current moment into the trained feature annotation model, and obtain the annotated water hazard image corresponding to each frame of water hazard monitoring image, and the annotated water hazard image forms the annotated video frame sequence at the current moment in chronological order; input each frame of annotated water hazard image and its corresponding water hazard monitoring image in the annotated video frame sequence at the current moment into the image segmentation model, and obtain the first segmented image respectively; input the first segmented image of the first frame and the annotated video frame sequence into the video tracking model, and obtain the second segmented image at the current moment. It should be noted that the video tracking model and the image segmentation model obtain not only the first segmented image and the second segmented image at the current moment, but also the first frame, the second frame, and the third frame of the water hazard monitoring image will also obtain the corresponding first segmented image and the second segmented image, and the video tracking model outputs a second segmented image of one frame each time. Therefore, the first segmented image and the second segmented image corresponding to the current moment (current frame) and each moment before the current moment can be obtained.

[0068] The water damage segmentation image at the current moment is obtained according to the first segmentation image at the current moment and the second segmentation image at the current moment. Specifically, the first segmentation image and the second segmentation image at the current moment are fused pixel by pixel. The pixel-level weighted average or logical operation can be used to merge their outputs. The pixel value of each identified pixel position can calculate the weighted average of the outputs of the two models. The weights can be determined according to the performance of the model on the training set, and are recorded as W1 and W2 respectively. That is, the pixel value of the pixel point of the water damage segmentation image is obtained by the weighted average of the pixel values ​​of the two pixel points at the same position on the two images. The water damage segmentation images at other moments are obtained in the same way.

[0069] It should be noted that, since the image is frame by frame, the current moment is also the current frame, where the video frame sequence corresponding to the current moment is the video sequence from 0 to the current moment. For example, if the current moment is moment t, the length of the video frame sequence at the current moment is from 0 to t frames. The water damage segmentation image includes the water area and other areas.

[0070] Step S4, obtaining a water hazard warning index based on the flooded area in the water hazard segmentation image, and performing a water hazard warning according to the water hazard warning index.

[0071] Binarize the water damage segmentation image at the current moment to obtain a binary image, and obtain the number of pixels in the water area in the binary image; obtain the absolute value of the difference in the number of pixels in the water area corresponding to each moment, the so-called absolute value of the difference in the number of pixels in the water area corresponding to each moment is the absolute value of the difference between the number of pixels in the water area corresponding to the next moment and the number of pixels in the water area corresponding to the previous moment, and then use the same calculation method to obtain the absolute value of the difference in the number of pixels in the water area corresponding to the current moment; then calculate the pixel change rate at the current moment, and the calculation formula is:

[0072] Among them, B represents the pixel change rate at the current moment; Δ Q Indicates the absolute value of the difference in the number of pixels in the watering area corresponding to the current moment; Represents the mean of the absolute values ​​of the differences in the number of pixels in the watering area at each moment.

[0073] The weighted sum of the pixel change rate at the current moment and the number of pixels in the water sprinkling area is used to obtain the water hazard warning index at the current moment. The calculation formula is: P = ω1 × pixel change rate + ω2 × number of pixels in the water sprinkling area

[0074] Among them, ω1 and ω2 are weight coefficients. According to experience, the initial values ​​are set to ω1=0.8 and ω2=0.2. The weight coefficients can be adjusted based on historical data analysis and actual application conditions.

[0075] The first threshold β1 and the second threshold β2 are set according to the value of the water hazard warning indicator and the average of historical data. The first threshold β1 is a low threshold to monitor minor anomalies or potential warning signals; the second threshold β2 is a high threshold to monitor serious anomalies that require immediate processing.

[0076] When the value of the water hazard warning indicator is greater than the first threshold and less than the second threshold, a low-level warning is triggered; when the value of the water hazard warning indicator is greater than or equal to the second threshold, a high-level warning is triggered.

[0077] The first threshold β1 and the second threshold β2 can be adjusted according to historical data analysis and actual application to improve the accuracy and reliability of the early warning system. Regular optimization is performed to ensure effectiveness under different conditions.

[0078] The present invention analyzes the signs of water inrush before the occurrence of coal mine water disasters, combines the current industry needs and technological development, proposes a mine water disaster identification method based on image segmentation technology, and uses the pixel point change trend for early warning. It has a reference significance for the prediction and early warning of coal mine water disasters. This method does not require the use of a large number of data sets for training, and effectively solves the problem of the small number of water inrush images in coal mines. Compared with traditional algorithms, the recognition effect is better, and the degree of pixel point change is used for prediction and early warning. It is compared and verified with the existing water inrush warning indicators. The maximum value point of the pixel point change rate is located before the highest point of the water inrush change rate. At present, this method only recognizes and verifies the water sprinkling situation. In the future, the recognition of other signs of water inrush can be added to make the recognition method more accurate and effective.

[0079] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The above is a description of a specific embodiment of this specification. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0080] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0081] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying coal mine water hazard signs based on video monitoring, characterized in that: The method includes: A video acquisition device is set up in a flood-prone area to collect mine flood monitoring videos, and the mine flood monitoring videos are pre-processed to obtain a video frame sequence; Constructing a feature annotation model, an image segmentation model and a video tracking model; and training the feature annotation model, the image segmentation model and the video tracking model using a video frame sequence; Based on the video frame sequence corresponding to the current moment and the trained feature annotation model, image segmentation model and video tracking model, a water damage segmentation image at the current moment is obtained; the water damage segmentation image includes the water-sprinkling area and other areas; A water damage warning index is obtained based on the water-sprinkling area in the water damage segmentation image, and a water damage warning is performed according to the water damage warning index.

2. A method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The preprocessing of the mine water hazard monitoring video to obtain a video frame sequence includes: The video brightness, frame rate, resolution and aspect ratio of the mine water hazard monitoring video are adjusted, and time information and position information are added to the mine water hazard detection video to obtain a video frame sequence, wherein each frame image in the video frame sequence is a water hazard monitoring image.

3. The method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The construction of the feature annotation model, the image segmentation model and the video tracking model includes: The feature annotation model is the DINO self-supervised recognition annotation model; the image segmentation model is the SAM model; and the video tracking model is the XMem model.

4. The method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The method of using the video frame sequence to train the feature annotation model, the image segmentation model and the video tracking model includes: Obtaining a training set of a feature annotation model, wherein the training set consists of a video frame sequence, and using the training set of the feature annotation model to train the feature annotation model; A training set of an image segmentation model is obtained according to the trained feature annotation model and the training set of the feature annotation model, and the image segmentation model is trained using the training set of the image segmentation model; A training set of a video tracking model is obtained according to the trained feature annotation model, the training set of the feature annotation model, the image segmentation model and the training set of the image segmentation model, and the video tracking model is trained using the training set of the video tracking model.

5. A method for identifying coal mine water hazard signs based on video monitoring according to claim 4, characterized in that: The step of obtaining a training set of an image segmentation model based on the trained feature annotation model and the training set of the feature annotation model includes: The training set of the feature annotation model is input into the trained feature annotation model to obtain the annotated water hazard image corresponding to each frame of water hazard monitoring image. Each frame of water hazard monitoring image and the corresponding annotated water hazard image constitute the training set of the image segmentation model.

6. A method for identifying coal mine water hazard signs based on video monitoring according to claim 4, characterized in that: The method of obtaining a training set of a video tracking model based on the trained feature annotation model, the training set of the feature annotation model, the image segmentation model, and the training set of the image segmentation model includes: The training set of the feature labeling model is input into the feature labeling model to obtain the labeled water damage images corresponding to each video frame sequence to form a labeled video frame sequence; the training set of the image segmentation model is input into the image segmentation model to obtain the first segmented image corresponding to each frame of the labeled water damage image. The labeled video frame sequence and the first segmented image corresponding to the first frame of the labeled water damage image in the labeled video frame sequence are a sample in the training set of the video tracking model.

7. A method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The method of obtaining the water damage segmentation image at the current moment based on the video frame sequence corresponding to the current moment and the trained feature annotation model, image segmentation model and video tracking model includes: Obtain a video frame sequence corresponding to the current moment; input each frame of water hazard monitoring image in the video frame sequence corresponding to the current moment into a trained feature annotation model to obtain an annotated water hazard image corresponding to each frame of water hazard monitoring image, and the annotated water hazard images form an annotated video frame sequence in a time sequence; Input the annotated water damage image of each frame in the annotated video frame sequence at the current moment and its corresponding water damage monitoring image into the image segmentation model to obtain the first segmented image respectively; input the first segmented image of the first frame and the annotated video frame sequence at the current moment into the video tracking model to obtain the second segmented image at the current moment; A water damage segmentation image at the current moment is obtained according to the first segmentation image and the second segmentation image at the current moment.

8. A method for identifying coal mine water hazard signs based on video monitoring according to claim 7, characterized in that: The method of obtaining the water damage segmentation image at the current moment according to the first segmentation image and the second segmentation image at the current moment comprises: The pixel values ​​of two pixels at the same position in the first segmented image and the second segmented image at the current moment are weighted averaged to obtain the water damage segmented image at the current moment.

9. A method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The method of obtaining a water hazard warning indicator based on the water-sprinkling area in the water hazard segmentation image includes: Binarize the water damage segmentation image at the current moment to obtain a binary image, and obtain the number of pixels in the water area in the binary image; obtain the absolute value of the difference in the number of pixels in the water area corresponding to the current moment, and the absolute value of the difference in the number of pixels in the water area corresponding to the current moment is the absolute value of the difference between the number of pixels in the water area corresponding to the current moment and the number of pixels in the water area corresponding to the previous moment; calculate the pixel change rate at the current moment, and the calculation formula is: Among them, B represents the pixel change rate at the current moment; Δ Q Indicates the absolute value of the difference in the number of pixels in the watering area corresponding to the current moment; Represents the mean of the absolute values ​​of the differences in the number of pixels in the watering area corresponding to each moment; The water hazard warning index at the current moment is obtained by taking a weighted sum of the pixel change rate at the current moment and the number of pixels in the watering area.

10. The method for identifying coal mine water hazard signs based on video monitoring according to claim 1, characterized in that: The flood warning according to the flood warning indicators includes: A first threshold and a second threshold are set; when the value of the water hazard warning indicator is greater than the first threshold and less than the second threshold, a low-level warning is triggered; when the value of the water hazard warning indicator is greater than or equal to the second threshold, a high-level warning is triggered.

Images