Mine water inrush identification method and device, electronic equipment and storage medium

Abstract

According to the mine water inrush identification method and device, the electronic equipment and the storage medium, the preprocessed mine video frame data serves as the input of the initial model, and on the premise that the background influence is eliminated, the distinguishing capacity of the water flow dynamic characteristics and the static background can be enhanced. Moreover, the initial model comprises a residual shrinkage layer and a convolution block attention layer, thereby enhancing the attention of the water flow features and the inhibition of background interference, and remarkably improving the performances in water flow edge detection and complex background segmentation. Besides, the mask of the intermediate video frame data is used as an actual mine water inrush label to carry out back propagation adjustment on the initial model to obtain a video segmentation model, and the identification capability of the dynamic characteristics of the water flow is further enhanced, so that the video segmentation model can timely and accurately execute mine water inrush identification in a mine changeable environment.

Description

Technical Field

[0001] The present application relates to the field of mine safety technology, and in particular to a mine water inrush identification method, device, electronic equipment and storage medium. Background Art

[0002] Coal can provide necessary energy and raw materials for industrial production, power generation and heating, so the mining of coal resources is very important. However, mine water inrush is the main disaster for mine production and safety. The frequent occurrence of mine water inrush disasters has brought direct threats to the safe production of coal mines.

[0003] Based on the above situation, the mine water inrush identification method in the prior art cannot determine the mine water inrush area in a timely and accurate manner. Summary of the invention

[0004] In view of this, the purpose of this application is to propose a mine water inrush identification method, device, electronic device and storage medium to solve the above-mentioned technical problems.

[0005] Based on the above objectives, the first aspect of the present application provides a method for identifying water inrush in a mine, comprising:

[0006] Obtaining preprocessed mine video frame data;

[0007] Inputting the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer;

[0008] Performing feature extraction on the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data;

[0009] Using the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data;

[0010] Performing up-sampling convolution processing on the pre-processed mine video frame data through the second sub-residual contraction layer to obtain a first up-sampling convolution feature;

[0011] The adjusted multi-scale feature data and the up-sampled convolution feature are spliced ​​to obtain a training prediction mine water inrush area;

[0012] Determine intermediate video frame data from the preprocessed mine video frame data, and obtain actual mine water inrush labels of the intermediate video frame data;

[0013] Using the actual mine water inrush label and the training predicted mine water inrush area to perform back propagation adjustment on the initial model to obtain a video segmentation model;

[0014] Obtain the mine video frame data to be predicted;

[0015] The video frame data of the mine to be predicted is input into the video segmentation model, and the video frame data of the mine to be predicted is identified and predicted by the video segmentation model to obtain the predicted mine water inrush area.

[0016] Optionally, obtaining the pre-processed mine video frame data includes:

[0017] Collect video data sets inside mines;

[0018] Performing video analysis on the video data set to obtain a plurality of mine video frame data;

[0019] Multi-channel residual processing is performed on the multiple mine video frame data to obtain pre-processed mine video frame data.

[0020] Optionally, performing multi-channel residual processing on the plurality of mine video frame data to obtain pre-processed mine video frame data includes:

[0021] Obtain image channel data of each pixel position in each mine video frame data;

[0022] The image channel data corresponding to the pixel position of each mine video frame data is processed by a multi-channel average algorithm to obtain the average value of each pixel in each image channel data on the time axis;

[0023] Determine the intermediate video frame data of the plurality of mine video frame data, and perform difference processing using the intermediate video frame data and the average value to obtain a multi-channel residual image;

[0024] Determine a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data;

[0025] Performing feature extraction on the multi-channel residual image to obtain a first feature image;

[0026] Performing product processing on the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image to obtain a feature image subjected to product processing;

[0027] The pixel values ​​at the corresponding pixel positions of the fused feature image and the feature image subjected to product processing are summed to obtain preprocessed mine video frame data.

[0028] Optionally, the image channel data based on the pixel positions corresponding to the respective mine video frame data are processed by a multi-channel average value algorithm to obtain the average value of each pixel in the respective image channel data on the time axis, including:

[0029] Based on the image channel data corresponding to the pixel position of each mine video frame data, the average value of each pixel in each image channel data on the time axis is determined by the following formula, wherein the image channel data includes first channel data, second channel data and third channel data:

[0030]

[0031] in, Represents the average value of each pixel in the first channel data on the time axis. Represents the average value of each pixel in the second channel data on the time axis. Represents the average value of each pixel in the third channel data on the time axis, I i (x, y) 1 Represents the first channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 2 Represents the second channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 3 Represents the third channel data of the pixel position corresponding to the mine video frame data, t represents the total number of mine video frame data, and i represents the order of the number of mine video frame data.

[0032] Optionally, determining a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data comprises:

[0033] Extracting features from each mine video frame data to obtain a plurality of second feature images;

[0034] Processing the pixel values ​​of the plurality of second feature images at each pixel position by a splicing feature algorithm to obtain a splicing feature image;

[0035] The spliced ​​feature images are fused to obtain a fused feature image.

[0036] Optionally, the processing of the pixel values ​​at each pixel position of the plurality of second feature images by a splicing feature algorithm to obtain a spliced ​​feature image includes:

[0037] Based on the pixel values ​​of the plurality of second feature images at each pixel position, the spliced ​​feature image is determined by the following formula:

[0038]

[0039] Among them, O(i, j, k) represents the value of the spliced ​​feature image at the pixel position (i, j, k), A(i, j, k) represents the pixel value of the first second feature map in multiple second feature maps at the pixel position (i, j, k), B(i-8, j, k) represents the pixel value of the second second feature map in multiple second feature maps at the pixel position (i, j, k), and C(i, j, k) represents the pixel value of the third second feature map in multiple second feature maps at the pixel position (i, j, k).

[0040] Optionally, the fusing the spliced ​​feature images to obtain a fused feature image includes:

[0041] Performing down-sampling convolution processing on the spliced ​​feature image to obtain down-sampling convolution features;

[0042] Performing up-sampling convolution processing on the spliced ​​feature image to obtain a second up-sampling convolution feature;

[0043] The down-sampled convolution feature and the second up-sampled convolution feature are subjected to feature fusion to obtain a fused feature image.

[0044] Based on the same inventive concept, the second aspect of the present application provides a mine water inrush identification device, comprising:

[0045] A training acquisition module is configured to acquire pre-processed mine video frame data;

[0046] A training input module is configured to input the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer;

[0047] a feature extraction module configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data;

[0048] A feature adjustment module is configured to perform feature adjustment on the multi-scale feature data using the convolutional block attention layer to obtain adjusted multi-scale feature data;

[0049] an upsampling module, configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain a first upsampling convolution feature;

[0050] A splicing processing module is configured to splice the adjusted multi-scale feature data and the up-sampled convolution feature to obtain a training prediction mine water inrush area;

[0051] An intermediate frame determination module is configured to determine intermediate video frame data from the preprocessed mine video frame data, and obtain an actual mine water inrush label of the intermediate video frame data;

[0052] A back propagation module is configured to use the actual mine water inrush label and the training predicted mine water inrush area to perform back propagation adjustment on the initial model to obtain a video segmentation model;

[0053] A prediction acquisition module is configured to acquire the mine video frame data to be predicted;

[0054] The identification and prediction module is configured to input the mine video frame data to be predicted into the video segmentation model, identify and predict the mine video frame data to be predicted through the video segmentation model, and obtain the predicted mine water inrush area.

[0055] Based on the same inventive concept, the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executed on the processor, wherein the processor implements the method described in the first aspect above when executing the computer program.

[0056] Based on the same inventive concept, the fourth aspect of the present application provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method described in the first aspect above.

[0057] From the above, it can be seen that the mine water inrush identification method, device, electronic device and storage medium provided by the present application can enhance the ability to distinguish the dynamic characteristics of water flow from the static background under the premise of eliminating the background influence by taking the preprocessed mine video frame data as the input of the initial model. And the initial model includes a residual shrinkage layer and a convolutional block attention layer, which enhances the attention to water flow characteristics and the suppression of background interference, and significantly improves the performance in water flow edge detection and complex background segmentation. In addition, by using the mask of the intermediate video frame data as the actual mine water inrush label, the initial model is back-propagated and adjusted to obtain a video segmentation model, which further enhances the recognition ability of water flow dynamic characteristics, thereby ensuring that the video segmentation model can perform mine water inrush identification in a timely and accurate manner under the changing environment of the mine. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the present application or related technologies, the drawings required for use in the embodiments or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0059] Figure 1 A schematic diagram of a flow chart of a method for identifying water inrush in a mine according to an embodiment of the present application;

[0060] Figure 2A The original image segmentation model (U 2 Net) model diagram;

[0061] Figure 2B The video segmentation model (CBAM-U 2 Net) schematic diagram;

[0062] Figure 2C A schematic diagram of multi-channel residual preprocessing in an embodiment of the present application;

[0063] Figure 2D This is a schematic diagram of marking an intermediate frame according to an embodiment of the present application;

[0064] Figure 2E A schematic diagram of the detection result of a water hazard risk video according to an embodiment of the present application;

[0065] Figure 2F A schematic diagram of a mine water inrush identification process according to an embodiment of the present application;

[0066] Figure 2G A schematic diagram of a preprocessing process for a multi-channel residual attention module according to an embodiment of the present application;

[0067] Figure 3 This is a structural block diagram of a mine water inrush identification device according to an embodiment of the present application;

[0068] Figure 4 A schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0070] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0071] It is understandable that before using the technical solutions of each embodiment of the present application, the type, scope of use, usage scenarios, etc. of the personal information involved will be informed to the user in an appropriate manner, and the user's authorization will be obtained.

[0072] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly remind the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to the electronic device, application, server, storage medium or other software or hardware that performs the operation of the technical solution of the present application according to the prompt message.

[0073] As an optional but non-limiting implementation, in response to receiving the user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0074] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation method of this application. Other methods that meet relevant laws and regulations may also be applied to the implementation method of this application.

[0075] Complex geological conditions make mine water disasters more likely to occur. Once a mine water disaster occurs, it often causes serious economic losses, long rescue efforts and widespread social impact. Its high incidence and suddenness still pose a major threat to mine safety.

[0076] The three processes of water inrush risk identification, namely the initial stage from nothing to something, the expansion stage from small to large, and the uncontrolled stage exceeding the mine drainage capacity, are the core processes of the gradual evolution of water hazards. Therefore, identifying the dynamic changes of water inrush is crucial in mine water hazard prevention and control.

[0077] The present application finds that one of the reasons why the mine water inrush identification method in the prior art cannot accurately determine the mine water inrush area in a timely manner is that the existing mine water inrush identification method mainly relies on sensor monitoring and manual inspection. This has certain limitations in practical applications. For example, sensor monitoring is easily affected by the complex environment of the mine and may be damaged in an accident, resulting in unstable and unreliable monitoring. In addition, electromagnetic interference in the mine may also affect the data transmission accuracy of the sensor. Manual inspection requires a large amount of human resources, and when a water disaster occurs, the inspectors may need to evacuate urgently, making it impossible to provide timely information in a timely manner.

[0078] The embodiment of the present application provides a method for identifying water inrush in a mine. By using the preprocessed mine video frame data as the input of the initial model, the ability to distinguish the dynamic characteristics of the water flow from the static background can be enhanced under the premise of eliminating the background influence. And the initial model includes a residual shrinkage layer and a convolutional block attention layer, which enhances the attention to the water flow characteristics and the suppression of background interference, and significantly improves the performance in water flow edge detection and complex background segmentation. In addition, by using the mask of the intermediate video frame data as the actual mine water inrush label, the initial model is back-propagated and adjusted to obtain a video segmentation model, which further enhances the recognition ability of the dynamic characteristics of the water flow, thereby ensuring that the video segmentation model can perform mine water inrush identification in a timely and accurate manner under the changing environment of the mine.

[0079] like Figure 1 As shown, the method of this embodiment includes:

[0080] Step 101, obtaining pre-processed mine video frame data.

[0081] In this step, sensors, cameras and other equipment can be installed in the mine environment to capture video data such as water level changes and geological structure changes in the mine. The captured video data is then preprocessed, such as image enhancement (such as brightness adjustment, contrast enhancement), noise removal, size adjustment, etc., to improve data quality and make it more suitable for subsequent model processing.

[0082] Step 102, inputting the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer.

[0083] In this step, the residual U-block (RSU) layer is a structure in deep learning that combines residual learning and attention mechanism (Shrinkage Strategy). Residual learning allows the network to learn the residual between input and output, which helps solve the gradient vanishing problem in deep neural networks and enables the network to learn deeper features. The "shrinkage" part introduces the attention mechanism to weight the input features and suppress unimportant features, thereby enhancing the representation ability of the model.

[0084] Convolutional Block Attention Modul (CBAM) is a lightweight attention layer designed to improve the representation capability of Convolutional Neural Networks (CNN). CBAM emphasizes important features in the input image and suppresses unimportant features by sequentially applying channel attention and spatial attention. This mechanism helps the model focus more on information that is useful for the task, thereby improving performance.

[0085] The initial model is used to process pre-processed mine video frame data by combining residual shrinkage layers and convolutional block attention layers. This can improve the model's ability to capture key information in the mine environment, thereby supporting more accurate monitoring and analysis tasks.

[0086] like Figure 2A and Figure 2B As shown, the original image segmentation network (U 2 Net) model. For each RSU layer, it consists of a U-shaped structure composed of multiple convolutional blocks of Conv2d+BN+ReLU structure. After the output of each convolutional module on the left (i.e., the first sub-residual shrinkage layer), a convolutional block attention (CBAM) layer is added to output the feature map after CBAM processing.

[0087] Step 103: extracting features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data.

[0088] In this step, the preprocessed mine video frame data is processed using the first sub-residual shrinkage layer to extract useful multi-scale feature data from the data.

[0089] Multi-scale feature data refers to feature data extracted at different scales (or resolutions).

[0090] Due to the complex environment of mines, features at different scales may provide complementary information about scenes or objects. For example, at a larger scale, the network may focus on the overall structure and layout, while at a smaller scale, it may focus on details and textures. By combining these multi-scale features, the model's ability to understand and analyze the content of mine videos can be improved.

[0091] Step 104: Use the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data.

[0092] In this step, the multi-scale feature data is processed using CBAM to enhance or suppress certain parts of the feature map.

[0093] Specifically, CBAM reweights these features according to the importance of input features (adaptively adjusting weights by measuring attention weights) through channel and spatial attention mechanisms, thereby highlighting key features such as water flow and suppressing interference information in the background. This adjustment helps the model to more effectively perform hierarchical feature extraction and fine segmentation of water flow areas.

[0094] Step 105: performing up-sampling convolution processing on the pre-processed mine video frame data through the second sub-residual shrinkage layer to obtain a first up-sampling convolution feature.

[0095] In this step, in the upsampling convolution process, upsampling is achieved by applying a convolution operation (i.e., using a set of filters or kernels to slide over the image and perform a dot product operation).

[0096] In convolutional neural networks (CNNs), upsampling convolutional layers are often used to enlarge the size of feature maps to the same size as the input image or other feature maps for subsequent comparison or fusion operations.

[0097] Upsampled convolution features refer to feature maps obtained through upsampled convolution layer processing. This processing can generate more refined image features, which contain higher-level image information, such as edges, textures, or partial structures of objects. This information helps to improve accuracy in subsequent analysis or recognition prediction tasks.

[0098] Step 106, concatenating the adjusted multi-scale feature data and the up-sampled convolutional features to obtain a training prediction of a mine water inrush area.

[0099] In this step, the splicing process is the process of merging two or more data sets or feature maps into a larger set or feature map. In the context of mine water inrush area prediction, splicing the adjusted multi-scale feature data and upsampled convolution features means merging these two features in a certain dimension (such as the channel dimension) to form a feature set containing richer information, which represents the area where water inrush may occur in the mine.

[0100] like Figure 2B As shown in the figure, the feature map processed by the CBAM layer (i.e., the adjusted multi-scale feature data) is concatenated with the feature map in the upsampling path on the right side of the RSU layer (i.e., the second sub-residual shrinkage layer) (i.e., the upsampled convolution feature). The feature map on the right side (i.e., the upsampled convolution feature) is generated by the convolution operation of upsampling to restore the spatial resolution.

[0101] For U 2 A CBAM layer is embedded between each RSU layer on the left side of the Net, and is concatenated with each RSU layer on the right side.

[0102] Step 107, determining the intermediate video frame data from the preprocessed mine video frame data, and obtaining the actual mine water inrush label of the intermediate video frame data.

[0103] In this step, in the preprocessed mine video frame data, a frame or frames in the middle of the video sequence are determined as the intermediate video frame data according to information such as timestamp or frame number. This process may need to take into account factors such as the total number of frames and frame rate of the video.

[0104] For the extracted intermediate video frame data, it is necessary to obtain the corresponding actual mine water inrush labels. The actual mine water inrush labels refer to the real data about mine water inrush events obtained through observation, recording or other means in the actual mine environment. These data are usually labeled as "water inrush" or "non-water inrush", forming a labeled data set. These labels are the "correct answers" in the model training process, which are used to guide the model to learn how to distinguish between water inrush areas and non-water inrush areas.

[0105] Step 108, using the actual mine water inrush label and the training predicted mine water inrush area to perform back-propagation adjustment on the initial model to obtain a video segmentation model.

[0106] In this step, when the training predicted mine water inrush area is inconsistent with the actual mine water inrush label, it is necessary to adjust the model parameters through the backpropagation algorithm. Backpropagation reduces the risk of information loss in continuous frame processing by calculating the prediction error (that is, the difference between the training predicted mine water inrush area and the actual mine water inrush label). By comparing the features of the intermediate frames, the model adjusts the network weights to optimize the ability to capture the continuous dynamic changes of the water flow. This error is then backpropagated back to each layer of the network to update the network weights and bias parameters. This process aims to minimize the prediction error and make the model's predictions more accurate.

[0107] After multiple iterations of training (i.e., multiple predictions, error calculations, back propagation, and parameter adjustments), the model's performance will gradually improve until it reaches a satisfactory level of accuracy. Ultimately, this trained model is called a video segmentation model (CBAM-U2Net) because it can process video data and segment each frame in the video into water inrush areas and non-water inrush areas. Such a model can be used to monitor mine conditions in real time in practical applications and can accurately segment and identify areas with potential water inrush risks.

[0108] Step 109, obtaining the mine video frame data to be predicted.

[0109] In this step, sensors, cameras and other equipment can be installed in the mine environment to capture video data such as water level changes and geological structure changes in the mine, which are segmented into a series of video frames of the mine to be predicted (i.e., continuous static images). These video frames contain various information inside the mine, including possible water inrush areas.

[0110] It is of great significance for subsequent analysis of the safety status of the mine and monitoring of potential dangers or abnormal behaviors.

[0111] Step 1010: input the mine video frame data to be predicted into the video segmentation model, and identify and predict the mine video frame data to be predicted through the video segmentation model to obtain the predicted mine water inrush area.

[0112] In this step, the video segmentation model classifies or segments the input mine video frames at the pixel level. It uses learned features and patterns to identify different areas in the video frames, especially those that may indicate water inrush hazards. This process involves complex image processing and pattern recognition techniques.

[0113] After processing and analysis by the model, one or more prediction results are output, which indicate possible water inrush areas in the mine video frame. These prediction results may be presented in the form of highlighted areas, contours or other visual markers to help mine workers quickly identify potential danger areas, thereby improving the safety and efficiency of mine operations.

[0114] Through the above scheme, by taking the preprocessed mine video frame data as the input of the initial model, the ability to distinguish the dynamic characteristics of water flow from the static background can be enhanced under the premise of eliminating the background influence. And the initial model includes a residual shrinkage layer and a convolutional block attention layer, which enhances the attention to water flow characteristics and the suppression of background interference, and significantly improves the performance in water flow edge detection and complex background segmentation. In addition, by using the mask of the intermediate video frame data as the actual mine water inrush label, the initial model is back-propagated and adjusted to obtain a video segmentation model, which further enhances the recognition ability of water flow dynamic characteristics, thereby ensuring that the video segmentation model can perform mine water inrush recognition in a timely and accurate manner under the changing environment of the mine.

[0115] In some embodiments, step 101 includes:

[0116] Step A1, collecting video data sets in the mine.

[0117] Step A2: performing video analysis on the video data set to obtain a plurality of mine video frame data.

[0118] Step A3, performing multi-channel residual processing on the plurality of mine video frame data to obtain pre-processed mine video frame data.

[0119] In the above scheme, cameras or other video recording devices are installed inside the mine to capture different types of water hazard risk video data sets in the mine, including water seepage, water dripping, and sudden (gushing) water. At the same time, in order to enrich the visual characteristics of water hazard risks, a public set of water flow videos is collected. These video data may contain various situations in the mine, such as workers' operations, equipment operating status, environmental conditions (such as brightness, humidity, temperature, etc.) and possible safety hazards.

[0120] The collected video data set is the basis for subsequent analysis, so the integrity and accuracy of the video data must be ensured.

[0121] The mine water hazard risk videos and water flow videos are classified into water-free and water-containing ones, and then the open source computer vision library (opencv) is used to decompose the classified continuous video stream into single static image frames, i.e. video frames, and convert them into arrays and save them locally to capture the detailed information in each frame of the image. Subsequently, three consecutive frames are grouped as a group as the input of one end of the model, and these data will be used in the subsequent multi-channel residual processing steps.

[0122] The multi-channel residual processing method is used to pre-process the analyzed mine video frame data. It involves performing residual calculations on different color channels (such as red, green, and blue channels) or different feature channels of the image to distinguish dynamic objects (such as water flow) from static backgrounds in the video. Through residual calculation, the interference of the static part is removed and the water flow characteristics are retained. Among them, the residual refers to the difference between the original data and the data after some transformation or prediction.

[0123] Applying multi-channel residual processing on mine video frame data can highlight certain features or patterns in the image while reducing noise and redundant information. This preprocessing step helps improve the accuracy and efficiency of subsequent image analysis or machine learning models.

[0124] After multi-channel residual processing, the mine video frame data will be further optimized and more suitable for subsequent image recognition, classification, detection and other tasks.

[0125] In some embodiments, step A3 comprises:

[0126] Step B1, obtaining image channel data of each pixel position in each mine video frame data.

[0127] Step B2, processing the image channel data corresponding to the pixel position of each mine video frame data by a multi-channel average algorithm to obtain the average value of each pixel in each image channel data on the time axis.

[0128] Step B3, determining the middle video frame data of the plurality of mine video frame data, performing difference processing using the middle video frame data and the average value, and obtaining a multi-channel residual image.

[0129] Step B4, determining a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data.

[0130] Step B5, performing feature extraction on the multi-channel residual image to obtain a first feature image.

[0131] Step B6, multiplying the pixel values ​​of the first feature image and the fused feature image at corresponding pixel positions to obtain a feature image subjected to the product processing.

[0132] Step B7, summing up the pixel values ​​of the fused feature image and the feature image that has been multiplied to obtain preprocessed mine video frame data.

[0133] In the above scheme, image channel data (usually color channels such as RGB or grayscale) of each pixel position is extracted from the video frame data of each mine, because all information of the image is contained in the pixel and its color channel.

[0134] Based on the image channel data of the corresponding pixel position in each mine video frame data, a multi-channel average algorithm is used. This means that the same color channel value of each pixel in consecutive frames is averaged to obtain the average value of each color channel on the time axis. This helps to reduce noise and highlight stable features in the video.

[0135] An intermediate video frame data (which may be a midpoint in time sequence or a frame selected by other standards) is determined from a plurality of mine video frame data.

[0136] The multi-channel residual image is obtained by performing a difference process between the intermediate video frame data and the previously calculated multi-channel average value. The residual image reveals the changes relative to the average state, which helps to detect abnormal or dynamic changes. For example, the multiple mine video frame data input each time are three consecutive frames. The middle frame of the three consecutive frames is taken, and the corresponding average frame is subtracted to obtain a multi-channel residual image with a size of 3×448×448.

[0137] Based on the multi-channel residual image and the original multiple mine video frame data, a fused feature image is generated using a fusion technique (possibly weighted average, maximum fusion, etc.). The feature image is intended to combine the key information in the original video frame and the change information in the residual image.

[0138] Then, feature extraction is performed on the multi-channel residual image to obtain the first feature image. Feature extraction methods include edge detection, texture analysis and other image processing methods. For example, a two-dimensional convolution layer with a convolution kernel size of 3×3 is used to extract features from the multi-channel residual frame, and then the Batch Normalization (BN) layer and the Linear Rectification function (ReLU) activation function are passed to finally obtain a 1×448×448 feature image (i.e., the first feature image).

[0139] Then, the pixel values ​​of the first feature image and the pixel values ​​of the fused feature image at the corresponding pixel positions are multiplied. This product operation can enhance the common features in the image while suppressing irrelevant features. For example, the pixel values ​​of the first feature image and the pixel values ​​of the fused feature image at the corresponding pixel positions are multiplied element by element to obtain the feature image after product processing. The first feature image is passed into the continuous image frame as the attention module, so that the model focuses on the dynamic changes of the water flow in the image.

[0140] Finally, the pixel values ​​of the corresponding pixel positions of the fused feature image and the feature image processed by the product are summed to obtain the final preprocessed mine video frame data. This step combines the information of the two images to generate a preprocessed mine video frame data that contains both the original information and emphasizes the key features. For example, the pixel values ​​of the corresponding pixel positions of the fused feature image and the feature image processed by the product are summed to further retain the feature information of the original image frame. The 32×448×448 feature image (i.e., the preprocessed mine video frame data) obtained after the addition is used as the final output of the multi-channel residual attention module.

[0141] The preprocessed mine video frame data contains both original information and emphasizes key features, while reducing noise and redundancy, providing high-quality visual input for subsequent monitoring, analysis or automated decision-making.

[0142] In some embodiments, step B2 comprises:

[0143] Based on the image channel data corresponding to the pixel position of each mine video frame data, the average value of each pixel in each image channel data on the time axis is determined by the following formula, wherein the image channel data includes first channel data, second channel data and third channel data:

[0144]

[0145] in, Represents the average value of each pixel in the first channel data on the time axis. Represents the average value of each pixel in the second channel data on the time axis. Represents the average value of each pixel in the third channel data on the time axis, I i (x, y) 1 Represents the first channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 2 Represents the second channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 3 Represents the third channel data of the pixel position corresponding to the mine video frame data, t represents the total number of mine video frame data, and i represents the order of the number of mine video frame data.

[0146] In the above scheme, the mine video frame data is decomposed into three image channel data: first channel data (usually referred to as red channel R), second channel data (usually referred to as green channel G), and third channel data (usually referred to as blue channel B).

[0147] The corresponding pixel position refers to the pixels located at the same spatial position in all the mine video frames. For each such pixel position, the average values ​​of the three channels on the time axis (ie, the video frame sequence) are calculated respectively.

[0148] For the first channel (red channel), the average value of each pixel on the time axis is expressed as, and the calculation method is: add the first channel data values ​​of the pixel position in all video frames, and then divide it by the total number of video frames.

[0149] For the second channel (green channel) and the third channel (blue channel), the average value is calculated in the same way as the first channel.

[0150] By calculating the temporal average of each pixel position over three channels, a stable image representation with instantaneous noise removed can be obtained. This processing method is very useful in tasks such as video analysis, image enhancement, and target detection, especially when processing video data in complex environments such as mines where lighting conditions may vary, it can improve the stability and availability of video data.

[0151] In some embodiments, step B4 comprises:

[0152] Step C1, extracting features from each mine video frame data to obtain a plurality of second feature images.

[0153] Step C2: Processing the pixel values ​​of the plurality of second feature images at each pixel position by a splicing feature algorithm to obtain a splicing feature image.

[0154] Step C3, performing fusion processing on the spliced ​​feature images to obtain a fused feature image.

[0155] In the above scheme, feature extraction refers to extracting useful information or features from each frame of the image. These features can be color, texture, shape, edge, etc., depending on the problem to be solved and the algorithm used.

[0156] Through the feature extraction algorithm, one or more feature images (herein referred to as "second feature images") can be extracted from each frame of the image. These feature images may represent the information in the original image in different ways, for example, by emphasizing certain specific visual features or ignoring other unimportant information.

[0157] Multiple second feature images are spliced ​​together using a splicing feature algorithm. Some form of combination or fusion is performed based on their pixel values ​​at each pixel position. For example, three consecutive frames of feature images (i.e., second feature images) are spliced ​​together in the channel dimension to obtain a 24×448×448 feature image (i.e., spliced ​​feature image).

[0158] The stitching feature algorithm may involve weighted averaging of pixel values, maximum selection, minimum selection, or other more complex combination strategies. These algorithms aim to retain the useful information in each feature image while reducing redundancy and noise.

[0159] Through the processing of the splicing feature algorithm, a splicing feature image can be obtained, which integrates the information in multiple second feature images.

[0160] Then, the spliced ​​feature images are further fused to further integrate and optimize the information in the spliced ​​feature images to improve the quality and usability of the feature images. For example, an image segmentation network (U 2 Net) fuses the concatenated features and optimizes them to maintain a balance between the low-level features of the image (such as edges and textures) and the high-level features (such as the partial and overall structure of the object), and finally obtains a 32×448×448 feature image (i.e., the fused feature image).

[0161] RSU4 will perform several convolution operations on it and extract features layer by layer. The downsampling operation can be completed through convolution or pooling operations, so that the feature map is gradually reduced in spatial dimension and features of different scales are extracted. After the encoder extracts multi-scale features, the decoder branch of RSU4 will perform upsampling operations, gradually restore the spatial size of the feature map to 448×448, and fuse the upsampled features in the decoder with the features extracted by the encoder through jump connections, and finally obtain a 32×448×448 feature image (i.e., the fused feature image).

[0162] Among them, fusion processing may involve a variety of technologies, such as multi-scale fusion, multi-resolution fusion, learning-based fusion, etc. These technologies aim to utilize the complementarity between different feature images to improve the robustness and accuracy of feature images.

[0163] Through fusion processing, a fused feature image is finally obtained. This feature image may more accurately represent the key information in the mine video frame data, which helps to improve the accuracy of subsequent analysis and recognition tasks.

[0164] In some embodiments, step C2 comprises:

[0165] Based on the pixel values ​​of the plurality of second feature images at each pixel position, the spliced ​​feature image is determined by the following formula:

[0166]

[0167] Among them, O(i, j, k) represents the value of the spliced ​​feature image at the pixel position (i, j, k), A(i, j, k) represents the pixel value of the first second feature map in multiple second feature maps at the pixel position (i, j, k), B(i-8, j, k) represents the pixel value of the second second feature map in multiple second feature maps at the pixel position (i, j, k), and C(i, j, k) represents the pixel value of the third second feature map in multiple second feature maps at the pixel position (i, j, k).

[0168] In the above scheme, since all feature images are assumed to have the same size, the pixel values ​​at the same position of each image can be directly compared or combined.

[0169] By combining the pixel values ​​of multiple second feature images at the same position, a spliced ​​feature image that combines the advantages of all input images is accurately constructed.

[0170] In some embodiments, step C3 comprises:

[0171] Step D1, performing down-sampling convolution processing on the spliced ​​feature image to obtain down-sampling convolution features.

[0172] Step D2, performing upsampling convolution processing on the spliced ​​feature image to obtain a second upsampling convolution feature.

[0173] Step D3, performing feature fusion on the down-sampled convolution feature and the second up-sampled convolution feature to obtain a fused feature image.

[0174] In the above scheme, the downsampling convolution process can reduce the spatial dimensions (i.e. width and height) of the feature image while increasing its depth (i.e. number of channels). This is usually achieved by applying a convolution kernel with a stride greater than 1, or using a pooling layer. The purpose of downsampling is to reduce the amount of computation, control overfitting, and help the model learn the hierarchical structure of the image.

[0175] The downsampled convolutional features have smaller spatial dimensions and potentially greater depth than the original concatenated feature images.

[0176] In contrast to downsampling convolutions, upsampling convolutions aim to increase the spatial dimension of a feature image while potentially reducing its depth. This is usually achieved through transposed convolutions (also called deconvolutions) or upsampling layers such as bilinear interpolation. Upsampling is particularly important in tasks such as generative adversarial networks (GANs) and image segmentation because it can help restore spatial details of an image.

[0177] The down-sampled convolutional features and the second up-sampled convolutional features are combined to perform feature fusion to form a fused feature image. Feature fusion can be achieved by simple concatenation, addition, element-wise multiplication or more complex operations (such as attention mechanism).

[0178] The fused feature image combines features from different scales or levels, which helps the model understand the image content more comprehensively, and is crucial to improving the performance of tasks such as image classification, detection, and segmentation.

[0179] In some embodiments, Figure 2C As shown in FIG, a multi-channel residual preprocessing method is used to average each pixel of the original video frame in the time dimension to obtain an average frame of the three RGB channels. Subsequently, the multi-channel average frame is subtracted from the intermediate video frame to generate a multi-channel residual image.

[0180] like Figure 2D As shown in the figure, the process extracts multiple intermediate frames from continuous frames and generates corresponding mask labels for these intermediate frames. By fusing frame information from multiple time points, the precise features of dynamic targets (such as water flow) are extracted to reduce the noise or instability that may exist in a single frame. The mask label is used to annotate the target area in the intermediate frame. The white part represents the target to be learned by the model, and the black part is the background information.

[0181] like Figure 2E This is the detection result of a water hazard risk video. The sample is a video of water bursting from the bottom of an underground working face. Figure 2E The original video frame in (a) shows the scene of water gushing out from the bottom aquifer continuously. The water gushes out like a fountain and the amount of water increases over time. Final detection results Figure 2E As shown in (d), this method effectively segments the water bursting area, with a Dice similarity coefficient (Dice) value of 86.75% and an Intersection over Union (IoU) value of 80.23%.

[0182] In some embodiments, the mine water inrush identification process of the present application is as follows: Figure 2F As shown:

[0183] Step 201, obtaining a mine water hazard risk data set;

[0184] Step 202, performing video frame analysis on the mine water hazard risk dataset;

[0185] Step 203, preprocessing the video frame data after video parsing by a multi-channel residual attention module to obtain preprocessed video frame data;

[0186] Step 204, inputting the pre-processed video frame data into the pre-built initial model (CBAM-U2Net);

[0187] Step 205, performing intermediate frame back propagation training on the initial model to obtain a video segmentation model;

[0188] Step 206, using the video segmentation model to perform prediction, to obtain the predicted mine water inrush area.

[0189] The multi-channel residual attention module in step 203 performs preprocessing as follows: Figure 2G As shown:

[0190] Step 2031, parsing the mine water hazard risk data set into a mine water hazard risk video frame;

[0191] Step 2032, determining a multi-channel average frame;

[0192] Step 2033, performing multi-channel residual preprocessing based on the multi-channel average frame to obtain a multi-channel residual image;

[0193] Step 2034, extracting features from the multi-channel residual image to obtain a first feature image;

[0194] Step 2035, performing multi-frame fusion based on the multi-channel parameter image and the plurality of mine video frame data to determine a fused feature image;

[0195] Step 2036, during the fusion process, feature extraction is performed on each mine video frame data to obtain a plurality of second feature images, and feature splicing is performed based on the pixel values ​​of the plurality of second feature images at each pixel position to obtain a spliced ​​feature image;

[0196] Step 2037, fusing the spliced ​​feature images using the initial model (CBAM-RSU4) to obtain a fused feature image;

[0197] Step 2038, use the attention module to multiply the pixel values ​​of the first feature image and the fused feature image at the corresponding pixel positions to obtain a feature image processed by the product; sum the pixel values ​​of the fused feature image and the feature image processed by the product at the corresponding pixel positions to obtain a preprocessed mine video frame (i.e., output feature).

[0198] Step 2039, output the preprocessed mine video frame data (ie, output features).

[0199] In summary, the features of this application are as follows:

[0200] 1) The introduction of the multi-channel residual attention mechanism can effectively eliminate the interference of the static background by processing the residual information of the RGB channels, enhance the model's ability to capture the dynamic characteristics of water flow, and improve the accuracy and robustness of water hazard identification, especially showing strong generalization ability in complex underground environments.

[0201] 2) Improved U 2 Net network structure combined with CBAM module, in the traditional U 2 The convolutional block attention layer (CBAM) is introduced into the Net network to enhance the attention to water flow features and suppress background interference, significantly improving the performance in water flow edge detection and complex background segmentation, ensuring efficient water hazard identification in the changing environment of mines.

[0202] 3) Application of multi-frame fusion learning strategy. A multi-frame fusion learning method is proposed. By using continuous frames and intermediate frame masks as labels, the model's learning ability for water flow continuity and dynamic changes is enhanced, and the model's detection accuracy in sudden water disasters is improved, especially its performance in cross-scene water disaster detection.

[0203] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.

[0204] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0205] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a mine water inrush identification device.

[0206] refer to Figure 3 , the mine water inrush identification device comprises:

[0207] The training acquisition module 301 is configured to acquire pre-processed mine video frame data;

[0208] A training input module 302 is configured to input the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer;

[0209] A feature extraction module 303 is configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data;

[0210] A feature adjustment module 304 is configured to perform feature adjustment on the multi-scale feature data using the convolutional block attention layer to obtain adjusted multi-scale feature data;

[0211] The upsampling module 305 is configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain a first upsampling convolution feature;

[0212] A splicing processing module 306 is configured to splice the adjusted multi-scale feature data and the up-sampled convolution feature to obtain a training prediction mine water inrush area;

[0213] The intermediate frame determination module 307 is configured to determine the intermediate video frame data from the preprocessed mine video frame data, and obtain the actual mine water inrush label of the intermediate video frame data;

[0214] A back propagation module 308 is configured to perform back propagation adjustment on the initial model using the actual mine water inrush label and the training predicted mine water inrush area to obtain a video segmentation model;

[0215] The prediction acquisition module 309 is configured to acquire the mine video frame data to be predicted;

[0216] The identification and prediction module 310 is configured to input the mine video frame data to be predicted into the video segmentation model, identify and predict the mine video frame data to be predicted through the video segmentation model, and obtain the predicted mine water inrush area.

[0217] In some embodiments, the training acquisition module 301 includes:

[0218] A collection submodule, configured to collect a video data set in the mine;

[0219] A video analysis submodule is configured to perform video analysis on the video data set to obtain a plurality of mine video frame data;

[0220] The residual processing submodule is configured to perform multi-channel residual processing on the multiple mine video frame data to obtain pre-processed mine video frame data.

[0221] In some embodiments, the residual processing submodule includes:

[0222] An acquisition unit is configured to acquire image channel data of each pixel position in each mine video frame data;

[0223] An average value processing unit is configured to process the image channel data corresponding to the pixel position of each mine video frame data through a multi-channel average value algorithm to obtain the average value of each pixel in each image channel data on the time axis;

[0224] A difference processing unit is configured to determine the middle video frame data of the plurality of mine video frame data, and perform difference processing using the middle video frame data and the average value to obtain a multi-channel residual image;

[0225] A fusion unit, configured to determine a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data;

[0226] A feature extraction unit, configured to perform feature extraction on the multi-channel residual image to obtain a first feature image;

[0227] a product processing unit configured to perform product processing on the pixel values ​​of the first feature image and the pixel values ​​of the fused feature image at corresponding pixel positions to obtain a feature image subjected to product processing;

[0228] The summing processing unit is configured to sum the pixel values ​​of the fused feature image and the feature image that has been multiplied to obtain the pre-processed mine video frame data.

[0229] In some embodiments, the average value processing unit is specifically configured to:

[0230] Based on the image channel data corresponding to the pixel position of each mine video frame data, the average value of each pixel in each image channel data on the time axis is determined by the following formula, wherein the image channel data includes first channel data, second channel data and third channel data:

[0231]

[0232] in, Represents the average value of each pixel in the first channel data on the time axis. Represents the average value of each pixel in the second channel data on the time axis. Represents the average value of each pixel in the third channel data on the time axis, I i (x, y) 1 Represents the first channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 2 Represents the second channel data of the pixel position corresponding to the mine video frame data, I i (x, y) 3 Represents the third channel data of the pixel position corresponding to the mine video frame data, t represents the total number of mine video frame data, and i represents the order of the number of mine video frame data.

[0233] In some embodiments, the fusion unit comprises:

[0234] The feature extraction subunit is configured to extract features from each mine video frame data to obtain a plurality of second feature images;

[0235] A splicing subunit is configured to process the pixel values ​​of the plurality of second feature images at each pixel position by a splicing feature algorithm to obtain a splicing feature image;

[0236] The fusion subunit is configured to perform fusion processing on the spliced ​​feature images to obtain a fused feature image.

[0237] In some embodiments, the splicing subunit is specifically configured as:

[0238] Based on the pixel values ​​of the plurality of second feature images at each pixel position, the spliced ​​feature image is determined by the following formula:

[0239]

[0240] Among them, O(i, j, k) represents the value of the spliced ​​feature image at the pixel position (i, j, k), A(i, j, k) represents the pixel value of the first second feature map in multiple second feature maps at the pixel position (i, j, k), V(i-8, j, k) represents the pixel value of the second second feature map in multiple second feature maps at the pixel position (i, j, k), and C(i, j, k) represents the pixel value of the third second feature map in multiple second feature maps at the pixel position (i, j, k).

[0241] In some embodiments, the fusion subunit is specifically configured as:

[0242] Performing down-sampling convolution processing on the spliced ​​feature image to obtain down-sampling convolution features;

[0243] Performing up-sampling convolution processing on the spliced ​​feature image to obtain a second up-sampling convolution feature;

[0244] The down-sampled convolution feature and the second up-sampled convolution feature are subjected to feature fusion to obtain a fused feature image.

[0245] For the convenience of description, the above device is described in terms of functions divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0246] The device of the above embodiment is used to implement the corresponding mine water inrush identification method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0247] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the mine water inrush identification method described in any of the above embodiments is implemented.

[0248] Figure 4 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 405. The processor 401, the memory 402, the input / output interface 403, and the communication interface 404 are connected to each other in communication within the device through the bus 405.

[0249] The processor 401 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0250] The memory 402 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 402 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 402 and are called and executed by the processor 401.

[0251] The input / output interface 403 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0252] The communication interface 404 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0253] The bus 405 comprises a pathway for transmitting information between the various components of the device (eg, the processor 401 , the memory 402 , the input / output interface 403 , and the communication interface 404 ).

[0254] It should be noted that, although the above device only shows the processor 401, the memory 402, the input / output interface 403, the communication interface 404 and the bus 405, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0255] The electronic device of the above-mentioned embodiment is used to implement the corresponding mine water inrush identification method in any of the above-mentioned embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0256] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the mine water inrush identification method as described in any of the above embodiments.

[0257] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0258] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the mine water inrush identification method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0259] A person skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present application is limited to these examples. In line with the concept of the present application, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.

[0260] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present application, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered to be illustrative rather than restrictive.

[0261] Although the present application has been described in conjunction with specific embodiments of the present application, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0262] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the present application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the protection scope of the present application.

Claims

1. A method for identifying water inrush in a mine, characterized in that: include: Obtaining preprocessed mine video frame data; Inputting the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer; Performing feature extraction on the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data; Using the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data; Performing up-sampling convolution processing on the pre-processed mine video frame data through the second sub-residual contraction layer to obtain a first up-sampling convolution feature; The adjusted multi-scale feature data and the up-sampled convolution feature are spliced ​​to obtain a training prediction mine water inrush area; Determine intermediate video frame data from the preprocessed mine video frame data, and obtain actual mine water inrush labels of the intermediate video frame data; Using the actual mine water inrush label and the training predicted mine water inrush area to perform back propagation adjustment on the initial model to obtain a video segmentation model; Obtain the mine video frame data to be predicted; The video frame data of the mine to be predicted is input into the video segmentation model, and the video frame data of the mine to be predicted is identified and predicted by the video segmentation model to obtain the predicted mine water inrush area.

2. The method according to claim 1, characterized in that The obtaining of pre-processed mine video frame data includes: Collect video data sets inside mines; Performing video analysis on the video data set to obtain a plurality of mine video frame data; Multi-channel residual processing is performed on the multiple mine video frame data to obtain pre-processed mine video frame data.

3. The method according to claim 2, characterized in that The performing multi-channel residual processing on the plurality of mine video frame data to obtain pre-processed mine video frame data includes: Obtain image channel data of each pixel position in each mine video frame data; The image channel data corresponding to the pixel position of each mine video frame data is processed by a multi-channel average algorithm to obtain the average value of each pixel in each image channel data on the time axis; Determine the intermediate video frame data of the plurality of mine video frame data, and perform difference processing using the intermediate video frame data and the average value to obtain a multi-channel residual image; Determine a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data; Performing feature extraction on the multi-channel residual image to obtain a first feature image; Performing product processing on the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image to obtain a feature image subjected to product processing; The pixel values ​​at the corresponding pixel positions of the fused feature image and the feature image subjected to product processing are summed to obtain preprocessed mine video frame data.

4. The method according to claim 3, characterized in that The image channel data corresponding to the pixel position of each mine video frame data is processed by a multi-channel average algorithm to obtain the average value of each pixel in each image channel data on the time axis, including: Based on the image channel data corresponding to the pixel position of each mine video frame data, the average value of each pixel in each image channel data on the time axis is determined by the following formula, wherein the image channel data includes first channel data, second channel data and third channel data: in, Represents the average value of each pixel in the first channel data on the time axis. Represents the average value of each pixel in the second channel data on the time axis. Represents the average value of each pixel in the third channel data on the time axis, I i (x, y) 1 represents the first channel data of the pixel position corresponding to the mine video frame data, I i (x, y)2 represents the second channel data of the pixel position corresponding to the mine video frame data, I i (x, y)3 represents the third channel data of the pixel position corresponding to the mine video frame data, t represents the total number of mine video frame data, and i represents the order of the number of mine video frame data.

5. The method according to claim 3, characterized in that: The step of determining a fused feature image based on the multi-channel parameter image and the plurality of mine video frame data comprises: Extracting features from each mine video frame data to obtain a plurality of second feature images; Processing the pixel values ​​of the plurality of second feature images at each pixel position by a splicing feature algorithm to obtain a splicing feature image; The spliced ​​feature images are fused to obtain a fused feature image.

6. The method according to claim 5, characterized in that The step of processing the pixel values ​​at each pixel position of the plurality of second feature images by a splicing feature algorithm to obtain a splicing feature image comprises: Based on the pixel values ​​of the plurality of second feature images at each pixel position, the spliced ​​feature image is determined by the following formula: Among them, O(i, j, k) represents the value of the spliced ​​feature image at the pixel position (i, j, k), A(i, j, k) represents the pixel value of the first second feature map in multiple second feature maps at the pixel position (i, j, k), B(i-8, j, k) represents the pixel value of the second second feature map in multiple second feature maps at the pixel position (i, j, k), and C(i, j, k) represents the pixel value of the third second feature map in multiple second feature maps at the pixel position (i, j, k).

7. The method according to claim 5, characterized in that The step of fusing the spliced ​​feature images to obtain a fused feature image includes: Performing down-sampling convolution processing on the spliced ​​feature image to obtain down-sampling convolution features; Performing up-sampling convolution processing on the spliced ​​feature image to obtain a second up-sampling convolution feature; The down-sampled convolution feature and the second up-sampled convolution feature are subjected to feature fusion to obtain a fused feature image.

8. A mine water inrush identification device, characterized in that: include: A training acquisition module is configured to acquire pre-processed mine video frame data; A training input module is configured to input the preprocessed mine video frame data into a pre-built initial model, wherein the initial model includes a residual shrinkage layer and a convolutional block attention layer, wherein the residual shrinkage layer includes a first sub-residual shrinkage layer and a second sub-residual shrinkage layer; a feature extraction module configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data; A feature adjustment module is configured to perform feature adjustment on the multi-scale feature data using the convolutional block attention layer to obtain adjusted multi-scale feature data; an upsampling module, configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain a first upsampling convolution feature; A splicing processing module is configured to splice the adjusted multi-scale feature data and the up-sampled convolution feature to obtain a training prediction mine water inrush area; An intermediate frame determination module is configured to determine intermediate video frame data from the preprocessed mine video frame data, and obtain an actual mine water inrush label of the intermediate video frame data; A back propagation module is configured to use the actual mine water inrush label and the training predicted mine water inrush area to perform back propagation adjustment on the initial model to obtain a video segmentation model; A prediction acquisition module is configured to acquire the mine video frame data to be predicted; The identification and prediction module is configured to input the mine video frame data to be predicted into the video segmentation model, identify and predict the mine video frame data to be predicted through the video segmentation model, and obtain the predicted mine water inrush area.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 7.

Images