Methods, devices, electronic equipment and storage media for identifying mine water inrush

Abstract

This application provides a method, apparatus, electronic device, and storage medium for identifying mine water inrush. By using preprocessed mine video frame data as input to an initial model, the ability to distinguish between dynamic water flow features and static backgrounds can be enhanced while eliminating background interference. Furthermore, the initial model includes a residual shrinkage layer and a convolutional block attention layer, which enhances the focus on water flow features and suppresses background interference, significantly improving performance in water flow edge detection and complex background segmentation. In addition, by using the mask of intermediate video frame data as the actual mine water inrush label, the initial model is backpropagated and adjusted to obtain a video segmentation model, further strengthening the ability to identify dynamic water flow features. This ensures that the video segmentation model can perform timely and accurate mine water inrush identification in the variable environment of a mine.

Description

Technical Field

[0001] This application relates to the field of mine safety technology, and in particular to a method, device, electronic device and storage medium for identifying mine water inrush. Background Technology

[0002] Coal provides essential energy and raw materials for industrial production, power generation, and heating, making coal mining inherently important. However, mine water inrush is a major hazard to mine production and safety, and its frequent occurrence poses a direct threat to safe coal mine operations.

[0003] Based on the above, existing mine water inrush identification methods cannot accurately and timely determine the mine water inrush area. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide a method, device, electronic device and storage medium for identifying mine water inrush, so as to solve the above-mentioned technical problems.

[0005] To achieve the above objectives, the first aspect of this application provides a method for identifying mine water inrush, comprising:

[0006] Acquire preprocessed mine video frame data;

[0007] The preprocessed mine video frame data is input into a pre-built initial model, which includes a residual shrinking layer and a convolutional block attention layer. The residual shrinking layer includes a first sub-residual shrinking layer and a second sub-residual shrinking layer.

[0008] The preprocessed mine video frame data is used to extract features through the first sub-residual shrinkage layer to obtain multi-scale feature data.

[0009] The multi-scale feature data is adjusted using the convolutional block attention layer to obtain adjusted multi-scale feature data.

[0010] The preprocessed mine video frame data is upsampled and convolved by the second sub-residual shrinkage layer to obtain the first upsampled convolution feature.

[0011] The adjusted multi-scale feature data and the upsampled convolutional features are concatenated to obtain the trained prediction of the mine water inrush area.

[0012] Determine intermediate video frame data from preprocessed mine video frame data, and obtain the actual mine water inrush label of the intermediate video frame data;

[0013] The initial model is adjusted by backpropagation using the actual mine water inrush labels and the trained predicted mine water inrush areas to obtain a video segmentation model.

[0014] Acquire video frame data of the mine to be predicted;

[0015] The video frame data of the mine to be predicted is input into the video segmentation model, and the video segmentation model is used to identify and predict the video frame data of the mine to be predicted, so as to obtain the predicted water inrush area of ​​the mine.

[0016] Optionally, acquiring the preprocessed mine video frame data includes:

[0017] Collect video datasets from inside the mine;

[0018] The video dataset was parsed to obtain multiple mine video frame data;

[0019] Multi-channel residual processing is performed on the multiple mine video frame data to obtain preprocessed mine video frame data.

[0020] Optionally, the step of performing multi-channel residual processing on the plurality of mine video frame data to obtain preprocessed mine video frame data includes:

[0021] Obtain the image channel data for each pixel position in the video frame data of each mine;

[0022] The image channel data based on the pixel positions of each mine video frame is processed by a multi-channel averaging algorithm to obtain the average value of each pixel on the time axis in each image channel data.

[0023] The intermediate video frame data of multiple mine video frame data is determined, and the difference between the intermediate video frame data and the average value is calculated to obtain a multi-channel residual image.

[0024] Based on the multi-channel residual image and the multiple mine video frame data, a fused feature image is determined;

[0025] Feature extraction is performed on the multi-channel residual image to obtain a first feature image;

[0026] The pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image are multiplied to obtain the feature image after multiplication.

[0027] The pixel values ​​at corresponding pixel positions of the fused feature image and the product-processed feature image are summed to obtain preprocessed mine video frame data.

[0028] Optionally, determining the fused feature image based on the multi-channel residual image and the multiple mine video frame data includes:

[0029] Feature extraction is performed on each mine video frame data to obtain multiple second feature images;

[0030] The pixel values ​​at each pixel position of the multiple second feature images are processed by a concatenation feature algorithm to obtain a concatenated feature image;

[0031] The stitched feature images are fused to obtain fused feature images.

[0032] Optionally, the pixel values ​​at each pixel position of the plurality of second feature images are processed by a concatenation feature algorithm to obtain a concatenated feature image, including:

[0033] Based on the pixel values ​​at each pixel position of the multiple second feature images, the stitched feature image is determined by the following formula:

[0034]

[0035] in, Indicates the pixel position of the stitched feature image The value at that location, This indicates the pixel position of the first second feature map among multiple second feature maps. Pixel value at that location, This indicates the second second feature map among multiple second feature maps at pixel location. Pixel value at that location, This indicates the third second feature map among multiple second feature maps at pixel location. The pixel value at that location.

[0036] Optionally, the step of fusing the stitched feature images to obtain the fused feature image includes:

[0037] The stitched feature image is subjected to downsampling convolution processing to obtain downsampling convolution features;

[0038] The stitched feature image is subjected to upsampling convolution processing to obtain the second upsampling convolution feature;

[0039] The downsampled convolutional features and the second upsampled convolutional features are fused to obtain a fused feature image.

[0040] Based on the same inventive concept, a second aspect of this application provides a mine water inrush identification device, comprising:

[0041] The training acquisition module is configured to acquire preprocessed mine video frame data;

[0042] The training input module is configured to input preprocessed mine video frame data into a pre-built initial model, the initial model including a residual shrinking layer and a convolutional block attention layer, the residual shrinking layer including a first sub-residual shrinking layer and a second sub-residual shrinking layer;

[0043] The feature extraction module is configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data;

[0044] The feature adjustment module is configured to use the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data.

[0045] The upsampling module is configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain the first upsampling convolution feature;

[0046] The splicing processing module is configured to splice the adjusted multi-scale feature data and the upsampled convolutional features to obtain the trained prediction of the mine water inrush area.

[0047] The intermediate frame determination module is configured to determine intermediate video frame data from preprocessed mine video frame data and obtain the actual mine water inrush label of the intermediate video frame data.

[0048] The backpropagation module is configured to perform backpropagation adjustment on the initial model using the actual mine water inrush labels and the trained predicted mine water inrush areas to obtain a video segmentation model.

[0049] The prediction acquisition module is configured to acquire video frame data of the mine to be predicted;

[0050] The identification and prediction module is configured to input the video frame data of the mine to be predicted into the video segmentation model, and to identify and predict the video frame data of the mine to be predicted through the video segmentation model to obtain the predicted water inrush area of ​​the mine.

[0051] Based on the same inventive concept, a third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executed by the processor, wherein the processor, when executing the computer program, implements the method described in the first aspect above.

[0052] Based on the same inventive concept, a fourth aspect of this application provides a non-transitory computer-readable storage medium that stores computer instructions for causing a computer to perform the method described in the first aspect above.

[0053] As can be seen from the above, the mine water inrush identification method, apparatus, electronic device, and storage medium provided in this application, by using preprocessed mine video frame data as input to the initial model, can enhance the ability to distinguish between dynamic water flow features and static backgrounds while eliminating background influences. Furthermore, this initial model includes a residual shrinkage layer and a convolutional block attention layer, enhancing the focus on water flow features and suppressing background interference, significantly improving performance in water flow edge detection and complex background segmentation. In addition, by using the mask of intermediate video frame data as the actual mine water inrush label to perform backpropagation adjustment on the initial model, a video segmentation model is obtained, further strengthening the identification capability of dynamic water flow features, thereby ensuring that the video segmentation model can perform timely and accurate mine water inrush identification in the variable environment of a mine. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the mine water inrush identification method according to an embodiment of this application;

[0056] Figure 2A The original image segmentation model (U) of the embodiments of this application 2 Net model diagram;

[0057] Figure 2B The video segmentation model (CBAM-U) of this application embodiment 2 Net) diagram;

[0058] Figure 2C This is a schematic diagram of multi-channel residual preprocessing according to an embodiment of this application;

[0059] Figure 2D This is a schematic diagram of intermediate frame marking in an embodiment of this application;

[0060] Figure 2E This is a schematic diagram illustrating the detection results of a water hazard risk video according to an embodiment of this application;

[0061] Figure 2F This is a schematic diagram of the mine water inrush identification process according to an embodiment of this application;

[0062] Figure 2G A schematic diagram of the preprocessing process for the multi-channel residual attention module in this application embodiment;

[0063] Figure 3This is a structural block diagram of a mine water inrush identification device according to an embodiment of this application;

[0064] Figure 4 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0066] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0067] It is understood that before using the technical solutions of the various embodiments in this application, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0068] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described in this application.

[0069] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0070] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0071] Complex geological conditions can easily lead to mine water hazards. Once a mine water hazard occurs, it often causes severe economic losses, prolonged rescue operations, and widespread social impact. Its high frequency and suddenness continue to pose a significant threat to mine safety.

[0072] The three stages of identifying sudden water inrush risks—the initial stage from nothing to something, the expansion stage from small to large, and the uncontrolled stage exceeding the mine's drainage capacity—are the core processes of the gradual evolution of water hazards. Therefore, identifying the dynamic changes of sudden water inrushes is crucial in mine water hazard prevention and control.

[0073] This application finds that one reason why existing mine water inrush identification methods cannot accurately and promptly determine the area of ​​a mine water inrush is that they primarily rely on sensor monitoring and manual inspection. This has certain limitations in practical applications. For example, sensor monitoring is easily affected by the complex environment of a mine and may be damaged in an accident, leading to unstable and unreliable monitoring. Furthermore, electromagnetic interference within the mine can also affect the data transmission accuracy of the sensors. Manual inspection requires significant manpower, and in the event of a water inrush incident, inspection personnel may need to evacuate urgently, thus failing to provide timely information.

[0074] This application provides a method for identifying mine water inrush. By using preprocessed mine video frame data as input to an initial model, it enhances the ability to distinguish between dynamic water flow features and static backgrounds while eliminating background interference. Furthermore, the initial model includes a residual shrinkage layer and a convolutional block attention layer, enhancing the focus on water flow features and suppressing background interference, significantly improving performance in water flow edge detection and complex background segmentation. In addition, by using the mask of intermediate video frame data as the actual mine water inrush label, the initial model is backpropagated and adjusted to obtain a video segmentation model, further strengthening the ability to identify dynamic water flow features. This ensures that the video segmentation model can perform timely and accurate mine water inrush identification in the variable environment of a mine.

[0075] like Figure 1 As shown, the method in this embodiment includes:

[0076] Step 101: Obtain preprocessed mine video frame data.

[0077] In this step, sensors, cameras, and other equipment can be installed in the mine environment to capture video data such as changes in water level and geological structure. The captured video data is then preprocessed, such as image enhancement (brightness adjustment, contrast enhancement), noise removal, and size adjustment, to improve data quality and make it more suitable for subsequent model processing.

[0078] Step 102: Input the preprocessed mine video frame data into a pre-built initial model. The initial model includes a residual shrinking layer and a convolutional block attention layer. The residual shrinking layer includes a first sub-residual shrinking layer and a second sub-residual shrinking layer.

[0079] In this step, the residual U-block (RSU) layer is a deep learning structure that combines residual learning and an attention mechanism (Shrinkage Strategy). Residual learning allows the network to learn the residual between the input and output, which helps solve the gradient vanishing problem in deep neural networks, enabling the network to learn deeper features. The "shrinkage" part, by introducing an attention mechanism, weights the input features and suppresses unimportant features, thereby enhancing the model's representational power.

[0080] Convolutional Block Attention Model (CBAM) is a lightweight attention layer designed to enhance the representational power of Convolutional Neural Networks (CNNs). CBAM emphasizes important features in the input image and suppresses unimportant features by sequentially applying channel attention and spatial attention. This mechanism helps the model focus more on task-useful information, thereby improving performance.

[0081] This initial model combines a residual shrinking layer and a convolutional block attention layer to process pre-processed mine video frame data. This improves the model's ability to capture key information in the mine environment, thereby supporting more accurate monitoring and analysis tasks.

[0082] like Figure 2A and Figure 2B As shown, the original image segmentation network (U 2 The Net model embeds multiple Residual Shrinkage (RSU) layers. For each RSU layer, it consists of multiple convolutional blocks with Conv2d + BN + ReLU structure forming a U-shaped structure. After the output of each convolutional module on the left (i.e. the first sub-residual shrinkage layer), a Convolutional Block Attention (CBAM) layer is added to output the feature map after CBAM processing.

[0083] Step 103: Extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data.

[0084] In this step, the preprocessed mine video frame data is processed using the first sub-residual shrinkage layer to extract useful multi-scale feature data from these data.

[0085] Multiscale feature data refers to feature data extracted at different scales (or resolutions).

[0086] Due to the complexity of the mine environment, features at different scales can provide complementary information about the scene or objects. For example, at a larger scale, the network might focus on the overall structure and layout, while at a smaller scale, it might focus on details and textures. By combining these multi-scale features, the model's ability to understand and analyze mine video content can be improved.

[0087] Step 104: Use the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data.

[0088] In this step, CBAM is used to process the multi-scale feature data to enhance or suppress certain parts of the feature map.

[0089] Specifically, CBAM uses channel and spatial attention mechanisms to reweight input features based on their importance (adaptively adjusted using attention weights), thereby highlighting key features such as water flow and suppressing background interference. This adjustment helps the model more effectively extract features hierarchically and segment them finely in water flow regions.

[0090] Step 105: The preprocessed mine video frame data is upsampled and convolved through the second sub-residual shrinkage layer to obtain the first upsampled convolution feature.

[0091] In this step, upsampling is achieved in the upsampling convolution process by applying a convolution operation (i.e., using a set of filters or kernels to slide across the image and perform a dot product operation).

[0092] In convolutional neural networks (CNNs), upsampling convolutional layers are typically used to enlarge feature maps to the same size as the input image or other feature maps for subsequent comparison or fusion operations.

[0093] Upsampled convolutional features refer to feature maps obtained through upsampled convolutional layers. This process can generate more refined image features that contain higher-level image information, such as edges, textures, or partial structures of objects. This information helps improve accuracy in subsequent analysis or recognition and prediction tasks.

[0094] Step 106: The adjusted multi-scale feature data and the upsampled convolutional features are concatenated to obtain the trained prediction of the mine water inrush area.

[0095] In this step, the concatenation process is the process of merging two or more datasets or feature maps into a larger dataset or feature map. In the context of mine water inrush area prediction, concatenating adjusted multi-scale feature data and upsampled convolutional features means merging these two types of features in some dimension (such as the channel dimension) to form a feature set containing richer information that represents the area in the mine where water inrush may occur.

[0096] like Figure 2B As shown, the feature map processed by the CBAM layer (i.e., the adjusted multi-scale feature data) is concatenated with the feature map (i.e., the upsampled convolutional feature) in the upsampling path on the right side of the RSU layer (i.e., the second sub-residual shrinkage layer). The feature map on the right side (i.e., the upsampled convolutional feature) is generated by a convolution operation that restores spatial resolution through upsampling.

[0097] For U 2 In the Net, a CBAM layer is embedded between each RSU layer on the left and spliced ​​with each RSU layer on the right.

[0098] Step 107: Determine intermediate video frame data from the preprocessed mine video frame data, and obtain the actual mine water inrush label of the intermediate video frame data.

[0099] In this step, based on information such as timestamps or frame numbers, one or more frames located in the middle of the video sequence are identified as intermediate video frames from the preprocessed mine video frame data. This process may need to take into account factors such as the total number of frames and the frame rate of the video.

[0100] For the extracted intermediate video frame data, it is necessary to obtain its corresponding actual mine water inrush label. Actual mine water inrush labels refer to real data about mine water inrush events obtained through observation, recording, or other methods in actual mine environments. This data is usually labeled as "water inrush" or "non-water inrush," forming a labeled dataset. These labels are the "correct answers" in the model training process, used to guide the model in learning how to distinguish between water inrush areas and non-water inrush areas.

[0101] Step 108: Use the actual mine water inrush labels and the trained predicted mine water inrush areas to perform backpropagation adjustment on the initial model to obtain the video segmentation model.

[0102] In this step, when the training-predicted mine water inrush area is inconsistent with the actual mine water inrush label, the backpropagation algorithm is needed to adjust the model parameters. Backpropagation reduces the risk of information loss during continuous frame processing by calculating the prediction error (i.e., the difference between the training-predicted mine water inrush area and the actual mine water inrush label). By comparing features from intermediate frames, the model adjusts the network weights, thereby optimizing its ability to capture the continuous dynamic changes in water flow. This error is then backpropagated to each layer of the network to update the network weights and bias parameters. This process aims to minimize the prediction error and make the model's predictions more accurate.

[0103] Through multiple iterations of training (i.e., repeated predictions, error calculations, backpropagation, and parameter adjustments), the model's performance gradually improves until a satisfactory accuracy is achieved. Ultimately, this trained model is called the video segmentation model (CBAM-U²Net) because it can process video data and segment each frame into areas prone to water inrush and non-water inrush areas. In practical applications, such a model can monitor mine conditions in real time and accurately segment and identify areas with potential water inrush risks.

[0104] Step 109: Obtain video frame data of the mine to be predicted.

[0105] In this step, sensors, cameras, and other equipment can be installed in the mine environment to capture video data such as changes in water level and geological structure. This data is segmented into a series of video frames (i.e., continuous static images) of the mine to be predicted. These video frames contain various information about the mine interior, including potential water inrush areas.

[0106] This is of great significance for subsequent analysis of mine safety conditions and monitoring of potential hazards or abnormal behavior.

[0107] Step 1010: Input the video frame data of the mine to be predicted into the video segmentation model, and use the video segmentation model to identify and predict the video frame data of the mine to be predicted, so as to obtain the predicted mine water inrush area.

[0108] In this step, the video segmentation model performs pixel-level classification or segmentation on the input mine video frames. It utilizes learned features and patterns to identify different regions within the video frames, particularly those that may indicate a risk of water inrush. This process involves sophisticated image processing and pattern recognition techniques.

[0109] After processing and analysis by the model, one or more predictions will be output, indicating potential water inrush areas in the mine video frames. These predictions may be presented as highlighted areas, outlines, or other visual markers to help mine workers quickly identify potentially hazardous areas, thereby improving the safety and efficiency of mine operations.

[0110] By employing the aforementioned scheme and using preprocessed mine video frame data as input to the initial model, the ability to distinguish between dynamic water flow features and static backgrounds can be enhanced while eliminating background interference. Furthermore, this initial model includes a residual shrinkage layer and a convolutional block attention layer, which enhances the focus on water flow features and suppresses background interference, significantly improving performance in water flow edge detection and complex background segmentation. In addition, by using the mask of intermediate video frame data as an actual mine water inrush label, the initial model is backpropagated and adjusted to obtain a video segmentation model, further strengthening the ability to recognize dynamic water flow features. This ensures that the video segmentation model can perform timely and accurate mine water inrush identification in the variable environment of a mine.

[0111] In some embodiments, step 101 includes:

[0112] Step A1: Collect video datasets from inside the mine.

[0113] Step A2: Perform video parsing on the video dataset to obtain multiple mine video frame data.

[0114] Step A3: Perform multi-channel residual processing on the multiple mine video frame data to obtain preprocessed mine video frame data.

[0115] In the aforementioned approach, cameras or other video recording equipment are installed inside the mine to capture video datasets of different types of water hazard risks, including seepage, water spray, and sudden (rushing) water. Simultaneously, to enrich the visual characteristics of water hazard risks, a public set of water flow videos is collected. This video data may encompass various situations within the mine, such as worker operations, equipment operating status, environmental conditions (e.g., brightness, humidity, temperature), and potential safety hazards.

[0116] The collected video dataset is the foundation for subsequent analysis, so it is necessary to ensure the integrity and accuracy of the video data.

[0117] The mine water hazard risk videos and water flow videos are classified as having no water and having water. Then, the open-source computer vision library (OpenCV) is used to decompose the classified continuous video stream into individual static image frames, i.e., video frames. At the same time, these frames are converted into arrays and saved locally to capture detailed information in each frame. Subsequently, three consecutive frames are grouped together as input to one end of the model. These data will be used for subsequent multi-channel residual processing steps.

[0118] Preprocessing is performed on the parsed mine video frame data using a multi-channel residual processing method. This involves calculating residuals for different color channels (e.g., red, green, and blue channels) or different feature channels to distinguish dynamic objects (e.g., water flow) from the static background in the video. Residual calculation removes interference from static parts while preserving water flow characteristics. Here, residual refers to the difference between the original data and the data after some transformation or prediction.

[0119] Applying multi-channel residual processing to mine video frame data can highlight certain features or patterns in the image while reducing noise and redundant information. This preprocessing step helps improve the accuracy and efficiency of subsequent image analysis or machine learning models.

[0120] After multi-channel residual processing, the mine video frame data will be further optimized, making it more suitable for subsequent image recognition, classification, detection and other tasks.

[0121] In some embodiments, step A3 includes:

[0122] Step B1: Obtain the image channel data of each pixel position in the video frame data of each mine.

[0123] Step B2: Based on the image channel data corresponding to the pixel positions of each mine video frame data, the multi-channel averaging algorithm is used to process the image channel data to obtain the average value of each pixel on the time axis in each image channel data.

[0124] Step B3: Determine the intermediate video frame data of multiple mine video frame data, and perform subtraction processing using the intermediate video frame data and the average value to obtain a multi-channel residual image.

[0125] Step B4: Determine the fused feature image based on the multi-channel residual image and the multiple mine video frame data.

[0126] Step B5: Extract features from the multi-channel residual image to obtain the first feature image.

[0127] Step B6: Multiply the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image to obtain the feature image after multiplication.

[0128] Step B7: Summing the pixel values ​​at corresponding pixel positions of the fused feature image and the multiplied feature image to obtain preprocessed mine video frame data.

[0129] In the above scheme, image channel data (usually referring to color channels such as RGB or grayscale) of each pixel position is extracted from the video frame data of each mine, because all the information of the image is contained in the pixel and its color channel.

[0130] Based on the image channel data of corresponding pixel positions in each mine video frame, a multi-channel averaging algorithm is used. This means averaging the value of the same color channel for each pixel in consecutive frames to obtain the average value of each color channel over time. This helps reduce noise and highlight stable features in the video.

[0131] Determine an intermediate video frame from multiple mine video frame data (which may be the midpoint in time sequence or a frame selected by other criteria).

[0132] The intermediate video frame data is subtracted from the previously calculated multi-channel average to obtain a multi-channel residual image. The residual image reveals changes relative to the average state, which helps detect anomalies or dynamic changes. For example, if the input multiple mine video frame data are three consecutive frames, the middle frame of the three consecutive frames is taken, and the corresponding average frame is subtracted to obtain a multi-channel residual image with a size of 3×448×448.

[0133] Based on multi-channel residual images and multiple original mine video frames, a fusion technique (such as weighted averaging or maximum value fusion) is used to generate a fused feature image. This feature image aims to combine key information from the original video frames with variation information from the residual images.

[0134] Then, feature extraction is performed on the multi-channel residual image to obtain the first feature image. Feature extraction methods include image processing methods such as edge detection and texture analysis. For example, a two-dimensional convolutional layer with a kernel size of 3×3 is used to extract features from the multi-channel residual frame, which are then passed into a batch normalization (BN) layer and a linear rectification function (ReLU) activation function to finally obtain a 1×448×448 feature image (i.e., the first feature image).

[0135] The pixel values ​​at corresponding pixel positions in the first feature image and the fused feature image are then multiplied. This multiplication operation enhances common features in the image while suppressing irrelevant features. For example, by multiplying the pixel values ​​at corresponding pixel positions in the first feature image and the fused feature image element by element, a multiplied feature image is obtained. The first feature image is then used as an attention module and fed into consecutive image frames, allowing the model to focus on the dynamic changes in water flow within the image.

[0136] Finally, the pixel values ​​at corresponding pixel positions in the fused feature image and the product-processed feature image are summed to obtain the final preprocessed mine video frame data. This step combines the information from the two images to generate preprocessed mine video frame data that contains both original information and emphasizes key features. For example, summing the pixel values ​​at corresponding pixel positions in the fused feature image and the product-processed feature image further preserves the feature information of the original image frame. The resulting 32×448×448 feature image (i.e., the preprocessed mine video frame data) is used as the final output of the multi-channel residual attention module.

[0137] The preprocessed mine video frame data contains both original information and highlights key features, while reducing noise and redundancy, providing high-quality visual input for subsequent monitoring, analysis, or automated decision-making.

[0138] The mine video frame data is decomposed into three image channel data: the first channel data (usually the red channel R), the second channel data (usually the green channel G), and the third channel data (usually the blue channel B).

[0139] The corresponding pixel position refers to the pixel located in the same spatial position across all mine video frames. For each such pixel position, the average value of its three channels is calculated over the time axis (i.e., the video frame sequence).

[0140] For the first channel (red channel), the average value of each pixel on the time axis is represented by the following method: sum the first channel data values ​​of the pixel position in all video frames, and then divide by the total number of video frames.

[0141] The average value of the second channel (green channel) and the third channel (blue channel) is calculated in the same way as that of the first channel.

[0142] By calculating the temporal average of each pixel location across the three channels, a stable image representation free from transient noise can be obtained. This processing method is highly useful in tasks such as video analysis, image enhancement, and object detection, especially when dealing with video data from complex environments like mines where lighting conditions can vary, thereby improving the stability and usability of the video data.

[0143] In some embodiments, step B4 includes:

[0144] Step C1: Extract features from each mine video frame data to obtain multiple second feature images.

[0145] Step C2: Based on the pixel values ​​of the multiple second feature images at each pixel position, a feature concatenation algorithm is used to process the data to obtain a concatenated feature image.

[0146] Step C3: Perform fusion processing on the stitched feature images to obtain fused feature images.

[0147] In the above scheme, feature extraction refers to extracting useful information or features from each frame of the image. These features can be color, texture, shape, edges, etc., depending on the problem to be solved and the algorithm used.

[0148] Feature extraction algorithms can extract one or more feature images (referred to here as "secondary feature images") from each frame of an image. These feature images may represent information in the original image in different ways, for example, by emphasizing certain specific visual features or ignoring other unimportant information.

[0149] Multiple second feature images are stitched together using a feature stitching algorithm. Based on their pixel values ​​at each pixel location, they are combined or fused in some form. For example, stitching three consecutive frame feature images (i.e., second feature images) along the channel dimension yields a 24×448×448 feature image (i.e., the stitched feature image).

[0150] Feature stitching algorithms may involve weighted averaging of pixel values, maximum value selection, minimum value selection, or other more complex combination strategies. These algorithms aim to preserve useful information in each feature image while reducing redundancy and noise.

[0151] By processing the feature stitching algorithm, a stitched feature image can be obtained, which integrates information from multiple second feature images.

[0152] Then, the stitched feature images undergo further fusion processing to integrate and optimize the information within them, thereby improving the quality and usability of the feature images. For example, this can be achieved using an image segmentation network (U... 2 The residual shrinkage layer (RSU4) in the Net fuses and optimizes the stitched features to maintain a balance between low-level features (such as edges and textures) and high-level features (such as the partial and overall structure of objects), ultimately resulting in a 32×448×448 feature image (i.e., the fused feature image).

[0153] RSU4 performs several convolutional operations to extract features layer by layer. Downsampling can be achieved through convolution or pooling operations, gradually reducing the spatial dimension of the feature map to extract features at different scales. After the encoder extracts multi-scale features, the decoder branch of RSU4 performs upsampling operations, gradually restoring the spatial size of the feature map to 448×448, and then fuses the upsampled features from the decoder with the features extracted by the encoder through skip connections, finally obtaining a 32×448×448 feature image (i.e., the fused feature image).

[0154] The fusion process may involve various techniques, such as multi-scale fusion, multi-resolution fusion, and learning-based fusion. These techniques aim to leverage the complementarity between different feature images to improve the robustness and accuracy of the feature images.

[0155] Through fusion processing, a fused feature image is finally obtained. This feature image may more accurately represent the key information in the mine video frame data, helping to improve the accuracy of subsequent analysis and recognition tasks.

[0156] In some embodiments, step C2 includes:

[0157] Based on the pixel values ​​at each pixel position of the multiple second feature images, the stitched feature image is determined by the following formula:

[0158]

[0159] in, Indicates the pixel position of the stitched feature image The value at that location, This indicates the pixel position of the first second feature map among multiple second feature maps. Pixel value at that location, This indicates the second second feature map among multiple second feature maps at pixel location. Pixel value at that location, This indicates the third second feature map among multiple second feature maps at pixel location. The pixel value at that location.

[0160] In the above scheme, since all feature images are assumed to have the same size, the pixel values ​​of each image at the same location can be directly compared or combined.

[0161] By combining the pixel values ​​at the same location from multiple second feature images, a stitched feature image that integrates the advantages of all input images is accurately constructed.

[0162] In some embodiments, step C3 includes:

[0163] Step D1: Perform downsampling convolution processing on the stitched feature image to obtain downsampling convolution features.

[0164] Step D2: Perform upsampling convolution processing on the stitched feature image to obtain the second upsampling convolution feature.

[0165] Step D3: The downsampled convolutional features and the second upsampled convolutional features are fused to obtain a fused feature image.

[0166] In the above approach, downsampling convolutions can reduce the spatial dimensions (width and height) of the feature image while increasing its depth (number of channels). This is typically achieved by applying convolutional kernels with a stride greater than 1, or by using pooling layers. The purpose of downsampling is to reduce computation, control overfitting, and help the model learn the hierarchical structure of the image.

[0167] Downsampled convolutional features have smaller spatial dimensions and potentially greater depth than the original stitched feature image.

[0168] In contrast to downsampling convolution, upsampling convolution aims to increase the spatial dimension of a feature image while potentially reducing its depth. This is typically achieved through transposed convolution (also known as deconvolution) or upsampling layers such as bilinear interpolation. Upsampling is particularly important in tasks such as generative adversarial networks (GANs) and image segmentation because it helps recover spatial details from an image.

[0169] The downsampled convolutional features and the second upsampled convolutional features are combined and fused to form a fused feature image. Feature fusion can be achieved through simple concatenation, addition, element-wise multiplication, or more complex operations (such as attention mechanisms).

[0170] The fused feature images combine features from different scales or levels, which helps the model to understand the image content more comprehensively and is crucial for improving the performance of tasks such as image classification, detection, and segmentation.

[0171] In some embodiments, such as Figure 2C As shown, a multi-channel residual preprocessing method is used to average the original video frames for each pixel along the time dimension, resulting in an average frame for the RGB three channels. Subsequently, the average frame for the multi-channel video frames is subtracted from the intermediate video frames to generate a multi-channel residual image.

[0172] like Figure 2DAs shown, this process extracts multiple intermediate frames from consecutive frames and generates corresponding mask labels for these intermediate frames. By fusing frame information from multiple time points, precise features of dynamic targets (such as water flow) are extracted to reduce noise or instability that may exist in a single frame. The mask labels are used to annotate target regions in the intermediate frames; the white parts represent the target that the model needs to learn, while the black parts represent background information.

[0173] like Figure 2E The results of the detection of a certain water hazard risk video are based on a video of water inrush at the bottom of the well working face. Figure 2E The original video frame in (a) shows water continuously gushing out from the aquifer at the bottom, like a fountain, with the volume increasing over time. Final detection results. Figure 2E As shown in (d), the method effectively segmented the water inrush area, with a Dice similarity coefficient (Dice) of 86.75% and an Intersection over Union (IoU) value of 80.23%.

[0174] In some embodiments, the mine water inrush identification process of this application is as follows: Figure 2F As shown:

[0175] Step 201: Obtain the mine water hazard risk dataset;

[0176] Step 202: Perform video frame parsing on the mine water hazard risk dataset;

[0177] Step 203: The multi-channel residual attention module preprocesses the video frame data after video parsing to obtain preprocessed video frame data.

[0178] Step 204: Input the preprocessed video frame data into the pre-built initial model (CBAM-U²Net).

[0179] Step 205: Perform backpropagation training on the initial model using intermediate frames to obtain the video segmentation model;

[0180] Step 206: Use a video segmentation model to make a prediction and obtain the predicted mine water inrush area.

[0181] The preprocessing process of the multi-channel residual attention module in step 203 is as follows: Figure 2G As shown:

[0182] Step 2031: Parse the mine water hazard risk dataset into mine water hazard risk video frames;

[0183] Step 2032: Determine the multi-channel average frame;

[0184] Step 2033: Perform multi-channel residual preprocessing based on multi-channel average frames to obtain multi-channel residual images;

[0185] Step 2034: Extract features from the multi-channel residual image to obtain the first feature image;

[0186] Step 2035: Perform multi-frame fusion based on multi-channel residual images and multiple mine video frame data to determine the fused feature image;

[0187] Step 2036: During the fusion process, feature extraction is performed on each mine video frame data to obtain multiple second feature images. Based on the pixel values ​​of the multiple second feature images at each pixel position, feature stitching is performed to obtain a stitched feature image.

[0188] Step 2037: The stitched feature images are fused using the initial model (CBAM-RSU4) to obtain the fused feature images;

[0189] Step 2038: The attention module is used to multiply the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image to obtain the multiplied feature image; the pixel values ​​at corresponding pixel positions of the fused feature image and the multiplied feature image are summed to obtain the preprocessed mine video frame (i.e., the output feature).

[0190] Step 2039: Output the preprocessed mine video frame data (i.e., output features).

[0191] In summary, the features of this application are as follows:

[0192] 1) The introduction of the multi-channel residual attention mechanism effectively eliminates the interference of static background by processing the residual information of RGB channels, enhances the model's ability to capture dynamic features of water flow, and improves the accuracy and robustness of water hazard identification. In particular, it shows strong generalization ability in complex downhole environments.

[0193] 2) The improved U²Net network structure, combined with the CBAM module, introduces a convolutional block attention layer (CBAM) into the traditional U²Net network, which enhances the attention to water flow features and the suppression of background interference, significantly improving the performance in water flow edge detection and complex background segmentation, and ensuring efficient water hazard identification in the variable environment of mines.

[0194] 3) Application of multi-frame fusion learning strategy: A multi-frame fusion learning method is proposed. By using continuous frames and intermediate frame masks as labels, the model's ability to learn about the continuity and dynamic changes of water flow is enhanced, and the detection accuracy of the model in sudden water disaster events is improved, especially in cross-scene water disaster detection.

[0195] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0196] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0197] Based on the same inventive concept, and corresponding to any of the above embodiments, this application also provides a mine water inrush identification device.

[0198] refer to Figure 3 The mine water inrush identification device includes:

[0199] The training acquisition module 301 is configured to acquire preprocessed mine video frame data;

[0200] The training input module 302 is configured to input preprocessed mine video frame data into a pre-built initial model, the initial model including a residual shrinking layer and a convolutional block attention layer, the residual shrinking layer including a first sub-residual shrinking layer and a second sub-residual shrinking layer;

[0201] Feature extraction module 303 is configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data;

[0202] The feature adjustment module 304 is configured to use the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data.

[0203] Upsampling module 305 is configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain the first upsampling convolution feature;

[0204] The splicing processing module 306 is configured to splice the adjusted multi-scale feature data and the upsampled convolutional features to obtain the trained prediction of the mine water inrush area.

[0205] The intermediate frame determination module 307 is configured to determine intermediate video frame data from the preprocessed mine video frame data and obtain the actual mine water inrush label of the intermediate video frame data.

[0206] Backpropagation module 308 is configured to backpropagate and adjust the initial model using the actual mine water inrush labels and the trained predicted mine water inrush areas to obtain a video segmentation model.

[0207] The prediction acquisition module 309 is configured to acquire video frame data of the mine to be predicted.

[0208] The identification and prediction module 310 is configured to input the video frame data of the mine to be predicted into the video segmentation model, and to identify and predict the video frame data of the mine to be predicted through the video segmentation model to obtain the predicted water inrush area of ​​the mine.

[0209] In some embodiments, the training acquisition module 301 includes:

[0210] The acquisition submodule is configured to acquire video datasets within the mine.

[0211] The video parsing submodule is configured to perform video parsing on the video dataset to obtain multiple mine video frame data;

[0212] The residual processing submodule is configured to perform multi-channel residual processing on the multiple mine video frame data to obtain preprocessed mine video frame data.

[0213] In some embodiments, the residual processing submodule includes:

[0214] The acquisition unit is configured to acquire image channel data for each pixel position in each mine video frame data;

[0215] The average value processing unit is configured to process the image channel data based on the pixel positions of each mine video frame data through a multi-channel average value algorithm to obtain the average value of each pixel on the time axis in each image channel data.

[0216] The difference processing unit is configured to determine the intermediate video frame data of multiple mine video frame data, and perform difference processing using the intermediate video frame data and the average value to obtain a multi-channel residual image;

[0217] The fusion unit is configured to determine a fused feature image based on the multi-channel residual image and the multiple mine video frame data;

[0218] The feature extraction unit is configured to extract features from the multi-channel residual image to obtain a first feature image;

[0219] The product processing unit is configured to multiply the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image to obtain the product-processed feature image.

[0220] The summation processing unit is configured to sum the pixel values ​​at corresponding pixel positions of the fused feature image and the multiplicative feature image to obtain preprocessed mine video frame data.

[0221] In some embodiments, the fusion unit includes:

[0222] The feature extraction subunit is configured to extract features from each mine video frame data to obtain multiple second feature images;

[0223] The splicing subunit is configured to process the pixel values ​​at each pixel position of the plurality of second feature images using a splicing feature algorithm to obtain a spliced ​​feature image;

[0224] The fusion subunit is configured to perform fusion processing on the stitched feature image to obtain a fused feature image.

[0225] In some embodiments, the splicing subunit is specifically configured as follows:

[0226] Based on the pixel values ​​at each pixel position of the multiple second feature images, the stitched feature image is determined by the following formula:

[0227]

[0228] in, Indicates the pixel position of the stitched feature image The value at that location, This indicates the pixel position of the first second feature map among multiple second feature maps. Pixel value at that location, This indicates the second second feature map among multiple second feature maps at pixel location. Pixel value at that location, This indicates the third second feature map among multiple second feature maps at pixel location. The pixel value at that location.

[0229] In some embodiments, the fusion subunit is specifically configured as follows:

[0230] The stitched feature image is subjected to downsampling convolution processing to obtain downsampling convolution features;

[0231] The stitched feature image is subjected to upsampling convolution processing to obtain the second upsampling convolution feature;

[0232] The downsampled convolutional features and the second upsampled convolutional features are fused to obtain a fused feature image.

[0233] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0234] The apparatus described above is used to implement the corresponding mine water inrush identification method in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0235] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the mine water inrush identification method described in any of the above embodiments.

[0236] Figure 4 This illustration shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 401, a memory 402, an input / output interface 403, a communication interface 404, and a bus 405. The processor 401, memory 402, input / output interface 403, and communication interface 404 are interconnected internally via the bus 405.

[0237] The processor 401 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0238] The memory 402 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 402 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 402 and is called and executed by the processor 401.

[0239] Input / output interface 403 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0240] Communication interface 404 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0241] Bus 405 includes a pathway for transmitting information between various components of the device, such as processor 401, memory 402, input / output interface 403, and communication interface 404.

[0242] It should be noted that although the above-described device only shows the processor 401, memory 402, input / output interface 403, communication interface 404, and bus 405, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0243] The electronic devices described above are used to implement the corresponding mine water inrush identification methods in any of the foregoing embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0244] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to execute the mine water inrush identification method as described in any of the above embodiments.

[0245] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0246] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to execute the mine water inrush identification method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0247] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0248] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, the apparatus may be shown in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of the implementation of these block diagram apparatuses are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0249] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0250] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for identifying mine water inrush, characterized in that, include: Collect video datasets from inside the mine; The video dataset was parsed to obtain multiple mine video frame data; Obtain the image channel data for each pixel position in the video frame data of each mine; The image channel data based on the pixel positions of each mine video frame is processed by a multi-channel averaging algorithm to obtain the average value of each pixel on the time axis in each image channel data. The intermediate video frame data of multiple mine video frame data is determined, and the difference between the intermediate video frame data and the average value is calculated to obtain a multi-channel residual image. Based on the multi-channel residual image and the multiple mine video frame data, a fused feature image is determined; Feature extraction is performed on the multi-channel residual image to obtain a first feature image; The pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image are multiplied to obtain the feature image after multiplication. The pixel values ​​at corresponding pixel positions of the fused feature image and the multiplicative feature image are summed to obtain preprocessed mine video frame data. The preprocessed mine video frame data is input into a pre-built initial model, which includes a residual shrinking layer and a convolutional block attention layer. The residual shrinking layer includes a first sub-residual shrinking layer and a second sub-residual shrinking layer. The preprocessed mine video frame data is used to extract features through the first sub-residual shrinkage layer to obtain multi-scale feature data. The multi-scale feature data is adjusted using the convolutional block attention layer to obtain adjusted multi-scale feature data. The preprocessed mine video frame data is upsampled and convolved by the second sub-residual shrinkage layer to obtain the first upsampled convolution feature. The adjusted multi-scale feature data and the upsampled convolutional features are concatenated to obtain the trained prediction of the mine water inrush area. Determine intermediate video frame data from preprocessed mine video frame data, and obtain the actual mine water inrush label of the intermediate video frame data; The initial model is adjusted by backpropagation using the actual mine water inrush labels and the trained predicted mine water inrush areas to obtain a video segmentation model. Acquire video frame data of the mine to be predicted; The video frame data of the mine to be predicted is input into the video segmentation model, and the video segmentation model is used to identify and predict the video frame data of the mine to be predicted, so as to obtain the predicted water inrush area of ​​the mine.

2. The method according to claim 1, characterized in that, The process of determining the fused feature image based on the multi-channel residual image and the multiple mine video frame data includes: Feature extraction is performed on each mine video frame data to obtain multiple second feature images; The pixel values ​​at each pixel position of the multiple second feature images are processed by a concatenation feature algorithm to obtain a concatenated feature image; The stitched feature images are fused to obtain fused feature images.

3. The method according to claim 2, characterized in that, The pixel values ​​at each pixel position based on the multiple second feature images are processed by a concatenation feature algorithm to obtain a concatenated feature image, including: Based on the pixel values ​​at each pixel position of the multiple second feature images, the stitched feature image is determined by the following formula: in, Indicates the pixel position of the stitched feature image The value at that location, This indicates the pixel position of the first second feature map among multiple second feature maps. Pixel value at that location, This indicates the second second feature map among multiple second feature maps at pixel location. Pixel value at that location, This indicates the third second feature map among multiple second feature maps at pixel location. The pixel value at that location.

4. The method according to claim 2, characterized in that, The process of fusing the spliced ​​feature images to obtain the fused feature image includes: The stitched feature image is subjected to downsampling convolution processing to obtain downsampling convolution features; The stitched feature image is subjected to upsampling convolution processing to obtain the second upsampling convolution feature; The downsampled convolutional features and the second upsampled convolutional features are fused to obtain a fused feature image.

5. A mine water inrush identification device, characterized in that, include: The training acquisition module is configured to collect video datasets from inside the mine. The video dataset is parsed to obtain multiple mine video frame data; image channel data for each pixel position in each mine video frame data is obtained; based on the image channel data for the corresponding pixel positions in each mine video frame data, a multi-channel averaging algorithm is used to process the average value of each pixel on the time axis in each image channel data; intermediate video frame data of the multiple mine video frame data is determined, and the difference between the intermediate video frame data and the average value is calculated to obtain a multi-channel residual image; a fused feature image is determined based on the multi-channel residual image and the multiple mine video frame data; features are extracted from the multi-channel residual image to obtain a first feature image; the pixel values ​​at corresponding pixel positions of the first feature image and the fused feature image are multiplied to obtain a multiplied feature image; the pixel values ​​at corresponding pixel positions of the fused feature image and the multiplied feature image are summed to obtain preprocessed mine video frame data; The training input module is configured to input preprocessed mine video frame data into a pre-built initial model, the initial model including a residual shrinking layer and a convolutional block attention layer, the residual shrinking layer including a first sub-residual shrinking layer and a second sub-residual shrinking layer; The feature extraction module is configured to extract features from the preprocessed mine video frame data through the first sub-residual shrinkage layer to obtain multi-scale feature data; The feature adjustment module is configured to use the convolutional block attention layer to perform feature adjustment on the multi-scale feature data to obtain adjusted multi-scale feature data. The upsampling module is configured to perform upsampling convolution processing on the preprocessed mine video frame data through the second sub-residual shrinkage layer to obtain the first upsampling convolution feature; The splicing processing module is configured to splice the adjusted multi-scale feature data and the upsampled convolutional features to obtain the trained prediction of the mine water inrush area. The intermediate frame determination module is configured to determine intermediate video frame data from preprocessed mine video frame data and obtain the actual mine water inrush label of the intermediate video frame data. The backpropagation module is configured to perform backpropagation adjustment on the initial model using the actual mine water inrush labels and the trained predicted mine water inrush areas to obtain a video segmentation model. The prediction acquisition module is configured to acquire video frame data of the mine to be predicted; The identification and prediction module is configured to input the video frame data of the mine to be predicted into the video segmentation model, and to identify and predict the video frame data of the mine to be predicted through the video segmentation model to obtain the predicted water inrush area of ​​the mine.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 4.

Images