Mine water inrush cross-scene identification method and related equipment

Abstract

The invention provides a mine water inrush cross-scene identification method and related equipment. The method comprises the following steps: acquiring a to-be-identified video of a mine; performing residual preprocessing on the video to be identified to obtain a residual video frame; performing time feature extraction on the residual video frame to obtain a residual time feature, and performing spatial feature extraction on the residual video frame to obtain a residual spatial feature; fusing the residual time feature and the residual space feature to obtain a fused feature; and determining the mine water inrush type of the to-be-identified video based on the fusion feature.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to a method and related device for identifying mine water inrush across scenarios. Background Art

[0002] The environment underground in mines is complex, and it is difficult to identify the dynamic state of water flow in different scenarios, resulting in low recognition accuracy and low efficiency of existing recognition technologies in mine water disaster monitoring. Summary of the Invention

[0003] The present disclosure proposes a method and related device for identifying mine water inrush across scenarios to solve technical problems such as low recognition accuracy and low efficiency in identifying mine water inrush across scenarios to a certain extent.

[0004] In a first aspect of the present disclosure, there is provided a method for identifying mine water inrush across scenarios, including:

[0005] Obtaining a to-be-processed task from an initiator;

[0006] Obtaining a to-be-identified video of a mine;

[0007] Performing residual preprocessing on the to-be-identified video to obtain residual video frames;

[0008] Performing temporal feature extraction on the residual video frames to obtain residual temporal features, and performing spatial feature extraction on the residual video frames to obtain residual spatial features;

[0009] Fusing the residual temporal features and the residual spatial features to obtain a fused feature;

[0010] Determining the type of mine water inrush of the to-be-identified video based on the fused feature.

[0011] In a second aspect of the present disclosure, there is provided a device for identifying mine water inrush across scenarios, including:

[0012] An obtaining module, configured to obtain a to-be-identified video of a mine;

[0013] A residual preprocessing module, configured to perform residual preprocessing on the to-be-identified video to obtain residual video frames;

[0014] A feature extraction module, configured to perform temporal feature extraction on the residual video frames to obtain residual temporal features, and perform spatial feature extraction on the residual video frames to obtain residual spatial features;

[0015] A fusion module, configured to fuse the residual temporal features and the residual spatial features to obtain a fused feature;

[0016] An identification module, configured to determine the mine water inrush across scenarios of the to-be-identified video based on the fused feature.

[0017] In a third aspect of the present disclosure, there is provided an electronic device, including one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs include instructions for executing the method according to the first aspect.

[0018] In a fourth aspect of the present disclosure, there is provided a non-volatile computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to execute the method according to the first aspect.

[0019] In a fifth aspect of the present disclosure, there is provided a computer program product, including computer program instructions, which, when executed on a computer, cause the computer to execute the method according to the first aspect.

[0020] As can be seen from the above, an identification method for mine water inrush across scenarios and related devices provided by the present disclosure perform residual preprocessing on the video to be identified in the mine to obtain residual video frames, then respectively extract the temporal features and spatial features of these residual video frames, fuse these two types of features to form a fused feature, and finally determine the type of mine water inrush based on this fused feature. By fusing temporal features and spatial features, the accuracy and reliability of mine water inrush identification across scenarios are improved, providing strong technical support for mine safety monitoring and early warning, helping to detect and handle mine water inrush situations in a timely manner, and ensuring mine production safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the following will briefly introduce the drawings required for use in the embodiments or related technology descriptions. Obviously, the drawings in the following description are only embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0022] Figure 1 It is a schematic diagram of the identification architecture for mine water inrush across scenarios in an embodiment of the present disclosure.

[0023] Figure 2 It is a schematic diagram of the hardware structure of an exemplary electronic device in an embodiment of the present disclosure.

[0024] Figure 3 It is a schematic flowchart of the identification method for mine water inrush across scenarios in an embodiment of the present disclosure.

[0025] Figure 4 It is a schematic diagram of the training data in an embodiment of the present disclosure.

[0026] Figure 5 Schematic diagram of residual processing according to an embodiment of the present disclosure.

[0027] Figure 6 Schematic diagram of time feature extraction for residual video frames according to an embodiment of the present disclosure.

[0028] Figure 7 Schematic diagram of spatial feature extraction for residual video frames according to an embodiment of the present disclosure.

[0029] Figure 8 Schematic diagram of the denoising process according to an embodiment of the present disclosure.

[0030] Figure 9 Schematic diagram of the prediction result according to an embodiment of the present disclosure.

[0031] Figure 10 Schematic diagram of the training process of the recognition model according to an embodiment of the present disclosure.

[0032] Figure 11 Schematic diagram of the recognition device for mine water inrush across scenarios according to an embodiment of the present disclosure. Detailed implementation manners

[0033] To make the objectives, technical solutions and advantages of the present disclosure clearer and more understandable, the present disclosure will be further described in detail below with reference to specific embodiments and the accompanying drawings.

[0034] It should be noted that unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present disclosure should have the ordinary meanings understood by those of ordinary skill in the field to which the present disclosure belongs. The "first", "second" and similar terms used in the embodiments of the present disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. The terms such as "including" or "comprising" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0035] It can be understood that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, usage scopes, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to users and the authorization of users should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0036] For example, when a proactive request from a user is received, a prompt message is sent to the user to clearly prompt that the operation requested by the user will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, an application program, a server, or a storage medium that performs the operations of the present disclosure's technical solution based on the prompt message.

[0037] It can be understood that the above notification and user authorization process is only illustrative and does not limit the implementation manner of the present disclosure. Other manners that comply with relevant laws and regulations can also be applied to the implementation manner of the present disclosure.

[0038] Figure 1 The figure shows a schematic diagram of the identification architecture for mine water inrush across scenarios according to an embodiment of the present disclosure. Refer to Figure 1 , the identification architecture 100 for mine water inrush across scenarios may include a server 110, a terminal 120, and a network 130 providing a communication link. The server 110 and the terminal 120 can be connected through the wired or wireless network 130. Among them, the server 110 can be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, security services, and CDN.

[0039] The terminal 120 can be implemented in hardware or software. For example, when the terminal 120 is implemented in hardware, it can be various electronic devices with a display screen and supporting page display, including but not limited to smartphones, tablets, e-book readers, laptop portable computers, and desktop computers, etc. When the terminal 120 device is implemented in software, it can be installed in the above-listed electronic devices; it can be implemented as multiple software or software modules (such as software or software modules for providing distributed services), or it can be implemented as a single software or software module, and no specific limitation is made here.

[0040] It should be noted that the method for identifying mine water inrush across scenarios provided by the embodiments of the present application can be executed by the terminal 120 or by the server 110. It should be understood that Figure 1 the numbers of the terminals, the network, and the server in

[0041] Figure 2 The figure shows a schematic diagram of the hardware structure of an exemplary electronic device 200 provided by an embodiment of the present disclosure. As Figure 2As shown, the electronic device 200 may include: a processor 202, a memory 204, a network module 206, a peripheral interface 208, and a bus 210. Among them, the processor 202, the memory 204, the network module 206, and the peripheral interface 208 are communicatively connected to each other inside the electronic device 200 via the bus 210.

[0042] The processor 202 may be a central processing unit (CPU), a neural network processor (NPU), a microcontroller (MCU), a programmable logic device, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or one or more integrated circuits. The processor 202 may be used to execute functions related to the technologies described in this disclosure. In some embodiments, the processor 202 may further include multiple processors integrated as a single logic component. For example, as Figure 2 shown, the processor 202 may include multiple processors 202a, 202b, and 202c.

[0043] The memory 204 may be configured to store data (e.g., instructions, computer code, etc.). As Figure 2 shown, the data stored in the memory 204 may include program instructions (e.g., program instructions for implementing the mine water inrush cross-scenario recognition method of the embodiments of this disclosure) and data to be processed (e.g., the memory may store configuration files of other modules, etc.). The processor 202 may also access the program instructions and data stored in the memory 204 and execute the program instructions to operate on the data to be processed. The memory 204 may include a volatile storage device or a non-volatile storage device. In some embodiments, the memory 204 may include a random access memory (RAM), a read-only memory (ROM), an optical disc, a magnetic disk, a hard disk, a solid state drive (SSD), a flash memory, a memory stick, etc.

[0044] The network module 206 may be configured to provide communication with other external devices to the electronic device 200 via a network. The network may be any wired or wireless network capable of transmitting and receiving data. For example, the network may be a wired network, a local wireless network (e.g., Bluetooth, WiFi, near field communication (NFC), etc.), a cellular network, the Internet, or a combination of the above. It can be understood that the type of the network is not limited to the above specific examples. In some embodiments, the network module 206 may include any combination of any number of network interface controllers (NICs), radio frequency modules, transceivers, modems, routers, gateways, adapters, cellular network chips, etc.

[0045] The peripheral interface 208 can be configured to connect the electronic device 200 to one or more peripheral devices to achieve information input and output. For example, the peripheral devices may include input devices such as a keyboard, a mouse, a touchpad, a touch screen, a microphone, various sensors, etc., and output devices such as a display, a speaker, a vibrator, an indicator light, etc.

[0046] The bus 210 can be configured to transfer information between various components of the electronic device 200 (such as the processor 202, the memory 204, the network module 206, and the peripheral interface 208), such as an internal bus (e.g., the processor-memory bus), an external bus (USB port, PCI-E bus), etc.

[0047] It should be noted that although the architecture of the above-mentioned electronic device 200 only shows the processor 202, the memory 204, the network module 206, the peripheral interface 208, and the bus 210, in the specific implementation process, the architecture of the electronic device 200 may also include other components necessary for normal execution. In addition, those skilled in the art can understand that the architecture of the above-mentioned electronic device 200 may also only include the components necessary to implement the solution of the embodiments of the present disclosure, and do not necessarily include all the components shown in the figure.

[0048] Mine water disasters are a major type of disaster in mine safety production. Their concealment, suddenness, and disaster-causing nature make water disaster identification and monitoring an important means of preventing accident disasters. With the development of artificial intelligence, visual recognition technology has been widely applied in the fields of underground production and safety monitoring. However, the underground environment is complex, and the dynamic state of water flow in different scenarios is difficult to identify, resulting in the ineffective application of visual recognition technology in mine water disaster monitoring. And the dynamic change of water inrush is exactly the core process of the gradual evolution of water disasters. Therefore, identifying the dynamic change of water inrush is crucial in mine water disaster prevention and control. Although video monitoring has been applied in many underground fields at present, in the field of water inrush monitoring, it mainly relies on sensor monitoring and manual inspection, and the application of video monitoring is relatively less. The main reason is that limited samples cannot cover all water inrush scenarios. When a water inrush phenomenon occurs in an unknown scenario, it is difficult for conventional visual recognition methods to accurately capture the water inrush characteristics. This situation makes the potential of video monitoring in water disaster prevention and control not fully explored. Therefore, how to accurately capture these dynamic characteristics through visual recognition technology has become a key issue. A single image is difficult to effectively reflect the dynamic characteristics of underground water inrush, and deep learning is affected by environmental information during the feature extraction process, resulting in the difficulty of the algorithm to generalize in the variable underground environment. The underground environment is complex and diverse, and limited samples cannot cover all water inrush scenarios. Therefore, how to improve the recognition accuracy and efficiency of mine water inrush across scenarios has become a technical problem that urgently needs to be solved.

[0049] In view of this, the embodiments of the present disclosure provide a method and related equipment for identifying mine water inrush across scenarios. The method preprocesses the residual of the video to be identified in the mine to obtain residual video frames, then extracts the temporal features and spatial features of these residual video frames respectively, fuses these two types of features to form a fused feature, and finally determines the type of mine water inrush based on this fused feature. By fusing the temporal feature and the spatial feature, the accuracy and reliability of identifying mine water inrush across scenarios are improved, providing strong technical support for mine safety monitoring and early warning, helping to detect and handle mine water inrush situations in a timely manner, and ensuring the safety of mine production.

[0050] See Figure 3 , Figure 3 FIG. shows a schematic flowchart of a method for identifying mine water inrush across scenarios according to an embodiment of the present disclosure. The method for identifying mine water inrush across scenarios according to an embodiment of the present disclosure can be deployed on a terminal or a server side. Figure 3 In, the method 300 for identifying mine water inrush across scenarios may further include the following steps.

[0051] In step S310, obtain the video to be identified in the mine.

[0052] Among them, the video to be identified may refer to the video captured by the monitoring system inside the mine. Specifically, the video to be identified may come from various monitoring cameras installed in the mine, such as Internet Protocol Cameras (IPCs). These cameras capture the images inside the mine in real time or regularly and generate video data. When it is necessary to identify whether there is a water inrush scenario in the mine, one or more segments are selected from the videos captured by these cameras as the video to be identified. The video to be identified may include various environmental and equipment conditions inside the mine, such as roadways, working faces, drainage equipment, etc., and may also include dynamic information such as personnel activities and equipment operation. By analyzing and processing the video to be identified, various features related to mine water inrush can be extracted, so as to achieve accurate identification of mine water inrush across scenarios.

[0053] In step S320, perform residual preprocessing on the video to be identified to obtain residual video frames.

[0054] Among them, in the method for identifying cross-scene water inrush in a mine, performing residual preprocessing on the video to be identified can extract residual video frames containing important information or changes from the original video, which can more clearly reflect the dynamic characteristics during mine water inrush. Feature extraction is performed on each video frame, and the obtained features may include color, brightness, texture, motion information, etc., which can reflect the changes in the internal environment and objects of the mine. Based on feature extraction, the residual between adjacent video frames is calculated. The residual refers to the difference in feature values between adjacent frames and reflects the dynamic changes in the video. For cross-scene mine water inrush, obvious dynamic changes will occur during water inrush, such as water flow movement, water level rise, etc., and these changes will be reflected in the residual. Through residual preprocessing, key information related to mine water inrush can be extracted from the original video, providing more valuable input for subsequent feature extraction and identification. This method can reduce the interference of noise and redundant information and improve the accuracy and efficiency of identification.

[0055] In some embodiments, performing residual preprocessing on the video to be identified to obtain residual video frames includes:

[0056] Decomposing the video to be identified to obtain video frames to be identified;

[0057] Converting the video frames to be identified into corresponding grayscale image frames;

[0058] Determining the mode of each pixel on the time axis in consecutive grayscale image frames to obtain a mode frame;

[0059] Performing difference calculation and smoothing on the grayscale image frames and the mode frame to obtain the residual video frames.

[0060] Specifically, an identification model can be used to implement the method for identifying cross-scene water inrush in the embodiments of the present disclosure. During the training process of the identification model, the mine water inrush video dataset and the water flow video dataset can be classified according to the presence or absence of water, and the mine water inrush video dataset can be used as the test set, and the water flow video dataset can be used as the training set to test the cross-scene identification effect, as Figure 4 shown, Figure 4 shows a schematic diagram of training data according to an embodiment of the present disclosure. For the test video, according to the visual characteristics of water inrush, it is subdivided into seepage state, ground flow state (i.e., the characteristics of water flowing on the ground), jet state, and fissure water state. In addition, tests are also carried out in the heading and mining video datasets to verify the practicability and effectiveness of the algorithm. The identification model trained based on the training data can be used for the identification of water inrush types and the segmentation of water inrush areas.

[0061] The trained recognition model decomposes the classified video into video frames and converts them into grayscale images for saving locally. Calculate the mode value of each pixel in the continuous grayscale frames on the time axis, and the calculation formula is as follows:

[0062] In the formula: t is the frame number, M direct (x, y) is the pixel value at position (x, y) in the time mode frame, is the operation of finding the pixel value i that maximizes the subsequent summation, where i ∈ {0,..., 255} represents the set of intensity values, I j (x, y) is the indicator function used to check whether the pixel value at position (x, y) in the th frame is equal to i.

[0063] Perform a difference calculation between the grayscale image frame and the mode frame to "smooth" common changes in the mine environment, such as the movement of miners and the operation of machinery. See Figure 5 , Figure 5 shows a schematic diagram of residual processing according to an embodiment of the present disclosure. In Figure 5 the left figure from top to bottom shows the video frames in the spraying state, the fissure state, and the non-water state in sequence. The right figure shows the video frames after residual processing using formula (1). The areas that show significant differences from the time mode frame are considered potential water flow burst areas, while other areas are regarded as the background.

[0064] In step S330, extract the residual time features from the residual video frames and extract the residual spatial features from the residual video frames.

[0065] Among them, the time features can reveal the dynamic evolution of mine water inrush across scenarios between consecutive frames, such as changes in water flow velocity, rise and fall of water levels, etc. By analyzing these time features, the temporal law of the water inrush process can be better understood. Specific methods may include calculating frame differences, extracting motion vectors, or using time series analysis and other techniques. In contrast, the spatial features can depict the spatial distribution of mine water inrush across scenarios at a certain moment, such as the location of the water inrush point, the diffusion of water flow, etc. It helps to visually identify the water inrush area and its influence range. Methods for extracting spatial features may include image segmentation, edge detection, texture analysis and other image processing techniques. By extracting the time features and spatial features from the residual video frames, more comprehensive and in-depth information about mine water inrush across scenarios can be obtained, which helps to more accurately identify the type of water inrush.

[0066] In some embodiments, extracting the residual time features from the residual video frames includes:

[0067] An average luminance feature is obtained based on the average value of the luminance values of pixel blocks in a plurality of consecutive residual video frames.

[0068] The Fourier transform and normalization are performed on the average luminance feature to obtain the residual time feature.

[0069] Specifically, the average value of the luminance values of each n×n pixel block in m consecutive residual frames is taken to generate an m-dimensional signal, and the Fourier transform is performed on it. The calculation method is as follows:

[0070]

[0071] Subsequently, in order to obtain a descriptor insensitive to amplitude changes, l 2 Normalization is performed, thereby generating a descriptor insensitive to luminance changes. The finally adjusted descriptor is as follows:

[0072]

[0073] See Figure 6 , Figure 6 which shows a schematic diagram of time feature extraction for residual video frames according to an embodiment of the present disclosure. Figure 6 In, 200 frames are extracted as time descriptors, and the signals in different regions are significantly separated after Fourier transform. Among them, red represents non-water areas, and blue represents water areas, indicating that this method has strong monitoring ability for dynamic water area changes.

[0074] In some embodiments, spatial feature extraction is performed on the residual video frames to obtain residual spatial features, including:

[0075] Extracting the local binary feature of a sub-region in the residual video frame;

[0076] Based on the local binary feature, histogram analysis is performed to obtain the normalized local binary histogram feature;

[0077] Determining the local binary histogram feature as the residual spatial feature.

[0078] Specifically, a sub-region (subBox) can be extracted from the residual video frame, and an array matching the shape of the sub-region is initialized to store the LBP features on each channel of each frame image. Through a double loop, the LBP calculation is performed on each channel of each frame. The calculation method is as follows:

[0079]

[0080] In the formula: g p represents the gray level of the p th pixel in the neighborhood, g cis the gray level of the central pixel. Subsequently, the result array is reshaped into a two-dimensional form and histogram analysis is performed, and finally the normalized LBP histogram feature is returned to represent the spatial feature of the video.

[0081] See Figure 7 , Figure 7 shows a schematic diagram of spatial feature extraction for the residual video frame according to an embodiment of the present disclosure. Figure 7 In, the LBP value of the residual image is calculated using formula (4). As Figure 7 shown, this variant designs 8 directly adjacent pixel points. By comparing the brightness value of the central pixel with its surrounding 8 adjacent pixels, the LBP value of the central pixel is calculated. The original LBP operator is extended to a circular neighborhood, the 8 adjacent pixels are repositioned on the circumference, and comparison is performed in a clockwise or counterclockwise manner to determine its LBP value.

[0082] In step S340, the residual temporal feature and the residual spatial feature are fused to obtain a fused feature.

[0083] In some embodiments, merging the residual temporal feature and the residual spatial feature to obtain a fused feature includes:

[0084] Concatenating the residual temporal feature and the residual spatial feature to obtain the fused feature.

[0085] Specifically, the temporal description vector and the spatial description vector are concatenated and input into a random forest model for classification training to divide the image pixels into water areas and non-water areas.

[0086] In step S350, the mine water inrush type of the video to be recognized is determined based on the fused feature.

[0087] Among them, after extracting and fusing the temporal feature and the spatial feature of the residual video frame, the fused feature synthesizes the dynamic changes and spatial distribution information of the mine water inrush across scenes in the temporal dimension and the spatial dimension. Based on this fused feature, various methods can be used to determine the mine water inrush type of the video to be recognized. For example, machine learning algorithms (such as support vector machines, random forests, neural networks, etc.) can be used to classify the fused feature to identify different water inrush types. These algorithms can learn the mapping relationship between the water inrush type and the fused feature from a large amount of data and apply this mapping relationship to predict the water inrush type in the new video to be recognized. In addition, expert experience and prior knowledge of the mine water inrush across scenes can be combined to formulate rules or thresholds, and the water inrush type is determined by comparing the fused feature with these rules or thresholds.

[0088] In some embodiments, determining the mine water inrush type of the video to be recognized based on the fused feature includes:

[0089] Classify the image pixels in the residual video frame into water pixels and non - water pixels based on the fusion feature;

[0090] Eliminate the noisy water pixels in the residual video frame to obtain a denoised video frame;

[0091] Determine the type of mine water inrush based on the denoised video frame.

[0092] Among them, based on the previously extracted fusion feature, each image pixel in the residual video frame can be carefully classified into water pixels and non - water pixels. The pixels classified as water can be further processed to identify and eliminate the noisy water pixels among them. Noisy water pixels may be caused by various factors, such as poor video quality, light changes, or other non - water - inrush - related dynamic changes. By eliminating these noisy pixels, a clearer and more accurate water area distribution map, that is, a denoised video frame, can be obtained. Based on this denoised video frame, the algorithm will comprehensively consider multiple factors such as the diffusion speed, range, and shape of the water area to determine the type of mine water inrush. For example, further analysis of the denoised video frame, such as calculating the growth rate of the water area and analyzing the morphological characteristics of the water area, can be used to obtain an accurate judgment of the water - inrush type.

[0093] This not only improves the accuracy of mine water - inrush type recognition but also enhances the robustness of the algorithm, enabling it to better cope with the complex and changeable mine environment and water - inrush scenarios. By accurately distinguishing water pixels and non - water pixels and effectively eliminating noisy pixels.

[0094] Specifically, after multiple iterations using formula (5), the classification results of the water outlet points show obvious regional characteristics. Especially after the 11th iteration, the original noisy points almost completely disappear, and the image not only clearly shows the main water outlet areas but also precisely retains the water outlet areas at the video edges, as Figure 8 shown, Figure 8 shows a schematic diagram of the denoising process according to an embodiment of the present disclosure.

[0095] Specifically, splice the time - description vector and the space - description vector and input them into a random forest model for classification training to divide the image pixels into water and non - water areas.

[0096] Eliminate the generated scattered noisy water - pixel points, and construct an energy function that describes the pixel points and their interactions. By minimizing this energy function, the denoised image classification result can be obtained, and then the water - inrush area and non - water - inrush area can be effectively identified. The specific calculation method is as follows:

[0097]

[0098] In the formula, the energy function E(x, y) represents the objective to be minimized. The data term E(x, y) represents the probability that the pixel p belongs to a certain label L p (such as foreground or background). The parameter K (p，q) is used for the smoothness term, indicating the cost generated if adjacent pixels p and q are assigned different labels L p and L q . The smoothness term encourages adjacent pixels to maintain the same label to maintain the regional coherence of the image. t(L p ≠L q ) is a transfer function that can adjust the weight of the smoothness term according to the label difference of adjacent pixels.

[0099] In some embodiments, determining the mine water inrush type based on the denoised video frame includes:

[0100] Determining the mine water inrush type based on the video features in the denoised video frame, wherein the mine water inrush type includes: seepage type, ground flow type, jet type, and fissure water type.

[0101] Specifically, as Figure 9 shown, Figure 9 shows a schematic diagram of the prediction result according to an embodiment of the present disclosure. Figure 9 Among them, (a) is the jet state, (b) is the ground flow state, (c) is the fissure state, and (d) is the seepage state. Among these types, the descriptors are particularly prominent in the jet and fissure states. (d) also shows a large amount of noise, mainly caused by the intermittent flashing of rock particles when water seeps from the rock or coal wall. In this case, although accurate annotation at the single-pixel level is challenging, combined with the analysis of the descriptors, these subtle seepage points can still be effectively identified.

[0102] It can be seen that according to the method of the embodiment of the present disclosure, the static background is removed by the residual preprocessing method, and only the dynamic features are retained. In the feature extraction of the water inrush video, it can be divided into spatial and temporal features. The spatial features capture the water flow texture through the local binary pattern (LBP); the temporal features analyze the periodic changes of the water flow through the Fourier transform to describe its dynamic texture. According to the method of the embodiment of the present disclosure, by learning the water flow features, the identification of water inrush from scratch is realized, and the accuracy and reliability of cross-scene identification of mine water inrush are improved.

[0103] In summary, according to the method of the present disclosure embodiment, in mine water inrush identification, due to the difficulty in obtaining water inrush samples in the target scenario, many scenarios are still in an unknown state. At the same time, the high variability of the background makes it infeasible to rely solely on single-scenario training. Therefore, one of the key steps in identifying real mine water inrush videos is to eliminate background interference. The fluidity of water flow is mainly affected by color, texture, ripple, and lighting conditions, while static parts such as rock walls and coal walls will interfere with water flow identification. To overcome these challenges and accurately capture the dynamic changes of water flow, the present invention adopts a background removal method based on residual preprocessing, namely a technique based on the temporal mode frame. The core of this method lies in removing the static background in the video, highlighting the fluidity of water flow, and making the water flow characteristics clearer.

[0104] According to the method of the present disclosure embodiment, by calculating the average brightness of the local area around the pixel and analyzing the temporal characteristics of the water flow, the problem that the brightness of a single pixel is easily affected by noise is solved. In spatio-temporal video data, taking the average of the brightness values of each pixel block of consecutive frames, the generated signal presents a sine waveform, which is consistent with the dynamic properties of water, helps to identify the brightness changes caused by underground water inrush, and realizes real-time monitoring and early warning. The feature descriptor proposed by the present invention has invariance to temporal displacement, brightness displacement, and brightness amplitude, can exclude interference factors such as noise and illumination changes, and ensures reliable discrimination of the presence or absence of water inrush. By calculating the temporal domain and brightness variance changes of the signal through fast Fourier transform, the invariance of the descriptor is enhanced, and it has high monitoring accuracy and stability.

[0105] According to the method of the present disclosure embodiment, a method for effectively capturing the spatial characteristics of water bodies without the need for precise modeling of water body fluctuations and ripples is proposed, which is particularly suitable for mine water inrush monitoring. In view of the highly variable characteristics of the water body morphology, the present invention adopts the local binary pattern (LBP) histogram as a solution. The core advantage of the LBP histogram lies in its simplified processing method, which only depends on the spatial configuration of the pixel neighborhood, thus facilitating the analysis of the water body deformation area. In addition, the LBP histogram has high-dimensional characteristics, significantly enhancing its uniqueness and recognition in feature description, and improving the recognition accuracy of water body characteristics.

[0106] See Figure 10 , Figure 10 shows a schematic diagram of the identification model training process according to the embodiment of the present disclosure. Figure 10 Among them, a dataset of water inrush videos of different visual types underground is collected, including seepage water, gravity water, and confined water, etc., and used as the test set. To improve the generalization ability of the model's feature learning, in this paper, by learning the color, texture, and temporal characteristics of the water flow, the model can understand the water flow changes under different conditions, and this is used as the training set.

[0107] The collected video data is first subjected to frame parsing to convert the continuous video stream into static frames to capture the detailed information in each frame image, and then it is converted into a grayscale image. Based on the parsed frame data, the change of pixel values at each pixel position in the video over time is analyzed, the occurrence frequency of each possible pixel intensity value in the entire video sequence is counted, and finally the pixel intensity value with the highest frequency is selected as the value at that position in the time mode frame. The difference between the grayscale image and the time mode frame is calculated, and finally the residual image is returned for subsequent processing or analysis. Thus, dynamic change elements such as sudden water inrush can be effectively separated from the video.

[0108] The average of the brightness values of each n×n pixel block in consecutive m frames is taken to generate an m-dimensional signal. For any given m-dimensional signal, through fast Fourier transform, the changes in the time domain and brightness variance of the signal are calculated to create the invariance of the descriptor. LBP calculation is performed on each channel of each frame, and then the resulting array is reshaped into a two-dimensional form and histogram analysis is carried out, and finally the normalized LBP histogram features are returned to represent the spatial features of the video.

[0109] Using the time and space descriptors as feature vectors, random forest is used for training to classify each pixel point into water and non-water pixels. During the analysis of the underground sudden water inrush video, the intermittent reflected light on the surface of the rock or coal wall often causes difficulties in image analysis. These reflections form sporadic, noise-like light-reflecting pixels in the video frames. The method based on Markov random field is used to distinguish and eliminate the noise caused by intermittent rock or coal wall reflections and clearly identify the sudden water inrush area.

[0110] It should be noted that the method of the embodiments of the present disclosure can be executed by a single device, such as a computer or a server, etc. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In this case of a distributed scenario, one of the multiple devices can only execute one or more steps of the method of the embodiments of the present disclosure, and these multiple devices will interact with each other to complete the described method.

[0111] It should be noted that some embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order from that in the above embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0112] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present disclosure further provides an identification device for mine water inrush across scenarios. Refer to Figure 11 , for the identification device for mine water inrush across scenarios, the device includes:

[0113] An acquisition module, configured to acquire a video to be identified of a mine;

[0114] A residual preprocessing module, configured to perform residual preprocessing on the video to be identified to obtain residual video frames;

[0115] A feature extraction module, configured to extract temporal features from the residual video frames to obtain residual temporal features, and extract spatial features from the residual video frames to obtain residual spatial features;

[0116] A fusion module, configured to fuse the residual temporal features and the residual spatial features to obtain fusion features;

[0117] An identification module, configured to determine the mine water inrush across scenarios of the video to be identified based on the fusion features.

[0118] For the convenience of description, the above device is described by dividing its functions into various modules. Of course, when implementing the present disclosure, the functions of each module can be implemented in one or more software and / or hardware.

[0119] The device of the above embodiment is used to implement the corresponding method for identifying mine water inrush across scenarios in any of the foregoing embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0120] Based on the same inventive concept, corresponding to the method of any of the above embodiments, the present disclosure further provides a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause the computer to execute the method for identifying mine water inrush across scenarios as described in any of the foregoing embodiments.

[0121] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0122] The computer instructions stored in the storage medium of the above embodiment are used to cause the computer to execute the mine water inrush cross-scenario recognition method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be elaborated here.

[0123] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples; under the idea of the present disclosure, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations in different aspects of the embodiments of the present disclosure as described above. For the sake of brevity, they are not provided in detail.

[0124] In addition, for the sake of simplicity of description and discussion, and in order not to make the embodiments of the present disclosure difficult to understand, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. In addition, the devices may be shown in block diagram form to avoid making the embodiments of the present disclosure difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present disclosure are to be implemented (i.e., these details should be fully within the understanding of those skilled in the art). In the case where specific details (such as circuits) are set forth to describe the exemplary embodiments of the present disclosure, it will be apparent to those skilled in the art that the embodiments of the present disclosure can be implemented without these specific details or with variations of these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0125] Although the present disclosure has been described in connection with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0126] Embodiments of the present disclosure are intended to cover all such alternatives, modifications, and variations that fall within the broad scope of the appended claims. Accordingly, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for identifying water inrush in mines across different scenarios, comprising: Obtain the video to be identified in the mine; Performing residual preprocessing on the video to be identified to obtain a residual video frame; Extracting temporal features of the residual video frame to obtain a residual temporal feature, and extracting spatial features of the residual video frame to obtain a residual spatial feature; Fusing the residual temporal feature and the residual spatial feature to obtain a fused feature; The mine water inrush type of the video to be identified is determined based on the fusion features.

2. The method according to claim 1, wherein: Performing residual preprocessing on the video to be identified to obtain a residual video frame includes: Decomposing the to-be-recognized video to obtain a to-be-recognized video frame; Converting the to-be-recognized video frame into a corresponding grayscale image frame; Determine the mode of each pixel in the continuous grayscale image frames on the time axis to obtain a mode frame; The grayscale image frame and the mode frame are difference-calculated and smoothed to obtain the residual video frame.

3. The method according to claim 1, wherein: Extracting the time feature of the residual video frame to obtain the residual time feature includes: Obtaining an average brightness feature based on an average of brightness values ​​of pixel blocks in a plurality of consecutive residual video frames; The average brightness feature is Fourier transformed and normalized to obtain the residual time feature.

4. The method according to claim 1, wherein: Extracting spatial features from the residual video frame to obtain residual spatial features includes: Extracting local binary features of a sub-region in the residual video frame; Performing histogram analysis based on the local binary features to obtain normalized local binary histogram features; The local binary histogram feature is determined as the residual space feature.

5. The method according to claim 1, wherein: Determining the mine water inrush type of the video to be identified based on the fusion feature includes: Based on the fusion feature, the image pixels in the residual video frame are divided into water area pixels and non-water area pixels; Eliminating the noise water area pixels in the residual video frame to obtain a denoised video frame; The type of mine water inrush is determined based on the denoised video frame.

6. The method according to claim 5, wherein: Determining the type of mine water inrush based on the denoised video frame includes: The mine water inrush type is determined based on the video features in the denoised video frame, wherein the mine water inrush types include: seepage type, ground flow type, injection type and fissure water type.

7. The method according to claim 1, wherein: Combining the residual time feature and the residual space feature to obtain a fusion feature includes: The residual time feature and the residual space feature are concatenated to obtain the fusion feature.

8. A mine water inrush cross-scenario identification device, comprising: An acquisition module, used to acquire the video to be identified in the mine; A residual preprocessing module, used for performing residual preprocessing on the video to be identified to obtain a residual video frame; A feature extraction module, configured to extract temporal features from the residual video frame to obtain a residual temporal feature, and to extract spatial features from the residual video frame to obtain a residual spatial feature; A fusion module, used for fusing the residual time feature and the residual space feature to obtain a fusion feature; An identification module is used to determine the mine water inrush cross-scene of the video to be identified based on the fusion features.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the program. 10 . A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the method according to claim 1 .

Images