Method for identifying state of underground air door
The state of the downhole damper is identified through image acquisition and neural network attention mechanism, which solves the problem of ventilation system damage caused by the long-term closing or not being closed tightly, and achieves normal access to wind flow and efficient operation of ventilation system.
Patent Information
- Application Number
- CN202411711143.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-26
- Publication Date
- 2025-03-28
AI Technical Summary
The coal mine sanitation gates are closed or not tight for a long time, which will damage the ventilation system of the coal mine and cause harmful gases to accumulate, bringing safety hazards to coal mine work.
A method for identifying the state of the downhole damper is proposed, and multiple probability calculations of the damper state are carried out through image acquisition, Gaussian background modeling, feature vector extraction and the attention mechanism of the neural network to accurately identify the switching state of the damper.
Ensure that fresh airflow can reach all wind-using locations, avoid short circuits or disorders of airflow, optimize the ventilation system of the mine, make the ventilation system operate more efficiently, and reduce energy waste.
Smart Images
Figure CN119723120A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of coal mine production safety, and in particular to a method for identifying the status of an underground air door. Background Art
[0002] Coal mine dampers are an important equipment in the coal mining industry. The function of the dampers is to ensure safe production in the mine, the stability of the ventilation system, reduce ineffective air leakage, and ensure the normal breathing of the workers at the bottom of the mine, and the discharge of toxic substances. However, if the dampers are closed for a long time or not closed tightly, it will damage the ventilation system of the coal mine, causing the accumulation of harmful gases and posing a safety hazard to coal mine work. Summary of the invention
[0003] The purpose of this application is to solve one of the technical problems in the related art at least to some extent.
[0004] To this end, the first purpose of this application is to propose a method for identifying the status of underground dampers so as to accurately identify the switch status of the dampers, thereby ensuring that fresh air can reach various air-using locations, avoiding air short circuit or turbulence, and thus optimizing the mine ventilation system.
[0005] To achieve the above-mentioned purpose, the first aspect of the present application proposes a method for identifying the state of an underground damper, comprising: performing image acquisition on the damper to obtain an image of the damper, and performing Gaussian background modeling based on the region of interest in the image to determine the background image and foreground image of the damper; extracting a first feature vector of the damper from the background image and a second feature vector of the damper from the foreground image based on the first region of interest in the background image and the second region of interest in the foreground image; determining the cosine values of the first feature vector and the second feature vector; in response to the cosine value being greater than a set threshold, inputting the foreground image into a neural network, and performing multiple probability calculations on the state of the damper in the foreground image based on the attention mechanism of the neural network, and determining a target probability value of the state of the damper in the foreground image based on the probability calculation results; and determining a target state of the damper based on the target probability value.
[0006] The method for identifying the state of an underground damper provided in the present application can determine the foreground image and background image of the damper by collecting an image of the underground damper and performing Gaussian background modeling on the image. Then, according to the respective regions of interest in the foreground image and the background image, the first eigenvector and the second eigenvector of the damper can be extracted, and the cosine value between the first eigenvector and the second eigenvector can be determined to make a preliminary judgment on the state of the damper according to the cosine value. When it is determined that the cosine value is greater than the set threshold, the state of the damper in the foreground image is further identified based on the neural network. By performing multiple probability calculations on the state of the damper, the target probability value of the damper state can be determined, and the target state of the damper can be determined according to the target probability value. By accurately identifying the switch state of the damper, it can be ensured that the fresh airflow can reach each air-using location, avoiding short circuit or turbulence of the airflow, thereby optimizing the ventilation system of the mine, making the ventilation system more efficient, and reducing energy waste.
[0007] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0008] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0009] Figure 1 A schematic flow chart of a method for identifying a downhole damper state provided in an embodiment of the present application;
[0010] Figure 2 A flow chart of another method for identifying the state of a downhole damper provided in an embodiment of the present application;
[0011] Figure 3 A flow chart of another method for identifying the state of a downhole damper provided in an embodiment of the present application;
[0012] Figure 4 A schematic diagram of the structure of a downhole damper status identification device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0013] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.
[0014] The following describes a method and device for identifying the status of a downhole damper according to an embodiment of the present application with reference to the accompanying drawings.
[0015] Figure 1 is a flow chart of a method for identifying a downhole damper state according to an exemplary embodiment. Figure 1 As shown, the method for identifying the downhole damper state of the embodiment of the present application includes but is not limited to the following steps:
[0016] S101, performing image acquisition on the damper to obtain an image of the damper, and performing Gaussian background modeling based on a region of interest in the image to determine a background image and a foreground image of the damper.
[0017] It should be noted that the execution subject of the method for identifying the state of the underground damper provided in the embodiment of the present application is an electronic device, which may be a terminal device. Optionally, the terminal device may be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc., and the non-mobile electronic device may be a personal computer (PC), a television, etc. The embodiment of the present application is not specifically limited.
[0018] In some implementations, an image acquisition device may be installed around the damper to acquire an image of the damper. For example, a camera may be installed opposite the damper to acquire an image of the area including the damper to obtain an image of the damper.
[0019] Further, the region of interest is determined from the acquired image of the damper. Optionally, the position of the damper in the image may be identified, and the region of interest is determined based on the position, that is, the region of interest includes the damper.
[0020] Further, Gaussian background modeling is performed on the region of interest to obtain background image and foreground image of the damper. Optionally, multiple frames of images of the damper can be collected, and Gaussian background modeling is performed based on the region of interest in the multiple frames of images to obtain accurate foreground image and background image.
[0021] S102, based on the first region of interest in the background image and the second region of interest in the foreground image, extract a first feature vector of the damper from the background image, and extract a second feature vector of the damper from the foreground image.
[0022] In some implementations, edge detection may be performed on the first region of interest in the background image to determine edge features of the damper, and the gradient direction of each pixel may be determined based on the edge features, and then the first feature vector of the damper may be determined based on the feature direction of each pixel.
[0023] In some implementations, edge detection is performed on the second region of interest in the foreground image to determine edge features of the damper, and the gradient direction of each pixel is determined based on the edge features, and then the second feature vector of the damper can be determined based on the feature direction of each pixel.
[0024] Optionally, an edge detection algorithm may be used to perform edge detection on the region of interest to obtain edge features of the damper. For example, a Canny edge detection algorithm may be used to perform edge detection.
[0025] S103, determining the cosine values of the first eigenvector and the second eigenvector.
[0026] Optionally, the cosine value of the first eigenvector and the second eigenvector can be obtained by calculating the cosine value of the angle between the first eigenvector and the second eigenvector. In other words, the cosine value can be determined by obtaining the dot product of the first eigenvector and the second eigenvector and the modulus, that is, the length, of the first eigenvector and the second eigenvector.
[0027] Furthermore, the cosine value can be used to preliminarily determine whether the damper is in an open state or in a closed state. If the cosine value is greater than the set threshold, it means that the similarity between the first eigenvector and the second eigenvector is large, and it is determined that the damper is closed; if the cosine value is less than or equal to the set threshold, it means that the similarity between the first eigenvector and the second eigenvector is small, and it is determined that the damper is not closed.
[0028] It is understandable that due to the high noise in the mine, image recognition will be affected. For example, dim light, flickering of mining lamps, flickering of vehicle lights, etc., a series of noises will affect the judgment of whether the damper is open or closed. In addition, since there are many angles for the damper to be open, such as 15 degrees, 30 degrees, and 45 degrees, coupled with the influence of the noise in the mine, the state of the damper switch is very similar, and further identification is required for the damper that is initially judged to be closed.
[0029] S104, in response to the cosine value being greater than a set threshold, the foreground image is input into the neural network, and based on the attention mechanism of the neural network, multiple probability calculations are performed on the state of the damper in the foreground image, and a target probability value of the damper state in the foreground image is determined based on the probability calculation results.
[0030] In some implementations, a neural network based on deep learning may be used to further identify the state of the damper. Optionally, the damper may be divided into different states based on the attention mechanism of the neural network, and the probability values of the damper being in different states may be determined.
[0031] In some implementations, the foreground image is input into a neural network, and the attention mechanism of the neural network performs multiple probability calculations on the state of the damper in the foreground image to obtain multiple probability calculation results, and the multiple probability results are summed to obtain the target probability value of the damper state.
[0032] Optionally, the structure of the neural network includes: a convolution layer, an activation layer, a pooling layer, a feature layer, an attention mechanism layer, and a fully connected layer. The fully connected layer in the neural network performs multiple probability calculations on the state of the damper in the foreground image to obtain a probability calculation result.
[0033] S105: Determine a target state of the damper based on the target probability value.
[0034] In some implementations, the target state of the damper may be determined based on the target probability value and the probability threshold. Optionally, the target state of the damper may be determined by determining a probability threshold that the damper is in a closed state and comparing the target probability value with the probability threshold.
[0035] Optionally, if the target probability value is less than or equal to the probability threshold, the target state of the damper is determined to be a closed state; if the target probability value is greater than the probability threshold, the target state of the damper is determined to be an open state.
[0036] That is, the first damper state recognition is performed by the cosine value between the first eigenvector and the second eigenvector. If it is determined that the cosine value is greater than the set threshold, the second damper state recognition is performed based on the neural network to determine the target state of the damper. If it is determined that the cosine value is less than or equal to the set threshold, it can be determined that the target state of the damper is the open state.
[0037] In the method for identifying the state of an underground damper provided in an embodiment of the present application, by collecting an image of the underground damper and performing Gaussian background modeling on the image, the foreground image and background image of the damper can be determined. Then, according to the respective regions of interest in the foreground image and the background image, the first eigenvector and the second eigenvector of the damper can be extracted, and the cosine value between the first eigenvector and the second eigenvector can be determined to make a preliminary judgment on the state of the damper according to the cosine value. When it is determined that the cosine value is greater than the set threshold, the state of the damper in the foreground image is further identified based on the neural network. By performing multiple probability calculations on the state of the damper, the target probability value of the damper state can be determined, and the target state of the damper can be determined according to the target probability value. By accurately identifying the switch state of the damper, it can be ensured that the fresh airflow can reach each air-using location, avoiding short circuit or turbulence of the airflow, thereby optimizing the ventilation system of the mine, making the ventilation system more efficient, and reducing energy waste.
[0038] Figure 2 is a flow chart of a method for identifying a downhole damper state according to an exemplary embodiment. Figure 2 As shown, the method for identifying the downhole damper state of the embodiment of the present application includes but is not limited to the following steps:
[0039] S201, performing image acquisition on the damper to obtain an image of the damper, and performing Gaussian background modeling based on a region of interest in the image to determine a background image and a foreground image of the damper.
[0040] In the embodiment of the present application, the implementation method of step S201 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0041] S202, based on the first region of interest in the background image and the second region of interest in the foreground image, extract a first feature vector of the damper from the background image, and extract a second feature vector of the damper from the foreground image.
[0042] In the embodiment of the present application, the implementation method of step S202 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0043] S203, determining the cosine values of the first eigenvector and the second eigenvector.
[0044] In the embodiment of the present application, the implementation method of step S203 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0045] S204, in response to the cosine value being greater than the set threshold, inputting the foreground image into the neural network.
[0046] Optionally, the target state of the damper can be preliminarily determined by comparing the cosine value with a set threshold value. That is, in response to the cosine value being less than or equal to the set threshold value, the target state of the damper is determined to be an open state.
[0047] S205, performing a convolution operation on the foreground image based on the convolution layer of the first neural network, and inputting the convolved foreground image into the activation layer and the pooling layer for down-sampling to obtain a feature layer corresponding to the foreground image, wherein the feature layer includes two branches: an attention mechanism layer and a fully connected layer.
[0048] In some implementations, the structure of the neural network includes: a convolution layer, an activation layer, a pooling layer, a feature layer, an attention mechanism layer, and a fully connected layer. By inputting the foreground image into the first neural network, the convolution layer of the first neural network performs a convolution operation on the foreground image, and downsampling the convolved foreground image, a feature layer corresponding to the foreground image can be obtained, wherein the feature layer includes two branches: an attention mechanism layer and a fully connected layer.
[0049] Optionally, the convolved foreground image may be downsampled by the activation layer and the pooling layer of the first neural network.
[0050] S206, performing a first probability calculation on the foreground image by the fully connected layer to obtain a first probability value of the foreground image.
[0051] In some implementations, the fully connected layer can perform a first probability calculation on the foreground image based on the SOFTMAX function to obtain a first probability value, recorded as Loss1. Optionally, the first probability value can be determined based on the SOFTMAX function according to the positive sample with the damper in the closed state and the negative sample with the damper in the open state.
[0052] That is to say, by obtaining positive and negative samples of the damper, the damper of the positive sample is in a closed state, and the damper of the negative sample is in an open state, and then the SOFTMAX function can be used to perform a first probability calculation based on the foreground image, positive samples and negative samples to obtain a first probability value.
[0053] S207, the attention mechanism layer processes the foreground image and the second region of interest to obtain a target image of interest, and the target image of interest is input into the second neural network. The fully connected layer in the second neural network performs a second probability calculation on the target image of interest to obtain a second probability value of the target image of interest. The second neural network has different parameters from the first neural network.
[0054] In some implementations, the attention mechanism layer can perform image segmentation on the foreground image based on the second region of interest, and use the image segmentation result as the target image of interest, so that the target image of interest is input into a second neural network with different parameters from the first neural network, and the second neural network performs a second probability calculation based on the target image of interest to obtain a second probability value.
[0055] In some implementations, the second region of interest can be annotated based on the attention mechanism layer to obtain a labeled box of the second region of interest and the center point coordinates of the second region of interest, and based on the labeled box and the center point coordinates, the foreground image can be segmented to obtain a third region of interest in the foreground image.
[0056] Furthermore, the scale information of the foreground image is obtained as the scale feature, and based on the scale feature of the foreground image, the third region of interest is scale-enlarged to obtain the target image of interest. That is, the target image of interest has the same scale feature as the foreground image.
[0057] Furthermore, the target image of interest is input into the second neural network, the fully connected layer corresponding to the target image of interest is determined, and the fully connected layer performs a second probability calculation on the target region of interest to obtain a second probability value of the target image of interest.
[0058] That is to say, the target image of interest is input into the second neural network, the convolution layer of the second neural network performs a convolution operation on the target image of interest, and the convolved target image of interest is input into the activation layer and the pooling layer for down-sampling to obtain the feature layer corresponding to the target image of interest. The feature layer includes two branches: the attention mechanism layer and the fully connected layer.
[0059] Furthermore, the fully connected layer in the second neural network performs a second probability calculation on the target image of interest based on the SOFTMAX function to obtain a second probability value, which is recorded as Loss2.
[0060] Optionally, the different parameters between the first neural network and the second neural network may be: weights, biases, activation function parameters, or other parameters used for the network architecture, etc., which are not specifically limited in the embodiments of the present application.
[0061] S208, repeatedly inputting the foreground image into the first neural network and subsequent steps to obtain a third probability value and a fourth probability value of the foreground image, and determining a target probability value based on the first probability value, the second probability value, the third probability value and the fourth probability value.
[0062] In some implementations, the foreground image is repeatedly input into the first neural network, and the fully connected layer in the first neural network performs a third probability calculation to obtain a third probability value, which is recorded as Loss3. Furthermore, the target image of interest of the attention mechanism layer in the first neural network is input into the second neural network, and the fully connected layer of the second neural network performs a fourth probability calculation to obtain a fourth probability value, which is recorded as Loss4.
[0063] Furthermore, the target probability value can be obtained by summing the first probability value, the second probability value, the third probability value and the fourth probability value. That is, the target probability value Loss = Loss1 + Loss2 + Loss3 + Loss4.
[0064] For example, the steps of calculating the probability of the foreground image by the neural network are as follows:
[0065] 1. Input the foreground image into the convolution layer of the neural network, and then go through downsampling operations such as activation layer and pooling layer, and the foreground image becomes the c1 feature layer;
[0066] 2. The c1 feature layer now has two branches. One branch generates the attention mechanism layer through convolution, and the other branch generates the fully connected layer for subsequent SOFTMAX classification and generates Loss1.
[0067] 3. The attention mechanism layer mainly contains three variables, cx, cy, and tl, where cx and cy are the coordinates of the center point of the region of interest generated by the attention mechanism, and tl is the side length of the annotation box. Many regions of interest will be generated by the attention mechanism, and these regions are likely to be places with subtle features;
[0068] 4. Feed these attention regions of interest back to the foreground image for matting, then scale them up to the same size as the foreground image and feed them into the neural network again. The design of the neural network in step 4 is the same shape as the neural network in step 1, but the parameters are different. Then repeat steps 2 and 3 to generate Loss2;
[0069] 5. Repeat steps 1, 2, 3, and 4 again, and generate Loss3 and Loss4. The whole process generates 4 recursive attentions, and the final total target probability value LOSS is Loss = Loss1 + Loss2 + Loss3 + Loss4. Through the above fine-grained attention mechanism, the opening and closing state of the damper can be analyzed.
[0070] S209, determining a target state of the damper based on the target probability value.
[0071] In the embodiment of the present application, the implementation method of step S209 can be implemented by any method in the various embodiments of the present application, which is not limited here and will not be repeated.
[0072] In the method for identifying the state of an underground damper provided in an embodiment of the present application, by collecting an image of the underground damper and performing Gaussian background modeling on the image, the foreground image and background image of the damper can be determined. Then, according to the respective regions of interest in the foreground image and the background image, the first eigenvector and the second eigenvector of the damper can be extracted, and the cosine value between the first eigenvector and the second eigenvector can be determined to make a preliminary judgment on the state of the damper according to the cosine value. When it is determined that the cosine value is greater than the set threshold, the state of the damper in the foreground image is further identified based on the neural network. Based on the fully connected layer in the neural network, multiple probability calculations are performed on the damper state, and the target probability value of the damper state can be determined to determine the target state of the damper according to the target probability value. By accurately identifying the switch state of the damper, it can be ensured that the fresh airflow can reach each air-using location, avoiding short circuit or turbulence of the airflow, thereby optimizing the ventilation system of the mine, making the ventilation system more efficient, and reducing energy waste.
[0073] Figure 3 is a flow chart of a method for identifying a downhole damper state according to an exemplary embodiment. Figure 3 As shown, the method for identifying the downhole damper state of the embodiment of the present application includes but is not limited to the following steps:
[0074] S301, performing image acquisition on the damper to obtain an image of the damper, and performing Gaussian background modeling based on a region of interest in the image to determine a background image and a foreground image of the damper.
[0075] In the embodiment of the present application, the implementation method of step S301 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0076] S302, based on the first region of interest in the background image and the second region of interest in the foreground image, extract a first feature vector of the damper from the background image, and extract a second feature vector of the damper from the foreground image.
[0077] In some implementations, for any image in the background image and the foreground image, edge detection is performed on the region of interest in any image to determine the edge features of the damper, and then the map features of the pixel points of the edge features are obtained as pixel information, and based on the pixel information of the edge features, the feature vector corresponding to any image is determined.
[0078] That is to say, when any image is a background image, the feature vector corresponding to any image is a first feature vector. When any image is a foreground image, the feature vector corresponding to any image is a second feature vector.
[0079] S303: Determine the cosine values of the first eigenvector and the second eigenvector.
[0080] In the embodiment of the present application, the implementation method of step S303 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0081] S304, in response to the cosine value being greater than a set threshold, the foreground image is input into the neural network, and based on the attention mechanism of the neural network, multiple probability calculations are performed on the state of the damper in the foreground image, and a target probability value of the damper state in the foreground image is determined based on the probability calculation results.
[0082] In the embodiment of the present application, the implementation method of step S304 can be implemented by any method in the embodiments of the present application, which is not limited here and will not be repeated.
[0083] S305, determining a probability threshold that the damper is in a closed state, and if the target probability value is less than or equal to the probability threshold, determining that the target state of the damper is a closed state.
[0084] S306: If the target probability value is greater than the probability threshold, determine that the target state of the damper is an open state.
[0085] In some implementations, when the target probability value of the damper state is determined, the probability threshold of the damper being in the closed state can be determined, and by comparing the target probability value with the probability threshold, it can be determined whether the target state of the damper is open or closed.
[0086] Optionally, if the target probability value is less than or equal to the probability threshold, the target state of the damper is determined to be a closed state; if the target probability value is greater than the probability threshold, the target state of the damper is determined to be an open state.
[0087] In some implementations, when the target state of the damper is determined to be open, an instruction message may be sent to instruct the damper to close, so as to avoid damaging the mine's ventilation system due to the damper being open for too long, resulting in consequences such as wind turbulence.
[0088] That is, in response to the target state of the damper being the open state, the first state of the damper before the set time is determined, and if the first state is the same as the target state, indication information indicating the damper is closed is generated to perform a closing operation on the damper.
[0089] In some implementations, when the target state of the damper is determined to be a closed state, an instruction message may be sent to instruct the damper to open, so as to avoid the accumulation of harmful gases due to the damper being closed for too long, thereby posing a safety hazard to coal mine work.
[0090] That is, in response to the target state of the damper being the closed state, the second state of the damper before the set time is determined, and if the second state is the same as the target state, indication information indicating that the damper is open is generated to perform an opening operation on the damper.
[0091] In the method for identifying the state of an underground damper provided in an embodiment of the present application, by collecting an image of the underground damper and performing Gaussian background modeling on the image, the foreground image and background image of the damper can be determined. Then, according to the respective regions of interest in the foreground image and the background image, the first eigenvector and the second eigenvector of the damper can be extracted, and the cosine value between the first eigenvector and the second eigenvector can be determined to make a preliminary judgment on the state of the damper according to the cosine value. When it is determined that the cosine value is greater than the set threshold, the state of the damper in the foreground image is further identified based on the neural network. By performing multiple probability calculations on the state of the damper, the target probability value of the state of the damper can be determined, so as to determine the target state of the damper according to the target probability value and the probability threshold. By accurately identifying the switch state of the damper, it can be ensured that the fresh airflow can reach each air-using location, avoiding short circuit or turbulence of the airflow, thereby optimizing the ventilation system of the mine, making the ventilation system more efficient, and reducing energy waste.
[0092] In order to implement the above-mentioned embodiment, the present application also proposes a device for identifying the status of an underground damper.
[0093] Figure 4 A schematic diagram of the structure of a downhole damper status identification device provided in an embodiment of the present application.
[0094] like Figure 4 As shown, the downhole damper state identification device 400 includes:
[0095] The acquisition module 401 is used to acquire images of the damper to obtain an image of the damper, and perform Gaussian background modeling based on a region of interest in the image to determine a background image and a foreground image of the damper;
[0096] An extraction module 402, configured to extract a first feature vector of the damper from the background image and a second feature vector of the damper from the foreground image based on a first region of interest in the background image and a second region of interest in the foreground image;
[0097] A first determining module 403, configured to determine the cosine values of the first eigenvector and the second eigenvector;
[0098] a second determination module 404, configured to input the foreground image into a neural network in response to the cosine value being greater than a set threshold, and perform multiple probability calculations on the state of the damper in the foreground image based on an attention mechanism of the neural network, and determine a target probability value of the damper state in the foreground image based on the probability calculation results;
[0099] The third determination module 405 is configured to determine a target state of the damper based on the target probability value.
[0100] In a possible implementation of the embodiment of the present application, the second determination module 404 is further configured to: in response to the cosine value being less than or equal to the set threshold, determine that the target state of the damper is an open state.
[0101] In a possible implementation of the embodiment of the present application, the second determination module 404 is further used to: perform a convolution operation on the foreground image based on the convolution layer of the first neural network, and input the convolved foreground image into the activation layer and the pooling layer for downsampling to obtain a feature layer corresponding to the foreground image, wherein the feature layer includes two branches: an attention mechanism layer and a fully connected layer; perform a first probability calculation on the foreground image by the fully connected layer to obtain a first probability value of the foreground image; process the foreground image and the second region of interest by the attention mechanism layer to obtain a target image of interest, and input the target image of interest into the second neural network, and perform a second probability calculation on the target image of interest by the fully connected layer in the second neural network to obtain a second probability value of the target image of interest, wherein the second neural network has different parameters from the first neural network; repeatedly input the foreground image into the first neural network and subsequent steps to obtain a third probability value and a fourth probability value of the foreground image, and determine the target probability value based on the first probability value, the second probability value, the third probability value and the fourth probability value.
[0102] In a possible implementation of the embodiment of the present application, the second determination module 404 is further used to: obtain positive samples and negative samples of the damper, the damper of the positive sample is in a closed state, and the damper of the negative sample is in an open state; perform a first probability calculation based on the foreground image, the positive sample and the negative sample to obtain the first probability value.
[0103] In a possible implementation of an embodiment of the present application, the second determination module 404 is further used to: mark the second region of interest based on the attention mechanism layer to obtain a marking box of the second region of interest and the center point coordinates of the second region of interest; perform image segmentation on the foreground image based on the marking box and the center point coordinates to obtain a third region of interest in the foreground image; and scale the third region of interest based on the scale features of the foreground image to obtain the target image of interest.
[0104] In a possible implementation of an embodiment of the present application, the second determination module 404 is further used to: input the target image of interest into the second neural network, determine the fully connected layer corresponding to the target image of interest, and perform a second probability calculation on the target region of interest by the fully connected layer to obtain a second probability value of the target image of interest.
[0105] In a possible implementation of an embodiment of the present application, the third determination module 405 is further used to: determine a probability threshold that the damper is in a closed state; if the target probability value is less than or equal to the probability threshold, determine that the target state of the damper is a closed state; if the target probability value is greater than the probability threshold, determine that the target state of the damper is an open state.
[0106] In a possible implementation of the embodiment of the present application, the extraction module 402 is also used to: for any image among the background image and the foreground image, perform edge detection on the area of interest in the any image to determine the edge features of the damper; based on the pixel information of the edge features, determine the feature vector corresponding to the any image; wherein, if the any image is the background image, the feature vector corresponding to the any image is the first feature vector; if the any image is the foreground image, the feature vector corresponding to the any image is the second feature vector.
[0107] In a possible implementation of an embodiment of the present application, the third determination module 405 is also used to: in response to the target state of the damper being an open state, determine the first state of the damper before a set time period, and if the first state is the same as the target state, generate indication information indicating that the damper is closed, so as to perform a closing operation on the damper.
[0108] In a possible implementation of an embodiment of the present application, the third determination module 405 is further used to: in response to the target state of the damper being a closed state, determine the second state of the damper before the set time period, and if the second state is the same as the target state, generate indication information indicating that the damper is open, so as to perform an opening operation on the damper.
[0109] In the device for identifying the state of underground dampers provided in the embodiment of the present application, by collecting images of underground dampers and performing Gaussian background modeling on the images, the foreground image and background image of the dampers can be determined. Then, according to the respective regions of interest in the foreground image and the background image, the first eigenvector and the second eigenvector of the damper can be extracted, and the cosine value between the first eigenvector and the second eigenvector can be determined to make a preliminary judgment on the state of the damper according to the cosine value. When it is determined that the cosine value is greater than the set threshold, the state of the damper in the foreground image is further identified based on the neural network. By performing multiple probability calculations on the state of the damper, the target probability value of the state of the damper can be determined, and the target state of the damper can be determined according to the target probability value. By accurately identifying the switch state of the damper, it can be ensured that the fresh airflow can reach each air-using location, avoiding short circuit or turbulence of the airflow, thereby optimizing the ventilation system of the mine, making the ventilation system run more efficiently, and reducing energy waste.
[0110] It should be noted that the above explanation of the embodiment of the method for identifying the status of a downhole damper is also applicable to the device for identifying the status of a downhole damper in this embodiment, and will not be repeated here.
[0111] The collection, storage, use, processing, transmission, provision and application of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.
[0112] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.
[0113] This application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, this application is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting data. In addition, when applicable, personal identifiers are removed from such personal information to protect the privacy of users.
[0114] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.
[0115] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0116] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.
[0117] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0118] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0119] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0120] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0121] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.
Claims
1. A method for identifying the state of an underground damper, characterized in that: The method comprises: Capturing an image of the damper to obtain an image of the damper, and performing Gaussian background modeling based on a region of interest in the image to determine a background image and a foreground image of the damper; Based on a first region of interest in the background image and a second region of interest in the foreground image, extracting a first feature vector of the damper from the background image and extracting a second feature vector of the damper from the foreground image; determining cosine values of the first eigenvector and the second eigenvector; In response to the cosine value being greater than a set threshold, the foreground image is input into a neural network, and based on an attention mechanism of the neural network, a plurality of probability calculations are performed on the state of the damper in the foreground image, and a target probability value of the state of the damper in the foreground image is determined based on the probability calculation results; Based on the target probability value, a target state of the damper is determined.
2. The method according to claim 1, characterized in that The method further comprises: In response to the cosine value being less than or equal to the set threshold, it is determined that the target state of the damper is an open state.
3. The method according to claim 1, characterized in that The attention mechanism based on the neural network performs multiple probability calculations on the state of the damper in the foreground image, and determines a target probability value of the damper state in the foreground image based on the probability calculation results, including: Performing a convolution operation on the foreground image based on the convolution layer of the first neural network, and inputting the convolved foreground image into the activation layer and the pooling layer for downsampling to obtain a feature layer corresponding to the foreground image, wherein the feature layer includes two branches: an attention mechanism layer and a fully connected layer; The fully connected layer performs a first probability calculation on the foreground image to obtain a first probability value of the foreground image; The attention mechanism layer processes the foreground image and the second region of interest to obtain a target image of interest, and the target image of interest is input into a second neural network, and the fully connected layer in the second neural network performs a second probability calculation on the target image of interest to obtain a second probability value of the target image of interest, wherein the second neural network has different parameters from the first neural network; Repeat the steps of inputting the foreground image into the first neural network and subsequent steps to obtain a third probability value and a fourth probability value of the foreground image, and determine the target probability value based on the first probability value, the second probability value, the third probability value and the fourth probability value.
4. The method according to claim 3, characterized in that: The performing a first probability calculation on the foreground image by the fully connected layer to obtain a first probability value of the foreground image includes: Acquire positive samples and negative samples of the damper, wherein the damper of the positive sample is in a closed state, and the damper of the negative sample is in an open state; A first probability calculation is performed based on the foreground image, the positive sample, and the negative sample to obtain the first probability value.
5. The method according to claim 3, characterized in that: The foreground image and the second region of interest are processed by the attention mechanism layer to obtain a target image of interest, including: Annotate the second region of interest based on the attention mechanism layer to obtain a labeling box of the second region of interest and coordinates of a center point of the second region of interest; Based on the marked box and the center point coordinates, the foreground image is segmented to obtain a third region of interest in the foreground image; Based on the scale feature of the foreground image, the third region of interest is scaled up to obtain the target image of interest.
6. The method according to claim 5, characterized in that The step of inputting the target image of interest into a second neural network, and performing a second probability calculation on the target image of interest by a fully connected layer in the second neural network to obtain a second probability value of the target image of interest includes: The target image of interest is input into the second neural network, a fully connected layer corresponding to the target image of interest is determined, and the fully connected layer performs a second probability calculation on the target region of interest to obtain a second probability value of the target image of interest.
7. The method according to claim 1, characterized in that Determining the target state of the damper based on the target probability value includes: Determine a probability threshold that the damper is in a closed state, and if the target probability value is less than or equal to the probability threshold, determine that the target state of the damper is a closed state; If the target probability value is greater than the probability threshold, it is determined that the target state of the damper is an open state.
8. The method according to claim 1, characterized in that The extracting a first feature vector of the damper from the background image and a second feature vector of the damper from the foreground image based on the first region of interest in the background image and the second region of interest in the foreground image comprises: For any image of the background image and the foreground image, edge detection is performed on a region of interest in any image to determine edge features of the damper; Determining a feature vector corresponding to any one of the images based on pixel information of the edge feature; Among them, if any one of the images is the background image, the feature vector corresponding to the any one of the images is the first feature vector; if any one of the images is the foreground image, the feature vector corresponding to the any one of the images is the second feature vector.
9. The method according to claim 1, characterized in that: After determining the target state of the damper, the method further includes: In response to the target state of the damper being an open state, a first state of the damper before a set time period is determined, and if the first state is the same as the target state, indication information indicating that the damper is closed is generated to perform a closing operation on the damper.
10. The method according to claim 9, characterized in that The method further comprises: In response to the target state of the damper being a closed state, a second state of the damper before the set time period is determined, and if the second state is the same as the target state, indication information indicating that the damper is opened is generated to perform an opening operation on the damper.