Methods and devices for identifying hazardous areas in underground coal mines

Abstract

This invention discloses a method and apparatus for identifying hazardous areas in underground coal mines. The method includes: acquiring underground coal mine images captured by cameras installed underground; inputting the underground coal mine images into a preset hazardous area identification model to identify hazardous areas in the underground coal mine images, wherein the hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data. This invention achieves real-time and accurate identification of hazardous areas in underground coal mines, which helps to improve the safety of underground coal mine production.

Description

Technical Field

[0001] This invention relates to the field of coal mine safety production, and more specifically, to a method and apparatus for identifying dangerous areas in underground coal mines. Background Technology

[0002] In underground coal mine working faces, identifying hazardous areas and preventing unauthorized entry into these areas is crucial for ensuring the safety of workers. However, current methods for identifying hazardous areas in coal mines are not yet fully mature. Compared to other methods, they primarily rely on pre-marking, meaning manually pre-setting hazardous areas. This method is not very adaptable; it fails when the camera rotates or the scene changes, making it difficult to accurately determine hazardous areas in underground coal mine working faces. Therefore, existing technology lacks a reliable solution for accurately identifying hazardous areas in underground coal mine working faces. Summary of the Invention

[0003] In order to solve at least one of the technical problems in the background art, the present invention proposes a method and device for identifying dangerous areas in coal mines.

[0004] To achieve the above objectives, according to one aspect of the present invention, a method for identifying hazardous areas in underground coal mines is provided, the method comprising:

[0005] Acquire underground coal mine images captured by cameras installed underground in the coal mine;

[0006] The underground coal mine image is input into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data.

[0007] Optionally, the method for identifying hazardous areas in underground coal mines further includes:

[0008] The underground personnel in the coal mine images are identified using a preset target detection algorithm model.

[0009] If it is determined that an identified underground worker is located within the danger zone, an early warning message is generated to remind the worker to stay away from the danger zone.

[0010] Optionally, acquiring underground coal mine image data collected by cameras installed underground specifically includes:

[0011] When the coal mining machine is unlocked, real-time video data collected by the camera is acquired, and then the underground coal mine image is extracted from the video data. The camera is activated to collect video data when the coal mining machine is unlocked.

[0012] The method for identifying hazardous areas in underground coal mines also includes:

[0013] Simultaneously with generating the early warning information, a coal mining machine lockout command is generated to lock the coal mining machine according to the coal mining machine lockout command.

[0014] Optionally, the method for identifying hazardous areas in underground coal mines further includes:

[0015] Dangerous areas are labeled on underground coal mine images used for model training to obtain the first labeled image;

[0016] Offline data augmentation processing is performed on each of the first labeled images to obtain the second labeled images;

[0017] Online data augmentation processing is performed on the first labeled image and the second labeled image to obtain the online data-augmented labeled image;

[0018] The image semantic segmentation algorithm model is improved to obtain the improved image semantic segmentation algorithm model;

[0019] The improved image semantic segmentation algorithm model is trained based on the labeled image after online data augmentation, and the trained model is determined as the danger area identification model when the recognition accuracy of the trained model meets the preset standard.

[0020] Optionally, offline data augmentation processing is performed on each of the first labeled images to obtain a second labeled image, specifically including:

[0021] For each of the first labeled images, perform perspective transformations by rotating around the x-axis, perspective transformations by rotating around the image center along two axes, affine transformations, and affine transformations with added shearing to obtain multiple second labeled images.

[0022] Optionally, the step of performing online data augmentation processing on the first labeled image and the second labeled image to obtain an online data-enhanced labeled image specifically includes:

[0023] The images in the first labeled image and the second labeled image are subjected to random brightness enhancement processing, random color enhancement processing, random contrast enhancement processing, random sharpness enhancement processing, and random salt and pepper noise addition processing with preset probabilities.

[0024] The images and labels in the first and second labeled images are horizontally flipped with a preset probability.

[0025] Optionally, the improvement of the image semantic segmentation algorithm model specifically includes:

[0026] The backbone network MobileNet of the image semantic segmentation algorithm model is replaced with ResNet101 to enhance the model's learning ability.

[0027] The dilatation rate of each dilatation in the ASPP module of the image semantic segmentation algorithm model is increased to increase the receptive field of the neural network model, thereby making the network more capable of global feature extraction and paying more attention to global information.

[0028] An attention mechanism module is added to the image semantic segmentation algorithm model;

[0029] The loss function is redesigned, and the redesigned loss function includes: the IOU between the predicted region and the label region, the area relationship between the predicted region and the label region, the perimeter relationship between the predicted region and the label region, the distance between the center points of the predicted region and the label region, the difference in the number of regions between the predicted region and the label region, and the number of regions with excessively small areas in the prediction result.

[0030] To achieve the above objectives, according to another aspect of the present invention, a coal mine underground hazardous area identification device is provided, the device comprising:

[0031] The image acquisition unit is used to acquire underground coal mine images captured by cameras installed underground in the coal mine.

[0032] The hazardous area identification unit is used to input the underground coal mine image into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data.

[0033] To achieve the above objectives, according to another aspect of the present invention, a computer device is also provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described method for identifying hazardous areas in underground coal mines.

[0034] To achieve the above objectives, according to another aspect of the present invention, a computer-readable storage medium is also provided, on which a computer program / instructions are stored, which, when executed by a processor, implement the steps of the above-described method for identifying hazardous areas in underground coal mines.

[0035] To achieve the above objectives, according to another aspect of the present invention, a computer program product is also provided, comprising a computer program / instructions that, when executed by a processor, implement the steps of the above-described method for identifying hazardous areas in underground coal mines.

[0036] The beneficial effects of this invention are as follows:

[0037] This invention acquires underground coal mine images captured by cameras installed underground in coal mines, and then inputs these images into a preset hazardous area identification model to identify hazardous areas in the underground coal mine images. This achieves the beneficial effect of real-time and accurate identification of hazardous areas in underground coal mines, which helps to improve the safety of underground coal mine production. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0039] Figure 1 This is a flowchart of the method for identifying dangerous areas in underground coal mines according to an embodiment of the present invention;

[0040] Figure 2 This is a flowchart illustrating the danger zone intrusion warning process according to an embodiment of the present invention;

[0041] Figure 3 This is a flowchart of model training in the first embodiment of the present invention;

[0042] Figure 4 This is a flowchart of model training according to the second embodiment of the present invention;

[0043] Figure 5 This is a flowchart of the model reasoning process of this invention;

[0044] Figure 6 This is a flowchart of the model optimization process of this invention;

[0045] Figure 7 This is a schematic diagram of the offline data enhancement of the present invention;

[0046] Figure 8 This is a schematic diagram of the original DeeplabV3 network model structure of this invention;

[0047] Figure 9 This is a schematic diagram of the improved DeeplabV3 network model structure of this invention;

[0048] Figure 10 This is a structural block diagram of the underground hazardous area identification device in a coal mine according to an embodiment of the present invention;

[0049] Figure 11 This is a schematic diagram of a computer device according to an embodiment of the present invention. Detailed Implementation

[0050] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0051] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0052] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such processes, methods, products or devices.

[0053] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0054] The purpose of this invention is to provide a method for identifying dangerous areas in underground coal mines. This method can identify dangerous areas in underground coal mines in real time, dynamically, and accurately. Combined with a personnel detection model, it can issue an alarm for personnel entering dangerous areas, thereby improving the safety factor of coal mines, reducing the accident rate, and enhancing the level of intelligence in coal mines.

[0055] To achieve the above objectives, this solution uses semantic segmentation to identify dangerous areas, and combines this with a personnel detection model to determine if someone has entered a dangerous area. It mainly consists of two parts: model training and model inference.

[0056] The specific process of model training is as follows: Figure 4As shown. The detailed steps are as follows: 1) Collect video of the scene to be identified from the scene; 2) Divide the video into frames, use the SSIM algorithm to remove images with similarity greater than the threshold, and save images with different scenes; 3) Perform data preprocessing on the filtered images, including overexposure processing, fill light processing, and changing brightness, to make the images easier to identify; 4) Use the semi-automatic annotation tool EISeg to label the data; 5) Use perspective transformation and affine transformation to perform offline data augmentation on the images; 6) Load and read the images to prepare them for training in the neural network, and perform online data augmentation on the loaded images (including brightness enhancement, color enhancement, contrast enhancement, sharpness enhancement, adding salt and pepper noise, and horizontal flipping, etc.); 7) Improve the model, including modifying the ASPP module, introducing an attention mechanism, redesigning the loss function, and modifying the backbone network, etc.; 8) Train the dataset using the improved model; 9) Evaluate the trained model. If the model accuracy meets the standard, save the model for inference; otherwise, continue training the model.

[0057] The specific process of model inference is as follows: Figure 5 As shown. The detailed steps are as follows: 1) Obtain the status of the coal mining machine. If the coal mining machine is locked, no judgment is made, and the status of the coal mining machine is continued until the coal mining machine is unlocked. If the coal mining machine is unlocked, the stream is taken; 2) Perform data preprocessing on the obtained images to make the images easier to identify; 3) Use an image semantic segmentation algorithm to identify objects in the coal mine at the pixel level, identifying 7 types of objects such as side guards, conveyor belts, rollers, coal, troughs, tracks, and cable troughs; 4) Further process the output predicted values ​​to filter out some obviously erroneous pixel values ​​(too small, too scattered, etc.); 5) Use a convex hull finding algorithm to regularize the identification area; 6) Use target detection methods to identify underground personnel; 7) Determine the positional relationship between personnel and dangerous areas. If personnel are in dangerous areas, send an alarm and lock the coal mining machine. Otherwise, repeat step 1).

[0058] In practical applications, new materials will emerge, which are invaluable learning resources for neural networks. Therefore, during use, we continuously collect and organize poorly recognized images, input them into the dataset for model training. The flowchart is as follows: Figure 6 As shown, this model optimization method allows the model to adapt to an increasing number of scenarios as its accuracy improves.

[0059] Figure 1 This is a flowchart of the method for identifying hazardous areas in underground coal mines according to an embodiment of the present invention, as follows: Figure 1 As shown, in one embodiment of the present invention, the method for identifying dangerous areas in underground coal mines includes steps S101 and S102.

[0060] Step S101: Obtain underground coal mine images captured by cameras installed underground in the coal mine.

[0061] Step S102: Input the underground coal mine image into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data.

[0062] Figure 2 This is a flowchart of the dangerous area intrusion warning process according to an embodiment of the present invention, such as... Figure 2 As shown, in one embodiment of the present invention, the dangerous area intrusion warning process of the present invention includes steps S201 and S202.

[0063] Step S201: The underground personnel in the coal mine image are identified by a preset target detection algorithm model.

[0064] Step S202: If it is determined that the identified underground personnel are in the danger zone, an early warning message is generated to remind the underground personnel to stay away from the danger zone and the coal mining machine is locked.

[0065] In one embodiment of the present invention, step S101, which involves acquiring underground coal mine image data collected by a camera installed underground in the coal mine, specifically includes:

[0066] When the coal mining machine is unlocked, real-time video data collected by the camera is acquired, and then the underground coal mine image is extracted from the video data. The camera is activated to collect video data when the coal mining machine is unlocked.

[0067] In one embodiment of the present invention, the method for identifying hazardous areas in underground coal mines further includes:

[0068] Simultaneously with generating the early warning information, a coal mining machine lockout command is generated to lock the coal mining machine according to the coal mining machine lockout command.

[0069] This invention improves the image semantic segmentation algorithm model and processes the data. Specifically:

[0070] Model improvements: 1) Modify the ASPP module in the network model and change the hole rate to a value suitable for the task; 2) Introduce an attention mechanism into the network; 3) Modify the loss function and add IOU, area, perimeter, center distance, and number of regions between the predicted and labeled regions; 4) Change the backbone network to ResNet101.

[0071] Data processing: Offline data enhancement such as perspective transformation and affine transformation is performed on tagged data; online data enhancement such as brightness enhancement, color enhancement, contrast enhancement, sharpness enhancement, addition of salt and pepper noise, and horizontal flipping are performed.

[0072] Figure 3 This is a flowchart of model training in the first embodiment of the present invention, as follows: Figure 3 As shown, the model training process of the present invention includes steps S301 to S305.

[0073] Step S301: Label the dangerous areas on the underground coal mine images used for model training to obtain the first labeled image.

[0074] This invention can use the semi-automatic annotation tool EISeg to label dangerous areas on images.

[0075] Step S302: Perform offline data augmentation processing on each of the first labeled images to obtain the second labeled image.

[0076] Step S303: Perform online data augmentation on the first labeled image and the second labeled image to obtain the online data-augmented labeled image.

[0077] Step S304: Improve the image semantic segmentation algorithm model to obtain the improved image semantic segmentation algorithm model.

[0078] Step S305: Train the improved image semantic segmentation algorithm model based on the labeled image after online data augmentation, and determine the trained model as the danger area identification model when the recognition accuracy of the trained model meets the preset standard.

[0079] This invention uses semantic segmentation technology to identify hazardous areas in underground coal mine working faces, enabling real-time, dynamic identification of hazardous areas at the pixel level. This invention innovates upon existing semantic segmentation algorithm models and data processing methods, improving the performance of semantic segmentation networks in underground scenarios.

[0080] This invention employs a combination of offline and online data enhancement. Offline data enhancement includes perspective transformation and affine transformation. Online data enhancement includes brightness enhancement, chroma enhancement, contrast enhancement, sharpness enhancement, addition of salt-and-pepper noise, and horizontal flipping.

[0081] In one embodiment of the present invention, the step S302 of performing offline data enhancement processing on each of the first labeled images to obtain a second labeled image specifically includes:

[0082] For each of the first labeled images, perform perspective transformations by rotating around the x-axis, perspective transformations by rotating around the image center along two axes, affine transformations, and affine transformations with added shearing to obtain multiple second labeled images.

[0083] In one embodiment of the present invention, during offline data augmentation, the same operation is performed on both the image and the label to ensure that the label is accurate after augmentation. Specifically, the data is transformed using perspective transformation (rotation around the x-axis), perspective transformation (rotation around the image center along two axes), affine transformation, and affine transformation with shearing, generating new data. Affine transformation transforms the data into another vector space by performing a linear transformation (multiplication by a matrix) and a translation (addition of a vector) in the vector space to create a new dataset. Perspective transformation, also called projection transformation, converts a two-dimensional image to a three-dimensional space and then projects it to generate a new image. Through these operations, the dataset is expanded to five times its original size. A schematic diagram is shown below. Figure 7 As shown.

[0084] In one embodiment of the present invention, step S303, which involves performing online data augmentation on the first labeled image and the second labeled image to obtain an online data-enhanced labeled image, specifically includes:

[0085] The images in the first labeled image and the second labeled image are subjected to random brightness enhancement processing, random color enhancement processing, random contrast enhancement processing, random sharpness enhancement processing, and random salt and pepper noise addition processing with preset probabilities.

[0086] The images and labels in the first and second labeled images are horizontally flipped with a preset probability.

[0087] In one embodiment of the present invention, online data augmentation sequentially performs the following operations on the labeled image: random brightness enhancement, random chroma enhancement, random contrast enhancement, random sharpness enhancement, addition of random salt-and-pepper noise, and horizontal flipping. The operations of random brightness enhancement, random chroma enhancement, random contrast enhancement, random sharpness enhancement, and addition of random salt-and-pepper noise are applied only to the image and not to the labels. Horizontal flipping is performed on both the image and the labels. In online data augmentation, whether or not the above operations are performed is random, with a preset probability, for example, 0.5, and the parameters fluctuate within a certain range. The enhancement factor for brightness enhancement is a normal distribution value between 0.6 and 1.6; the enhancement factor for chroma enhancement is a normal distribution value between 0.4 and 2.6; the enhancement factor for contrast enhancement is a normal distribution value between 0.6 and 1.6; the enhancement factor for sharpness enhancement is a normal distribution value between 0.4 and 4; adding random salt-and-pepper noise simulates image noise by randomly adding some white dots to the image; then, the data is horizontally flipped with a preset probability, for example, 0.5. Online data augmentation ensures that each input image to the network is unique, mitigating the overfitting problem.

[0088] In one embodiment of the present invention, the improvement of the image semantic segmentation algorithm model in step S304 specifically includes:

[0089] The backbone network MobileNet of the image semantic segmentation algorithm model is replaced with ResNet101 to enhance the model's learning ability.

[0090] The dilatation ratios of each dilatation ratio in the ASPP (Atrous Spatial Pyramid Pooling) module of the image semantic segmentation algorithm model are increased to increase the receptive field of the neural network model, thereby making the network more capable of global feature extraction and paying more attention to global information.

[0091] An attention mechanism module is added to the image semantic segmentation algorithm model;

[0092] The loss function is redesigned, and the redesigned loss function includes: the IOU between the predicted region and the label region, the area relationship between the predicted region and the label region, the perimeter relationship between the predicted region and the label region, the distance between the center points of the predicted region and the label region, the difference in the number of regions between the predicted region and the label region, and the number of regions with excessively small areas in the prediction result.

[0093] In a specific embodiment of the present invention, the image semantic segmentation algorithm model specifically adopts the DeeplabV3 model. The present invention improves the original DeeplabV3 network model to obtain an improved image semantic segmentation algorithm model, and then trains the model based on the improved image semantic segmentation algorithm model.

[0094] The original network model structure of DeeplabV3 is as follows: Figure 8 As shown, the structure of the improved image semantic segmentation algorithm model of this invention is as follows: Figure 9 As shown.

[0095] like Figure 8 and Figure 9 As shown, this invention modifies the ASPP module in the original DeeplabV3 network model, changing the dilatation ratio combination from 6, 12, 18 to 7, 14, 21. Increasing the dilatation ratio increases the receptive field of the network, allowing for more effective extraction of global features. This invention replaces the MobileNet backbone in the original DeeplabV3 network model with ResNet101, resulting in a stronger learning ability and the extraction of more effective features. Furthermore, this invention introduces the CBAM attention mechanism module into the original DeeplabV3 network model.

[0096] To further improve the model recognition performance, the present invention also redesigned the loss function. The loss function designed in this invention takes into account factors such as the IOU between the predicted region and the label region, the area relationship, the perimeter relationship, the distance between the center points, the difference in the number of regions, and the number of regions with too small area in the prediction result.

[0097] The specific steps are as follows: 1) Calculate the outer contours of the labels and predicted images using an algorithm for finding outer contours; 2) Filter out contours whose area is less than 10 pixels; 3) Calculate the center point of each contour. The loss function used in this invention is:

[0098] Loss = L Iou +L Area +L Perimeter +L Distance +L Num +L area_under100 (1)

[0099] In equation (1), L Iou The intersection-union ratio (IU) of the predicted region and the labeled region;

[0100] In equation (1),

[0101] Among them, Area real Area is the area of ​​the label region. pred The area of ​​the predicted result region;

[0102] In equation (1),

[0103] Among them, Perimeter realPerimeter is the perimeter of the label area. pred The perimeter of the predicted result area;

[0104] In equation (1),

[0105] Among them, (x real y real (x) represents the coordinates of the center point of the label area. pre y pre () represents the coordinates of the center point of the predicted region;

[0106] In equation (1),

[0107] Where, num real num represents the number of regions in the label. pre This represents the number of regions in the prediction results;

[0108] In equation (1),

[0109] The method proposed in this invention can accurately identify dangerous areas in real time when the camera moves or the scene changes, effectively ensuring the safety of workers at the underground working face and improving the level of automation at the working face.

[0110] It should be noted that all solutions in this invention are based on research and analysis of specific hazardous areas in underground coal mines, with a focus on the proposed region recognition algorithm based on an improved image semantic segmentation algorithm. Regarding the algorithmic solutions (a series of improvements to the image semantic segmentation algorithm), this improved model is only a specific application in this scenario, but the scope of protection of this invention is not limited thereto; applications of the improved model in other scenarios should also be covered within the scope of protection of this invention. Similarly, improvements to the semantic segmentation model, such as improvements to the loss function, data augmentation methods, and modifications to the ASPP module, are not limited to the deeplabV3 used in this invention; these improvements are also applicable to other semantic recognition models and should also be covered within the scope of protection of this invention.

[0111] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0112] Based on the same inventive concept, embodiments of the present invention also provide a coal mine underground hazardous area identification device, which can be used to implement the coal mine underground hazardous area identification method described in the above embodiments, as described in the following embodiments. Since the principle of the coal mine underground hazardous area identification device in solving the problem is similar to that of the coal mine underground hazardous area identification method, embodiments of the coal mine underground hazardous area identification device can refer to embodiments of the coal mine underground hazardous area identification method, and repeated details will not be elaborated further. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0113] Figure 10 This is a structural block diagram of the coal mine underground hazardous area identification device according to an embodiment of the present invention, as shown below. Figure 10 As shown, in one embodiment of the present invention, the coal mine underground hazardous area identification device of the present invention includes:

[0114] Image acquisition unit 1 is used to acquire underground coal mine images captured by a camera installed underground in the coal mine;

[0115] The hazardous area identification unit 2 is used to input the underground coal mine image into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data.

[0116] In one embodiment of the present invention, the coal mine underground hazardous area identification device of the present invention further includes:

[0117] The underground personnel identification unit is used to identify underground personnel in the coal mine underground image by using a preset target detection algorithm model;

[0118] The danger warning unit is used to generate a warning message if it is determined that an underground person is in the danger zone, so as to remind the underground person to stay away from the danger zone and lock the coal mining machine.

[0119] In one embodiment of the present invention, the present invention generates early warning information to remind underground personnel to stay away from the dangerous area, and at the same time locks the coal mining machine.

[0120] In one embodiment of the present invention, the image acquisition unit 1 is specifically used for:

[0121] When the coal mining machine is unlocked, real-time video data collected by the camera is acquired, and then the underground coal mine image is extracted from the video data. The camera is activated to collect video data when the coal mining machine is unlocked.

[0122] In one embodiment of the present invention, the coal mine underground hazardous area identification device of the present invention further includes:

[0123] The locking instruction sending unit is used to generate a coal mining machine locking instruction at the same time as generating the early warning information, so as to lock the coal mining machine according to the coal mining machine locking instruction.

[0124] In one embodiment of the present invention, the coal mine underground hazardous area identification device of the present invention further includes:

[0125] The labeling unit is used to label dangerous areas on underground coal mine images used for model training, resulting in the first labeled image;

[0126] An offline data augmentation processing unit is used to perform offline data augmentation processing on each of the first labeled images to obtain a second labeled image;

[0127] An online data augmentation processing unit is used to perform online data augmentation processing on the first labeled image and the second labeled image to obtain a labeled image after online data augmentation.

[0128] The model improvement unit is used to improve the image semantic segmentation algorithm model to obtain the improved image semantic segmentation algorithm model;

[0129] The training unit is used to train the improved image semantic segmentation algorithm model based on the labeled image after online data augmentation, and to determine the trained model as the danger area identification model when the recognition accuracy of the trained model meets the preset standard.

[0130] In one embodiment of the present invention, the offline data enhancement processing unit is specifically used for:

[0131] For each of the first labeled images, perform perspective transformations by rotating around the x-axis, perspective transformations by rotating around the image center along two axes, affine transformations, and affine transformations with added shearing to obtain multiple second labeled images.

[0132] In one embodiment of the present invention, the online data enhancement processing unit is specifically used for:

[0133] The images in the first and second labeled images are subjected to random brightness enhancement, random color enhancement, random contrast enhancement, random sharpness enhancement, and random salt and pepper noise addition with preset probabilities; the images and labels in the first and second labeled images are horizontally flipped with preset probabilities.

[0134] In one embodiment of the present invention, the model improvement unit is specifically used for:

[0135] The backbone network MobileNet of the image semantic segmentation algorithm model is replaced with ResNet101 to enhance the model's learning ability.

[0136] The dilatation rate of each dilatation in the ASPP module of the image semantic segmentation algorithm model is increased to increase the receptive field of the neural network model, thereby making the network more capable of global feature extraction and paying more attention to global information.

[0137] An attention mechanism module is added to the image semantic segmentation algorithm model;

[0138] The loss function is redesigned, and the redesigned loss function includes: the IOU between the predicted region and the label region, the area relationship between the predicted region and the label region, the perimeter relationship between the predicted region and the label region, the distance between the center points of the predicted region and the label region, the difference in the number of regions between the predicted region and the label region, and the number of regions with excessively small areas in the prediction result.

[0139] To achieve the above objectives, according to another aspect of this application, a computer device is also provided. For example... Figure 11 As shown, the computer device includes a memory, a processor, a communication interface, and a communication bus. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the steps in the method of the above embodiments.

[0140] The processor can be a central processing unit (CPU). The processor can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or combinations of the above types of chips.

[0141] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and units, such as the program units corresponding to the above-described method embodiments of the present invention. The processor executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions, and modules stored in the memory, thereby implementing the methods described in the above-described method embodiments.

[0142] The memory may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created by the processor, etc. Furthermore, the memory may include high-speed random access memory and non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory may optionally include memory remotely located relative to the processor, which can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0143] The one or more units are stored in the memory and, when executed by the processor, perform the methods described in the above embodiments.

[0144] The specific details of the aforementioned computer equipment can be understood by referring to the relevant descriptions and effects in the above embodiments, and will not be repeated here.

[0145] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is also provided, which stores a computer program that, when executed in a computer processor, implements the steps in the above-described method for identifying hazardous areas in underground coal mines. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0146] To achieve the above objectives, according to another aspect of this application, a computer program product is also provided, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method for identifying hazardous areas in underground coal mines.

[0147] Obviously, those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device, or fabricating them separately as individual integrated circuit modules, or fabricating multiple modules or steps as a single integrated circuit module. Thus, the present invention is not limited to any particular hardware and software combination.

[0148] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying hazardous areas in underground coal mines, characterized in that, include: Acquire underground coal mine images captured by cameras installed underground in the coal mine; The underground coal mine image is input into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data. The method further includes: Dangerous areas are labeled on underground coal mine images used for model training to obtain the first labeled image; Offline data augmentation processing is performed on each of the first labeled images to obtain the second labeled images; Online data augmentation processing is performed on the first labeled image and the second labeled image to obtain the online data-augmented labeled image; The image semantic segmentation algorithm model is improved to obtain the improved image semantic segmentation algorithm model; The improved image semantic segmentation algorithm model is trained based on the labeled image after online data augmentation, and the trained model is determined as the danger area identification model when the recognition accuracy of the trained model meets the preset standard. The improvement to the image semantic segmentation algorithm model specifically includes: The backbone network MobileNet of the image semantic segmentation algorithm model is replaced with ResNet101 to enhance the model's learning ability. The dilatation rate of each dilatation in the ASPP module of the image semantic segmentation algorithm model is increased to increase the receptive field of the neural network model, thereby making the network more capable of global feature extraction and paying more attention to global information. An attention mechanism module is added to the image semantic segmentation algorithm model; The loss function is redesigned, and the redesigned loss function includes: the IOU between the predicted region and the label region, the area relationship between the predicted region and the label region, the perimeter relationship between the predicted region and the label region, the distance between the center points of the predicted region and the label region, the difference in the number of regions between the predicted region and the label region, and the number of regions with excessively small areas in the prediction result.

2. The method for identifying hazardous areas in underground coal mines according to claim 1, characterized in that, Also includes: The underground personnel in the coal mine images are identified using a preset target detection algorithm model. If it is determined that an identified underground worker is located within the danger zone, an early warning message is generated to remind the worker to stay away from the danger zone.

3. The method for identifying hazardous areas in underground coal mines according to claim 2, characterized in that, The acquisition of underground coal mine image data collected by cameras installed underground specifically includes: When the coal mining machine is unlocked, real-time video data collected by the camera is acquired, and then the underground coal mine image is extracted from the video data. The camera is activated to collect video data when the coal mining machine is unlocked. The method for identifying hazardous areas in underground coal mines also includes: Simultaneously with generating the early warning information, a coal mining machine lockout command is generated to lock the coal mining machine according to the coal mining machine lockout command.

4. The method for identifying hazardous areas in underground coal mines according to claim 1, characterized in that, Offline data augmentation processing is performed on each of the first labeled images to obtain the second labeled image, specifically including: For each of the first labeled images, perform perspective transformations by rotating around the x-axis, perspective transformations by rotating around the image center along two axes, affine transformations, and affine transformations with added shearing to obtain multiple second labeled images.

5. The method for identifying hazardous areas in underground coal mines according to claim 1, characterized in that, The online data augmentation process for the first labeled image and the second labeled image to obtain the online data-enhanced labeled image specifically includes: The images in the first labeled image and the second labeled image are subjected to random brightness enhancement processing, random color enhancement processing, random contrast enhancement processing, random sharpness enhancement processing, and random salt and pepper noise addition processing with preset probabilities. The images and labels in the first and second labeled images are horizontally flipped with a preset probability.

6. A device for identifying hazardous areas in underground coal mines, characterized in that, include: The image acquisition unit is used to acquire underground coal mine images captured by cameras installed underground in the coal mine. The hazardous area identification unit is used to input the underground coal mine image into a preset hazardous area identification model to identify hazardous areas in the underground coal mine image. The hazardous area identification model is obtained by training an image semantic segmentation algorithm model using training data. The device further includes: The labeling unit is used to label dangerous areas on underground coal mine images used for model training, resulting in the first labeled image; An offline data augmentation processing unit is used to perform offline data augmentation processing on each of the first labeled images to obtain a second labeled image; An online data augmentation processing unit is used to perform online data augmentation processing on the first labeled image and the second labeled image to obtain a labeled image after online data augmentation. The model improvement unit is used to improve the image semantic segmentation algorithm model to obtain the improved image semantic segmentation algorithm model; The training unit is used to train the improved image semantic segmentation algorithm model based on the labeled image after online data augmentation, and to determine the trained model as the danger area identification model when the recognition accuracy of the trained model meets the preset standard. The model improvement unit is specifically used for: The backbone network MobileNet of the image semantic segmentation algorithm model is replaced with ResNet101 to enhance the model's learning ability. The dilatation rate of each dilatation in the ASPP module of the image semantic segmentation algorithm model is increased to increase the receptive field of the neural network model, thereby making the network more capable of global feature extraction and paying more attention to global information. An attention mechanism module is added to the image semantic segmentation algorithm model; The loss function is redesigned, and the redesigned loss function includes: the IOU between the predicted region and the label region, the area relationship between the predicted region and the label region, the perimeter relationship between the predicted region and the label region, the distance between the center points of the predicted region and the label region, the difference in the number of regions between the predicted region and the label region, and the number of regions with excessively small areas in the prediction result.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images