Coal mine smoke and fire detection method based on improved YOLOv8 and related device

Abstract

The invention relates to the technical field of data processing and coal mine smoke and fire detection, and discloses an improved YOLOv8-based coal mine smoke and fire detection method and related device, and the method comprises the steps: obtaining a smoke fire image of a coal mine and a to-be-detected coal mine image; preprocessing the smoke fire image to obtain an initial smoke and fire image; based on the improved YOLOv8 target detection model, constructing an initial coal mine smoke and fire detection model; training the initial coal mine smoke and fire detection model by using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model; and performing smoke and fire detection on the to-be-detected coal mine image by adopting the target coal mine smoke and fire detection model to obtain a target detection result. By implementing the method provided by the invention, the accuracy of coal mine smoke and fire detection can be improved, and the purpose of accurate and efficient coal mine smoke and fire detection can be achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and coal mine smoke and fire detection, and in particular to a coal mine smoke and fire detection method based on improved YOLOv8 and a related device. Background Art

[0002] Coal is the main energy source in my country. It plays the role of "ballast stone" and "stabilizer" in energy supply, and provides inexhaustible power for the vigorous development of the national economy. Coal mine fire detection is an important part of safe production in the coal mining industry, and requires reliable technology as a support and guarantee. Thanks to the rapid development of convolutional neural networks combined with deep learning, target detection algorithms based on deep learning, especially one-stage target detection algorithms represented by the YOLO series, have been widely used in the field of fire detection. However, since the smoke and flames in the early stage of a fire are often presented by small targets, the existing target detection algorithms have low detection accuracy and slow speed for small target features in the detection of smoke and flames, and may even miss detections and misdetections, which cannot achieve the purpose of accurate and efficient coal mine smoke and fire detection. Summary of the invention

[0003] The present invention provides a coal mine smoke and fire detection method and related devices based on improved YOLOv8, which can improve the accuracy of coal mine smoke and fire detection to solve the technical problem that accurate and efficient coal mine smoke and fire detection cannot be achieved.

[0004] In the first aspect, a coal mine smoke and fire detection method based on improved YOLOv8 is provided, including:

[0005] Acquire coal mine smoke and fire images and coal mine images to be detected;

[0006] Preprocessing the smoke and fire image to obtain an initial smoke and fire image;

[0007] Based on the improved YOLOv8 target detection model, an initial coal mine smoke and fire detection model was constructed;

[0008] Using the coal mine smoke and fire image dataset to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model;

[0009] The target coal mine smoke and fire detection model is used to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

[0010] In the second aspect, a coal mine smoke and fire detection device based on improved YOLOv8 is provided, comprising:

[0011] An acquisition module, used for acquiring smoke and fire images of coal mines and images of coal mines to be detected;

[0012] A first processing module, used for preprocessing the smoke and fire image to obtain an initial smoke and fire image;

[0013] The second processing module is used to build an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model;

[0014] A third processing module is used to train the initial coal mine smoke and fire detection model using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model;

[0015] The detection module is used to use the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

[0016] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above-mentioned coal mine smoke and fire detection method based on improved YOLOv8 are implemented.

[0017] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the above-mentioned coal mine smoke and fire detection method based on improved YOLOv8 are implemented.

[0018] In the scheme implemented by the above-mentioned coal mine smoke and fire detection method and related devices based on improved YOLOv8, by obtaining the smoke and fire image of the coal mine and the coal mine image to be detected, the smoke and fire image is preprocessed to obtain the initial smoke and fire image, and based on the improved YOLOv8 target detection model, the initial coal mine smoke and fire detection model is constructed, so that the coal mine smoke and fire image data set can be used to train the initial coal mine smoke and fire detection model to obtain the target coal mine smoke and fire detection model, and then the target coal mine smoke and fire detection model can be used to perform smoke and fire detection on the coal mine image to be detected to obtain more accurate target detection results, which can effectively improve the accuracy of coal mine smoke and fire detection, and is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0020] Figure 1 It is a flow chart of a coal mine smoke and fire detection method based on improved YOLOv8 in one embodiment of the present invention;

[0021] Figure 2A It is a structural schematic diagram of a target coal mine smoke and fire detection model of a coal mine smoke and fire detection method based on improved YOLOv8 in one embodiment of the present invention;

[0022] Figure 2B It is a structural schematic diagram of a C2f_DCNv4 module of a coal mine smoke and fire detection method based on improved YOLOv8 in one embodiment of the present invention;

[0023] Figure 2C It is a structural schematic diagram of an EMA module of a coal mine smoke and fire detection method based on improved YOLOv8 in one embodiment of the present invention;

[0024] Figure 3 It is a structural schematic diagram of a coal mine smoke and fire detection device based on improved YOLOv8 in one embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0026] Figure 5 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] See also Figure 1 As shown, Figure 1 A flow chart of a coal mine smoke and fire detection method based on improved YOLOv8 provided in an embodiment of the present invention includes the following steps:

[0029] S10: Acquire the smoke and fire image of the coal mine and the image of the coal mine to be detected.

[0030] Among them, the smoke and fire image of the coal mine may be an image containing target detection elements, and the target detection elements may be but are not limited to white smoke, gray smoke, flames, sparks, etc. It is understandable that the smoke and fire image of the coal mine can be an important basis for monitoring and early warning of the safety status of the coal mine. The smoke and fire image of the coal mine can be obtained by using network picture resources, that is, related coal mine smoke and fire images on the Internet, or by simulating the coal mine tunnel environment for shooting. The coal mine image to be detected can be an image that needs to be subjected to coal mine smoke and fire detection.

[0031] S20: Preprocessing the smoke and fire image to obtain an initial smoke and fire image.

[0032] The smoke and fire images in the coal mine environment can be preprocessed by random cropping, rotation, color conversion, background mixing and other operations to achieve data enhancement. The smoke and fire images can be annotated to generate a coal mine smoke and fire image dataset, that is, the initial smoke and fire images mentioned above, and the initial smoke and fire images can be divided into a training set, a validation set, and a test set according to a certain ratio.

[0033] Exemplarily, each coal mine scene smoke and fire image can be annotated by annotation software (such as the image annotation open source tool LabelImg). Among them, the label format can be YOLO, which can be divided into two types of labels: flame (Fire) and smoke (Smoke). The annotation information of each image can include the category number of the label and the center coordinates and relative width and height data of the annotation box. After obtaining the annotated coal mine smoke and fire image dataset, the dataset can be divided according to the ratio of 7:2:1 to obtain a training set, a verification set, and a test set respectively. This application does not limit this.

[0034] S30: Build an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model.

[0035] In a possible implementation, constructing an initial coal mine smoke and fire detection model based on an improved YOLOv8 target detection model can include the following steps: introducing a deformable convolution operator into the backbone network of the improved YOLOv8 target detection model, reconstructing the backbone network, and obtaining a reconstructed backbone network; introducing an efficient multi-scale attention module into the neck network of the improved YOLOv8 target detection model, reconstructing the neck network, and obtaining a reconstructed neck network; replacing the original loss function of the improved YOLOv8 target detection model to obtain a target loss function; and constructing an initial coal mine smoke and fire detection model according to the reconstructed backbone network, the reconstructed neck network, and the target loss function.

[0036] Specifically, when reconstructing the backbone network, this application introduces the fourth version of the deformable convolution (DeformableConvolutional Network v4, DCNv4) to construct a bottleneck module fused with deformable convolution (FasterImplementation of CSP Bottleneck with 2Convolutions_Deformable ConvolutionalNetwork v4, C2f_DCNV4), thereby obtaining a reconstructed backbone network. When reconstructing the neck network, this application introduces Efficient Multi-scale Attention (EMA) to construct an EMA module, thereby obtaining a reconstructed neck network.

[0037] Furthermore, the present application uses the Minimum Point Distance Intersection over Union (MPDIoU) loss function based on the minimum point distance to replace the original loss function, that is, MPDIoU is used to replace the Complete IoU (CIoU) loss function. Optionally, the following formula can be used to replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function:

[0038]

[0039] Among them, MPDIoU represents the target loss function; IoU represents the intersection over union ratio between the real box and the predicted box; Represents the square of the Euclidean distance between the upper left corner of the true box and the predicted box; represents the square of the Euclidean distance between the lower right corner of the real box and the predicted box; h represents the height of the input image; w represents the width of the input image; B gt represents the real box; B pre Represents the prediction box; The horizontal coordinate of the upper left corner of the prediction box; The horizontal coordinate of the upper left corner of the real box; The ordinate represents the coordinate of the upper left corner of the prediction box; The ordinate represents the coordinate of the upper left corner of the real box; The horizontal coordinate of the lower right corner of the prediction box; The horizontal coordinate of the lower right corner of the real box; The ordinate represents the coordinate of the lower right corner of the prediction box; The ordinate representing the lower right corner coordinate of the ground-truth box.

[0040] The initial coal mine smoke and fire detection model can be an untrained coal mine smoke and fire detection model constructed according to the reconstructed backbone network, the reconstructed neck network and the target loss function. It can be understood that compared with the improved YOLOv8 target detection model, this model can be more suitable for smoke and fire detection in coal mine scenarios, and can achieve the purpose of more accurate and efficient coal mine smoke and fire detection.

[0041] S40: Using the coal mine smoke and fire image dataset to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model.

[0042] The training set and validation set images obtained by dividing the aforementioned initial fireworks images in proportion can be input into the initial coal mine fireworks detection model for training to obtain the trained target coal mine fireworks detection model. The training environment can use Windows 10 operating system, NVIDIA Geforce GTX 1650 (4G video memory) graphics card, 16GB memory, Python 3.8 programming language, Pytorch 2.0.0 deep learning framework, CUDA 12.1 computing architecture; the main training parameters can be 200 training rounds, batch size -1, and initial learning rate 0.01.

[0043] Furthermore, the test set images in the aforementioned initial smoke and fire images can be input into the initial coal mine smoke and fire detection model for detection to output the coal mine environment smoke and fire detection results for the test set images. The detection results may include category names (such as smoke or fire) and confidence levels, etc., which are not limited in this application.

[0044] S50: using the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected, and obtaining a target detection result.

[0045] After obtaining the trained target coal mine smoke and fire detection model, the target coal mine smoke and fire detection model can be used to perform coal mine smoke and fire detection on the coal mine image to be detected, thereby obtaining a more accurate coal mine smoke and fire detection result. Specifically, the steps of using the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected and obtaining the target detection result can be referred to the detailed description in the following embodiment, which will not be repeated in this application.

[0046] Optionally, before using the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected, the coal mine image to be detected can be subjected to relevant preprocessing, thereby processing the coal mine image to be detected into an image that is easier for the model to process, thereby improving the detection accuracy and efficiency of the model. This application does not impose any restrictions on this.

[0047] In this embodiment, by obtaining a smoke and fire image of a coal mine and an image of a coal mine to be detected, the smoke and fire image is preprocessed to obtain an initial smoke and fire image, and based on the improved YOLOv8 target detection model, an initial coal mine smoke and fire detection model is constructed, so that the coal mine smoke and fire image data set can be used to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model, and then the target coal mine smoke and fire detection model can be used to perform smoke and fire detection on the coal mine image to be detected to obtain a more accurate target detection result, which can effectively improve the accuracy of coal mine smoke and fire detection, and is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection.

[0048] In a possible implementation, using the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result refers to the process of how to apply the trained target coal mine smoke and fire detection model. Optionally, using the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result may include the following steps:

[0049] A1: The coal mine image to be detected is subjected to multi-scale feature extraction through the bottleneck module of the target coal mine smoke and fire detection model that integrates deformable convolution to obtain a multi-scale coal mine feature image;

[0050] A2: The multi-scale coal mine feature image is passed through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to perform attention calculation to obtain a multi-scale attention coal mine feature image;

[0051] A3: performing feature fusion on the multi-scale attention coal mine feature image through the target coal mine smoke and fire detection model to obtain a fused coal mine feature image;

[0052] A4: Determine the target detection result according to the fused coal mine feature image through the target coal mine smoke and fire detection model.

[0053] See also Figure 2A , Figure 2A The structural diagram of a target coal mine smoke and fire detection model is shown as an example. The target coal mine smoke and fire detection model mainly consists of three parts: backbone network (Backbone), neck network (Neck) and detection head (Head).

[0054] After receiving the input image (i.e., the coal mine image to be detected), the target coal mine smoke and fire detection model can pass through the C2f_DCNv4 module in the backbone network (i.e., the bottleneck module fused with deformable convolution) to perform multi-scale feature extraction on the input image through a series of convolution processing and pooling, thereby gradually reducing the size of the feature map and increasing the number of channels at the same time to achieve a multi-scale feature extraction process, and then obtain a multi-scale coal mine feature image.

[0055] Furthermore, the extracted multi-scale coal mine feature image can be used for attention calculation through an efficient multi-scale attention module with an EMA attention mechanism added in the neck network, such as through operations such as feature pyramid, feature fusion, upsampling and downsampling, to efficiently extract and fuse multi-scale features, thereby obtaining a multi-scale attention coal mine feature image. Figure 2A As shown, the multi-scale attention coal mine feature image can be three feature maps at different levels. Finally, the detection head can analyze and process the feature maps at different levels to obtain the output result of the target coal mine smoke and fire detection model, that is, the target detection result for smoke and fire detection in the coal mine scene.

[0056] In a possible implementation, the step of extracting multi-scale features from the coal mine image to be detected through a bottleneck module of a fused deformable convolution of the target coal mine smoke and fire detection model to obtain a multi-scale coal mine feature image may include the following steps:

[0057] A11: performing grouping processing on the coal mine images to be detected through the bottleneck module of the target coal mine smoke and fire detection model fused with deformable convolution to obtain an initial coal mine image set;

[0058] A12: performing a deformable convolution operation on each of the initial coal mine images in the initial coal mine image set through a bottleneck module of the target coal mine smoke and fire detection model that is fused with a deformable convolution, to obtain an offset set and a weight set;

[0059] A13: Through the bottleneck module of the fused deformable convolution of the target coal mine smoke and fire detection model, each initial coal mine image in the initial coal mine image set is spliced ​​according to the corresponding offset and weight according to the offset set and the weight set to obtain a multi-scale coal mine feature image.

[0060] like Figure 2B As shown, Figure 2B The schematic diagram of the structure of a C2f_DCNv4 module is shown as an example. Figure 2BAs shown in the figure, the C2f_DCNv4 module can first group the input fireworks image feature map (i.e., the coal mine image to be detected) by channel, so as to further use convolution for each group of feature images to obtain the offset and weight of each position, and then calculate the convolution result according to the offset and weight. Finally, the convolution results of all groups are spliced ​​to obtain the output of the C2f_DCNv4 module, i.e., the multi-scale coal mine feature image.

[0061] It should be noted that compared with the previous generation deformable convolution operator (i.e. DCNv3), DCNv4 converges significantly faster and processes much faster, and the balance between model performance and efficiency can be comprehensively considered. The present invention replaces the ordinary convolution (Conv) in the bottleneck (Bottleneck) of the C2f module with a deformable convolution, and names it as C2f_DCNv4 as a whole, so as to update the second and third C2f modules in the backbone network to C2f_DCNv4 modules. By adaptively extracting features through deformable convolution, the sampling position can be made more consistent with the shape and scale of the target itself, thereby significantly improving the feature extraction capability of small targets.

[0062] In a possible implementation, the step of performing attention calculation on the multi-scale coal mine feature image through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain the multi-scale attention coal mine feature image may include the following steps:

[0063] A21: dividing the multi-scale coal mine feature image through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first feature map and a second feature map;

[0064] A22: encoding the first feature map along the spatial direction through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first encoded feature map;

[0065] A23: performing adaptive feature variable selection on the first encoding feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first adaptive feature map;

[0066] A24: performing a convolution operation on the second feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second multi-scale feature map;

[0067] A25: performing two-dimensional global pooling processing on the first adaptive feature map to obtain a first global feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model, and performing two-dimensional global pooling processing on the second multi-scale feature map to obtain a second global feature map;

[0068] A26: performing a matrix multiplication operation on the first global feature map and the second multi-scale feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first cross-space feature map;

[0069] A27: performing a matrix multiplication operation on the second global feature map and the first adaptive feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second cross-space feature map;

[0070] A28: The first cross-spatial feature map and the second cross-spatial feature map are aggregated through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a multi-scale attention coal mine feature image.

[0071] like Figure 2C As shown, Figure 2C The structural diagram of an EMA module is shown as an example. Figure 2C As shown in Figure 2, the EMA module can be mainly composed of two 1×1 branches, one 3×3 branch and a cross-space learning part. The EMA module can divide the input image (i.e., the multi-scale coal mine feature image) into multiple ( Figure 2C In the example of G, the sub-features are processed, that is, the first feature map and the second feature map are obtained by division. Two global average poolings can be used in the two 1×1 branches, such as encoding the channels along the X spatial direction and along the Y spatial direction, respectively, to obtain the first encoded feature map. Then the two encoded features are connected, sharing the 1×1 convolution, and the output can be decomposed into two vectors. The two-dimensional binomial distribution after linear convolution is fitted using the nonlinear activation function (Sigmoid Nonlinear Activation Function), and then after reweighted (Re-weight) adaptive feature variable selection, it is used as the output of the 1×1 branch, that is, the first adaptive feature map. In the 3×3 branch, a single 3×3 convolution can be used to extract multi-scale features as the output of the 3×3 branch, that is, the second multi-scale feature map.

[0072] The cross-space learning part can use two-dimensional global average pooling to encode the global spatial information of the outputs of the 1×1 branch and the 3×3 branch respectively to obtain the first global feature map and the second global feature map. The first global feature map and the second global feature map can be activated by the normalized exponential function (Softmax) and further matrix multiplication is performed, such as performing matrix multiplication on the normalized feature map of the 1×1 branch and the second multi-scale feature map of the 3×3 branch, and performing matrix multiplication on the normalized feature map of the 3×3 branch and the first adaptive feature map of the 1×1 branch to obtain the cross-space feature map, i.e., the first cross-space feature map and the second cross-space feature map. Furthermore, the two results (the first cross-space feature map and the second cross-space feature map) can be aggregated, and then activated by the Sigmoid function, and the Re-weight adaptive feature variable selection is performed to obtain the spatial attention map, i.e., the multi-scale attention coal mine feature image, which is conducive to highlighting the global context of the pixel.

[0073] It can be understood that the present invention adds an EMA module after the C2f module corresponding to each downsampling stage in the neck network Neck, so as to maximize the retention of feature information of small targets through channel reshaping and dimensional grouping, thereby effectively improving the detection performance of the model and facilitating the realization of accurate and efficient coal mine smoke and fire detection.

[0074] In this embodiment, the coal mine image to be detected is subjected to multi-scale feature extraction by passing it through the bottleneck module of the fused deformable convolution of the target coal mine smoke and fire detection model to obtain a multi-scale coal mine feature image. The multi-scale coal mine feature image can be further subjected to attention calculation by passing it through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a multi-scale attention coal mine feature image. Thus, the multi-scale attention coal mine feature image can be subjected to feature fusion by passing it through the target coal mine smoke and fire detection model to obtain a fused coal mine feature image. Furthermore, a more accurate target detection result can be determined according to the fused coal mine feature image by the target coal mine smoke and fire detection model, which is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection.

[0075] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0076] In one embodiment, a coal mine smoke and fire detection device based on improved YOLOv8 is provided, and the coal mine smoke and fire detection device based on improved YOLOv8 corresponds to the coal mine smoke and fire detection method based on improved YOLOv8 in the above embodiment. Figure 3As shown, the coal mine smoke and fire detection device based on improved YOLOv8 includes an acquisition module 101, a first processing module 102, a second processing module 103, a third processing module 104 and a detection module 105. The functional modules are described in detail as follows:

[0077] An acquisition module 101 is used to acquire a smoke and fire image of a coal mine and an image of a coal mine to be detected;

[0078] The first processing module 102 is used to pre-process the smoke and fire image to obtain an initial smoke and fire image;

[0079] The second processing module 103 is used to construct an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model;

[0080] The third processing module 104 is used to train the initial coal mine smoke and fire detection model using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model;

[0081] The detection module 105 is used to use the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

[0082] In one embodiment, the second processing module 103 is used to construct an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model, specifically for:

[0083] The deformable convolution operator is introduced into the backbone network of the improved YOLOv8 target detection model to reconstruct the backbone network and obtain the reconstructed backbone network.

[0084] The neck network of the improved YOLOv8 target detection model is introduced into the efficient multi-scale attention module to reconstruct the neck network and obtain the reconstructed neck network.

[0085] Replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function;

[0086] An initial coal mine smoke and fire detection model is constructed according to the reconstructed backbone network, the reconstructed neck network and the target loss function.

[0087] In one embodiment, the second processing module 103 is used to replace the original loss function of the improved YOLOv8 target detection model to obtain a target loss function, specifically for:

[0088] Use the following formula to replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function:

[0089]

[0090]

[0091] Among them, MPDIoU represents the target loss function; IoU represents the intersection over union ratio between the real box and the predicted box; Represents the square of the Euclidean distance between the upper left corner of the true box and the predicted box; represents the square of the Euclidean distance between the lower right corner of the real box and the predicted box; h represents the height of the input image; w represents the width of the input image; B gt represents the real box; B pre Represents the prediction box; The horizontal coordinate of the upper left corner of the prediction box; The horizontal coordinate of the upper left corner of the real box; The ordinate represents the coordinate of the upper left corner of the prediction box; The ordinate represents the coordinate of the upper left corner of the real box; The horizontal coordinate of the lower right corner of the prediction box; The horizontal coordinate of the lower right corner of the real box; The ordinate represents the coordinate of the lower right corner of the prediction box; The ordinate representing the lower right corner coordinate of the ground-truth box.

[0092] In one embodiment, the detection module 105 is used to use the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result, specifically for:

[0093] The coal mine image to be detected is subjected to multi-scale feature extraction through the bottleneck module of the target coal mine smoke and fire detection model that is fused with deformable convolution, so as to obtain a multi-scale coal mine feature image;

[0094] The multi-scale coal mine feature image is passed through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to perform attention calculation to obtain a multi-scale attention coal mine feature image;

[0095] The multi-scale attention coal mine feature image is subjected to feature fusion through the target coal mine smoke and fire detection model to obtain a fused coal mine feature image;

[0096] The target coal mine smoke and fire detection model is used to determine the target detection result according to the fused coal mine feature image.

[0097] In one embodiment, the detection module 105 is used to extract multi-scale features from the coal mine image to be detected through the bottleneck module of the fused deformable convolution of the target coal mine smoke and fire detection model to obtain a multi-scale coal mine feature image, specifically for:

[0098] The coal mine images to be detected are grouped and processed by a bottleneck module of the target coal mine smoke and fire detection model that is fused with deformable convolution to obtain an initial coal mine image set;

[0099] Through the bottleneck module of the target coal mine smoke and fire detection model fused with deformable convolution, each initial coal mine image in the initial coal mine image set is subjected to a deformable convolution operation to obtain an offset set and a weight set;

[0100] Through the bottleneck module of the fused deformable convolution of the target coal mine smoke and fire detection model, each initial coal mine image in the initial coal mine image set is spliced ​​according to the corresponding offset and weight according to the offset set and the weight set to obtain a multi-scale coal mine feature image.

[0101] In one embodiment, the detection module 105 is used to perform attention calculation on the multi-scale coal mine feature image through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain the multi-scale attention coal mine feature image, which is specifically used to:

[0102] The multi-scale coal mine feature image is divided and processed by the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first feature map and a second feature map;

[0103] By means of the efficient multi-scale attention module of the target coal mine smoke and fire detection model, the first feature map is encoded along the spatial direction to obtain a first encoded feature map;

[0104] The first encoding feature map is adaptively selected with a high-efficiency multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first adaptive feature map;

[0105] The second feature map is convolved by the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second multi-scale feature map;

[0106] Through the efficient multi-scale attention module of the target coal mine smoke and fire detection model, the first adaptive feature map is subjected to two-dimensional global pooling processing to obtain a first global feature map, and the second multi-scale feature map is subjected to two-dimensional global pooling processing to obtain a second global feature map;

[0107] Performing a matrix multiplication operation on the first global feature map and the second multi-scale feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first cross-space feature map;

[0108] Performing a matrix multiplication operation on the second global feature map and the first adaptive feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second cross-space feature map;

[0109] The first cross-space feature map and the second cross-space feature map are aggregated through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a multi-scale attention coal mine feature image.

[0110] The present invention provides a coal mine smoke and fire detection device based on improved YOLOv8. By acquiring a smoke and fire image of a coal mine and a coal mine image to be detected, the smoke and fire image is preprocessed to obtain an initial smoke and fire image, and an initial coal mine smoke and fire detection model is constructed based on an improved YOLOv8 target detection model. Therefore, the coal mine smoke and fire image data set can be used to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model. Then, the target coal mine smoke and fire detection model can be used to perform smoke and fire detection on the coal mine image to be detected to obtain a more accurate target detection result, which can effectively improve the accuracy of coal mine smoke and fire detection and is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection.

[0111] For the specific definition of the coal mine smoke and fire detection device based on improved YOLOv8, please refer to the definition of the coal mine smoke and fire detection method based on improved YOLOv8 above, which will not be repeated here. Each module in the above-mentioned coal mine smoke and fire detection device based on improved YOLOv8 can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0112] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 4 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes a function or step on the service side of a coal mine smoke and fire detection method based on an improved YOLOv8.

[0113] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server through a network connection. When the computer program is executed by the processor, it realizes a function or step on the client side of a coal mine smoke and fire detection method based on an improved YOLOv8.

[0114] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the following steps are implemented:

[0115] Acquire coal mine smoke and fire images and coal mine images to be detected;

[0116] Preprocessing the smoke and fire image to obtain an initial smoke and fire image;

[0117] Based on the improved YOLOv8 target detection model, an initial coal mine smoke and fire detection model was constructed;

[0118] Using the coal mine smoke and fire image dataset to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model;

[0119] The target coal mine smoke and fire detection model is used to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

[0120] The present invention provides a computer device, which obtains a smoke and fire image of a coal mine and an image of a coal mine to be detected, so as to pre-process the smoke and fire image to obtain an initial smoke and fire image, and constructs an initial coal mine smoke and fire detection model based on an improved YOLOv8 target detection model, so that the initial coal mine smoke and fire detection model can be trained by using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model, and then the target coal mine smoke and fire detection model can be used to perform smoke and fire detection on the coal mine image to be detected to obtain a more accurate target detection result, which can effectively improve the accuracy of coal mine smoke and fire detection, and is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection.

[0121] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0122] Acquire coal mine smoke and fire images and coal mine images to be detected;

[0123] Preprocessing the smoke and fire image to obtain an initial smoke and fire image;

[0124] Based on the improved YOLOv8 target detection model, an initial coal mine smoke and fire detection model was constructed;

[0125] Using the coal mine smoke and fire image dataset to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model;

[0126] The target coal mine smoke and fire detection model is used to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

[0127] The present invention provides a computer-readable storage medium, which obtains a smoke and fire image of a coal mine and a coal mine image to be detected, so as to pre-process the smoke and fire image to obtain an initial smoke and fire image, and construct an initial coal mine smoke and fire detection model based on an improved YOLOv8 target detection model, so that the initial coal mine smoke and fire detection model can be trained using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model, and then the target coal mine smoke and fire detection model can be used to perform smoke and fire detection on the coal mine image to be detected to obtain a more accurate target detection result, which can effectively improve the accuracy of coal mine smoke and fire detection, and is conducive to achieving the purpose of accurate and efficient coal mine smoke and fire detection.

[0128] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0129] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0130] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0131] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A coal mine smoke and fire detection method based on improved YOLOv8, characterized in that: The method comprises: Acquire coal mine smoke and fire images and coal mine images to be detected; Preprocessing the smoke and fire image to obtain an initial smoke and fire image; Based on the improved YOLOv8 target detection model, an initial coal mine smoke and fire detection model was constructed; Using the coal mine smoke and fire image dataset to train the initial coal mine smoke and fire detection model to obtain a target coal mine smoke and fire detection model; The target coal mine smoke and fire detection model is used to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

2. The coal mine smoke and fire detection method based on improved YOLOv8 according to claim 1 is characterized in that: The initial coal mine smoke and fire detection model is constructed based on the improved YOLOv8 target detection model, including: The deformable convolution operator is introduced into the backbone network of the improved YOLOv8 target detection model to reconstruct the backbone network and obtain the reconstructed backbone network. The neck network of the improved YOLOv8 target detection model is introduced into the efficient multi-scale attention module to reconstruct the neck network and obtain the reconstructed neck network. Replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function; An initial coal mine smoke and fire detection model is constructed according to the reconstructed backbone network, the reconstructed neck network and the target loss function.

3. The coal mine smoke and fire detection method based on improved YOLOv8 according to claim 2 is characterized in that: The original loss function of the improved YOLOv8 target detection model is replaced to obtain the target loss function, including: Use the following formula to replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function: Among them, MPDIoU represents the target loss function; IoU represents the intersection over union ratio between the real box and the predicted box; Represents the square of the Euclidean distance between the upper left corner of the true box and the predicted box; represents the square of the Euclidean distance between the lower right corner of the real box and the predicted box; h represents the height of the input image; w represents the width of the input image; B gt represents the real box; B pre Represents the prediction box; The horizontal coordinate of the upper left corner of the prediction box; The horizontal coordinate of the upper left corner of the real box; The ordinate represents the coordinate of the upper left corner of the prediction box; The ordinate represents the coordinate of the upper left corner of the real box; The horizontal coordinate of the lower right corner of the prediction box; The horizontal coordinate of the lower right corner of the real box; The ordinate represents the coordinate of the lower right corner of the prediction box; The ordinate representing the lower right corner coordinate of the ground-truth box.

4. The coal mine smoke and fire detection method based on improved YOLOv8 according to claim 2 is characterized in that: The target coal mine smoke and fire detection model is used to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result, including: The coal mine image to be detected is subjected to multi-scale feature extraction through the bottleneck module of the target coal mine smoke and fire detection model that is fused with deformable convolution, so as to obtain a multi-scale coal mine feature image; The multi-scale coal mine feature image is passed through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to perform attention calculation to obtain a multi-scale attention coal mine feature image; The multi-scale attention coal mine feature image is subjected to feature fusion through the target coal mine smoke and fire detection model to obtain a fused coal mine feature image; The target coal mine smoke and fire detection model is used to determine the target detection result according to the fused coal mine feature image.

5. The coal mine smoke and fire detection method based on improved YOLOv8 according to claim 4 is characterized in that: The method of extracting multi-scale features from the coal mine image to be detected by passing the bottleneck module of the target coal mine smoke and fire detection model through the fused deformable convolution to obtain a multi-scale coal mine feature image includes: The coal mine images to be detected are grouped and processed by a bottleneck module of the target coal mine smoke and fire detection model that is fused with deformable convolution to obtain an initial coal mine image set; Through the bottleneck module of the target coal mine smoke and fire detection model fused with deformable convolution, each initial coal mine image in the initial coal mine image set is subjected to a deformable convolution operation to obtain an offset set and a weight set; Through the bottleneck module of the fused deformable convolution of the target coal mine smoke and fire detection model, each initial coal mine image in the initial coal mine image set is spliced ​​according to the corresponding offset and weight according to the offset set and the weight set to obtain a multi-scale coal mine feature image.

6. The coal mine smoke and fire detection method based on improved YOLOv8 according to any one of claims 1-5, characterized in that: The multi-scale coal mine feature image is passed through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to perform attention calculation to obtain the multi-scale attention coal mine feature image, including: The multi-scale coal mine feature image is divided and processed by the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first feature map and a second feature map; By means of the efficient multi-scale attention module of the target coal mine smoke and fire detection model, the first feature map is encoded along the spatial direction to obtain a first encoded feature map; The first encoding feature map is adaptively selected with a high-efficiency multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first adaptive feature map; The second feature map is convolved by the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second multi-scale feature map; Through the efficient multi-scale attention module of the target coal mine smoke and fire detection model, the first adaptive feature map is subjected to two-dimensional global pooling processing to obtain a first global feature map, and the second multi-scale feature map is subjected to two-dimensional global pooling processing to obtain a second global feature map; Performing a matrix multiplication operation on the first global feature map and the second multi-scale feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a first cross-space feature map; Performing a matrix multiplication operation on the second global feature map and the first adaptive feature map through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a second cross-space feature map; The first cross-space feature map and the second cross-space feature map are aggregated through the efficient multi-scale attention module of the target coal mine smoke and fire detection model to obtain a multi-scale attention coal mine feature image.

7. A coal mine smoke and fire detection device based on improved YOLOv8, characterized in that: The coal mine smoke and fire detection device based on improved YOLOv8 includes: An acquisition module, used for acquiring smoke and fire images of coal mines and images of coal mines to be detected; A first processing module, used for preprocessing the smoke and fire image to obtain an initial smoke and fire image; The second processing module is used to build an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model; A third processing module is used to train the initial coal mine smoke and fire detection model using the coal mine smoke and fire image data set to obtain a target coal mine smoke and fire detection model; The detection module is used to use the target coal mine smoke and fire detection model to perform smoke and fire detection on the coal mine image to be detected to obtain a target detection result.

8. The coal mine smoke and fire detection method based on improved YOLOv8 according to claim 7, characterized in that: The second processing module is used to build an initial coal mine smoke and fire detection model based on the improved YOLOv8 target detection model, specifically for: The deformable convolution operator is introduced into the backbone network of the improved YOLOv8 target detection model to reconstruct the backbone network and obtain the reconstructed backbone network. The neck network of the improved YOLOv8 target detection model is introduced into the efficient multi-scale attention module to reconstruct the neck network and obtain the reconstructed neck network. Replace the original loss function of the improved YOLOv8 target detection model to obtain the target loss function; An initial coal mine smoke and fire detection model is constructed according to the reconstructed backbone network, the reconstructed neck network and the target loss function.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the coal mine smoke and fire detection method based on improved YOLOv8 as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the coal mine smoke and fire detection method based on improved YOLOv8 as described in any one of claims 1 to 6 is implemented.

Images