Visible light image-based coal rock identification method in coal mining process
By using deep convolutional neural network and wavelet transform to enhance the image in coal and rock identification technology, and combining it with the YOLO v5 model for transfer learning, the problem of low accuracy and efficiency of coal and rock identification in the complex environment of coal mines is solved, and higher recognition is achieved. Accuracy and robustness.
Patent Information
- Application Number
- CN202411356270.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2025-01-24
AI Technical Summary
Existing coal rock identification technology is difficult to achieve high-precision and high-efficiency exposed coal rock identification in complex coal mine underground environments.
The deep convolutional neural network method based on visible light images is adopted, and the migration learning of the YOLO V5 model is combined with the enhancement of the image pre -processing and the wavelet transformation.
It significantly improves the characteristic clarity and recognition accuracy of the coal rock interface, and enhances the robustness and efficiency of the identification model.
Smart Images
Figure CN119360353A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a coal-rock identification method in a coal mining process, belonging to the field of coal-rock identification. Background Art
[0002] Intelligent recognition of coal-rock interface is one of the key technologies to realize intelligent and unmanned mining and tunneling. At present, the recognition technology for exposed coal and rock is mainly based on visible light images. The principle is to first distinguish the respective areas of coal and rock according to the different characteristics of coal and rock in the image (such as grayscale, color, texture, shape and position); then, collect a large number of annotated coal and rock images as training samples to establish a detection or segmentation model that can automatically identify the coal-rock interface. However, due to the complex environmental conditions in coal mines (such as dust, vibration, water spray, etc.) and the limitations of sensor performance, how to achieve high-precision and high-efficiency recognition of exposed coal and rock in coal mining and tunneling working faces is still one of the technical problems that need to be solved urgently in the coal mining industry.
[0003] In order to solve the above problems, it is necessary to first enhance the features of the image to improve the quality, clarity and visualization of the coal-rock image, and improve the visual perception and usability of the image. Commonly used methods include Gabor filtering, gray-level co-occurrence matrix and image moment. These methods can improve the recognition effect to a certain extent by enhancing the local features of the image. However, the method based on manually designed features is difficult to fully capture the visual features such as the complex texture and shape of the coal-rock interface, and it may be difficult to achieve ideal recognition results for complex coal-rock scenes.
[0004] Secondly, it is necessary to select appropriate artificial intelligence algorithms to train exposed coal and rock images and establish machine learning or neural network models to achieve coal and rock detection. At present, the main algorithms include traditional machine learning methods such as support vector machines and random forests, as well as deep learning technologies such as convolutional neural networks. Support vector machines and random forests have certain advantages in processing linear and nonlinear data, but the performance of these algorithms depends largely on manually designed feature engineering. Once the features are improperly designed, the recognition effect will be seriously affected. In contrast, convolutional neural networks can automatically extract deep features in images and optimize the entire model through end-to-end training, so as to better adapt to complex coal and rock scenes.
[0005] In addition, traditional machine learning methods usually treat feature extraction and classification as two independent steps, which cannot achieve end-to-end optimization and make it difficult to give full play to the overall performance. This limitation makes the traditional methods unsatisfactory in complex coal and rock identification tasks.
[0006] Therefore, there is an urgent need for a coal-rock image recognition method that can adapt to complex coal-rock scenes and has high recognition accuracy. Summary of the invention
[0007] In view of the problems that the current coal-rock image recognition model has poor accuracy, efficiency and robustness, the present invention provides a method for coal-rock interface recognition based on visible light images using a deep convolutional neural network.
[0008] In order to effectively solve the above problems, the technical solution provided by the present invention is as follows:
[0009] In a first aspect, the present invention provides a method for identifying coal and rock in a coal mining process based on visible light images, comprising the following steps:
[0010] Obtaining the original image of coal rock;
[0011] Preprocessing the original image to obtain a preprocessed image;
[0012] Perform image enhancement on the preprocessed image to generate an enhanced image;
[0013] Annotate the enhanced images and divide them into training set, validation set and test set;
[0014] Use the YOLO v5 model as the basic model for training;
[0015] Obtain the final recognition model, perform coal and rock image detection, and output the results.
[0016] In one possible implementation, the preprocessing method includes:
[0017] Use Gaussian filtering algorithm to remove noise from the original image;
[0018] Convert the denoised image to grayscale.
[0019] In one possible implementation, the image enhancement method includes:
[0020] The preprocessed image is decomposed into low-frequency components and high-frequency components by multi-scale wavelet decomposition using Haar wavelet basis function;
[0021] Smooth the low-frequency components to preserve the overall structure of the image; enhance the high-frequency components to highlight the edges and detail features of the image;
[0022] Through inverse wavelet transform, the processed low-frequency components and high-frequency components are combined to generate an enhanced image.
[0023] In one possible implementation, the enhanced images are annotated and divided into a training set, a validation set, and a test set, including:
[0024] Use annotation tools to annotate the enhanced image; the annotation results include category, specific location and shape;
[0025] The labeled image dataset is divided into training set, validation set and test set.
[0026] In one possible implementation, the YOLO v5 model is used as the base model, and the training method includes:
[0027] Load the pre-trained YOLO v5 model and its weight parameters;
[0028] Input the training set into the YOLO v5 model for migration training;
[0029] Use the optimization algorithm SGD to adjust the network structure and parameters of the model;
[0030] Use the validation set to monitor and evaluate the training process in real time.
[0031] In one possible implementation, the output result includes the position coordinates, size, and classification confidence of the detection box.
[0032] In a second aspect, the present invention provides a device for identifying coal and rock in a coal mining process based on visible light images, comprising:
[0033] An acquisition module, used for acquiring the original image of coal and rock;
[0034] A preprocessing module, used for preprocessing the original image to obtain a preprocessed image;
[0035] An image enhancement module is used to perform image enhancement on the preprocessed image to generate an enhanced image;
[0036] The annotation module is used to annotate the enhanced images and divide them into training set, validation set and test set;
[0037] Use the YOLO v5 model as the basic model for training;
[0038] The output module is used to obtain the final recognition model, perform coal and rock image detection, and output the results.
[0039] In a third aspect, the present invention provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for identifying coal and rock during coal mining based on visible light images as described in the first aspect is implemented.
[0040] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for identifying coal and rock during coal mining based on visible light images as described in the first aspect.
[0041] In a fifth aspect, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the method for identifying coal and rock in the coal mining process based on visible light images as described in the first aspect.
[0042] Compared with the prior art, the present invention has the following beneficial effects:
[0043] The method of the present invention, by adopting an image enhancement method based on wavelet transform, can significantly improve the quality, details and contrast of the image, making the features of the coal-rock interface clearer, facilitating the subsequent recognition process and improving the recognition accuracy and efficiency.
[0044] The method described in the present invention utilizes the YOLO v5 model for transfer learning. Compared with traditional machine learning methods, the deep learning model can automatically extract high-level features of images, realize end-to-end training, and greatly improve recognition accuracy and stability. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0046] Figure 1 This is a flow chart of the coal-rock identification method in the coal mining process based on visible light images of the present invention;
[0047] Figure 2 is the error and prediction accuracy curve during the training process of the embodiment of the present invention;
[0048] Figure 3 It is a structural schematic diagram of a coal-rock identification device in a coal mining process based on visible light images provided by the present invention;
[0049] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0050] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0051] refer to Figure 1, a method for identifying coal and rock in coal mining process based on visible light image, comprising the following steps:
[0052] Step 1, obtaining the original image of coal rock.
[0053] Specifically, the intrinsically safe camera is used to obtain the original image of coal and rock in the coal mine. The installation position and angle of the camera are set to ensure that the key area can be covered and a clear image can be obtained.
[0054] Step 2: preprocess the original image to obtain a preprocessed image.
[0055] In one possible implementation,
[0056] 2.1, use Gaussian filtering algorithm to remove noise from the original image and improve the clarity of the image;
[0057] 2.2, the denoised image is converted into a grayscale image to reduce the computational complexity while retaining the main features of the image.
[0058] Step 3: Perform image enhancement on the preprocessed image to generate an enhanced image.
[0059] In one possible implementation, the image enhancement method includes:
[0060] 3.1, use Haar wavelet basis function to perform multi-scale wavelet decomposition on the image, decompose the image into low-frequency components and high-frequency components, and process them separately.
[0061] 3.2, smooth the low-frequency components to preserve the overall structure of the image, and enhance the high-frequency components to highlight the edges and details of the image.
[0062] 3.3, through inverse wavelet transform, the processed low-frequency components and high-frequency components are combined to generate an enhanced image, ensuring that the enhanced image has high contrast and detail clarity.
[0063] Step 4: annotate the enhanced images and divide them into training set, validation set and test set;
[0064] In a possible implementation, step 4 includes the following steps:
[0065] 4.1, use the professional labeling tool LabelImg for labeling. The labeling results include the category, specific location and shape of coal and rock in each image. Coal is labeled as "Coal" and rock is labeled as "Rock".
[0066] 4.2, the labeled image dataset is divided into training set, validation set and test set, which are 60%, 20% and 20% respectively. The training set is used for model training, the validation set is used to adjust model parameters, and the test set is used to evaluate model performance.
[0067] It should be noted that after completing the labeling in step 4.1, LabelImg saves the labeling information of each image as a file in JSON format. Each JSON file corresponds to an image file and records the coordinates and category information of all targets in the image.
[0068] Step 5: Select the pre-trained YOLO v5 model as the base model for training.
[0069] In a possible implementation, step 5 includes the following steps:
[0070] 5.1, load the pre-trained YOLO v5 model and its weight parameters.
[0071] 5.2, input the training set into the YOLO v5 model for transfer training (the image size is set to 640×640).
[0072] 5.3. Use the optimization algorithm SGD (Stochastic Gradient Descent) to adjust the network structure and parameters of the model to adapt to the coal and rock identification task.
[0073] 5.4, use the validation set to monitor and evaluate the training process in real time to ensure the accuracy and robustness of the model.
[0074] Furthermore, the pre-trained YOLO v5 model is obtained by pre-training a basic model on a large-scale public dataset (such as the COCO dataset).
[0075] Furthermore, in step 5.1, the parameters learned on the public dataset are used as initial weights.
[0076] Furthermore, in the use of the optimization algorithm SGD to adjust the network structure and parameters of the model, the parameters include learning rate, batch and number of rounds.
[0077] Specifically, in this embodiment, the training hyperparameter learning rate is set to 0.001, the batch size is set to 32, and the number of training rounds is set to 200.
[0078] Step 6, obtain the final recognition model, perform coal rock image detection, and output the results.
[0079] In one possible implementation, step 6 includes:
[0080] 6.1, Load the final recognition model.
[0081] 6.2, input the coal and rock images to be detected into the model.
[0082] 6.3, output the bounding box of coal and rock. The output results include the position coordinates, size and classification confidence of each detection box (the value is between 0 and 1, and the larger the value, the more reliable the prediction result).
[0083] Figure 2 The error (a) and prediction accuracy curve (b) during the training process of the embodiment of the present invention are shown in FIG. Figure 2 It can be seen that the model error gradually decreases during the training process ( Figure 2 a), the prediction accuracy gradually increases ( Figure 2 b).
[0084] The coal-rock identification device in the coal mining process based on visible light images provided by the present invention is described below. The coal-rock identification device in the coal mining process based on visible light images described below and the coal-rock identification method in the coal mining process based on visible light images described above can be referenced to each other.
[0085] Figure 3 is a schematic diagram of the structure of a coal-rock identification device in a coal mining process based on visible light images provided by an embodiment of the present invention. Figure 3 As shown, it includes: an acquisition module 31, a preprocessing module 32, an image enhancement module 33, a labeling module 34 and an output module 35, wherein:
[0086] An acquisition module 31 is used to acquire the original image of the coal rock;
[0087] The preprocessing module 32 is used to preprocess the original image to obtain a preprocessed image;
[0088] An image enhancement module 33 is used to perform image enhancement on the preprocessed image to generate an enhanced image;
[0089] Annotation module 34, used to annotate the enhanced images and divide them into training set, verification set and test set;
[0090] Use the YOLO v5 model as the basic model for training;
[0091] The output module 35 is used to obtain the final recognition model, perform coal-rock image detection, and output the results.
[0092] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the coal rock identification method in the coal mining process based on visible light images.
[0093] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.
[0094] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the coal and rock identification method based on visible light images in the coal mining process provided by the above-mentioned methods.
[0095] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the coal and rock identification method based on visible light images during coal mining provided by the above-mentioned methods.
[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.
[0097] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0098] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying coal and rock in coal mining based on visible light images, characterized in that: The following steps are involved: Obtaining the original image of coal rock; Preprocessing the original image to obtain a preprocessed image; Perform image enhancement on the preprocessed image to generate an enhanced image; Annotate the enhanced images and divide them into training set, validation set and test set; Use the YOLO v5 model as the basic model for training; Obtain the final recognition model, perform coal and rock image detection, and output the results.
2. The method for identifying coal and rock in coal mining based on visible light images according to claim 1, characterized in that: The pretreatment method comprises: Use Gaussian filtering algorithm to remove noise from the original image; Convert the denoised image to grayscale.
3. The method for identifying coal and rock in coal mining based on visible light images according to claim 1, characterized in that: The image enhancement method comprises: The preprocessed image is decomposed into low-frequency components and high-frequency components by multi-scale wavelet decomposition using Haar wavelet basis function; Smooth the low-frequency components to preserve the overall structure of the image; enhance the high-frequency components to highlight the edges and detail features of the image; Through inverse wavelet transform, the processed low-frequency components and high-frequency components are combined to generate an enhanced image.
4. The method for identifying coal and rock in coal mining based on visible light images according to claim 1, characterized in that: The method of annotating the enhanced images and dividing them into training sets, validation sets and test sets includes: Use annotation tools to annotate the enhanced image; the annotation results include category, specific location and shape; The labeled image dataset is divided into training set, validation set and test set.
5. The method for identifying coal and rock in coal mining based on visible light images according to claim 1, characterized in that: Using the YOLO v5 model as the base model, the training methods include: Load the pre-trained YOLO v5 model and its weight parameters; Input the training set into the YOLO v5 model for migration training; Use the optimization algorithm SGD to adjust the network structure and parameters of the model; Use the validation set to monitor and evaluate the training process in real time.
6. The method for identifying coal and rock in coal mining based on visible light images according to claim 1, characterized in that: The output result includes the position coordinates, size and classification confidence of the detection box.
7. A device for identifying coal and rock in the process of coal mining based on visible light images, characterized in that: include: An acquisition module, used for acquiring the original image of coal and rock; A preprocessing module, used for preprocessing the original image to obtain a preprocessed image; An image enhancement module is used to perform image enhancement on the preprocessed image to generate an enhanced image; The annotation module is used to annotate the enhanced images and divide them into training set, validation set and test set; Use the YOLO v5 model as the basic model for training; The output module is used to obtain the final recognition model, perform coal and rock image detection, and output the results.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for identifying coal and rock in the coal mining process based on visible light images as described in any one of claims 1 to 6 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying coal and rock in a coal mining process based on visible light images as described in any one of claims 1 to 6 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying coal and rock in a coal mining process based on visible light images as described in any one of claims 1 to 6 is implemented.