Coal seam hole wall crack capturing method based on YOLO algorithm

Abstract

The invention belongs to the field of convolutional neural networks, and particularly relates to a coal seam hole wall crack capturing method based on a YOLO algorithm. According to the method, a large number of high-precision coal seam hole wall crack pictures are manually collected, data are preprocessed, feature information is extracted and then put into a model for repeated training, and a loss function is continuously optimized through back propagation, so that the model achieves the effect of accurately recognizing the coal seam hole wall cracks; and finally applying the trained model to an actual coal seam hole wall image, and capturing a coal seam hole wall crack. According to the method, the high-precision automatic identification function of the hole wall cracks of the coal seam is realized, the manual workload is greatly reduced, meanwhile, a guarantee is provided for the personnel safety of a coal mine, the coal mining efficiency is improved, and a scheme is provided for underground route arrangement of the coal mine.

Description

Technical Field

[0001] The present invention relates to the field of convolutional neural networks, and in particular to a method for capturing coal seam hole wall cracks based on a YOLO algorithm. Background Art

[0002] With the improvement of mine automation, it is particularly important to accurately capture cracks in the borehole wall. For example, in measuring ground stress and the range of the loose circle of surrounding rock, accurate crack positioning can provide guarantee for the stability of the tunnel. Traditional crack detection methods often rely on manual visual inspection, but cracks are often not easy to detect. When observing and processing more borehole wall images, manual inspection is often inefficient and has limited accuracy, which ultimately leads to inaccurate analysis results. Reasonable and accurate measures are not taken for tunnel reinforcement, which may eventually lead to loosening of the tunnel and even cause casualties. Summary of the invention

[0003] The object of the present invention is to provide a coal seam hole wall crack capture method based on the YOLO algorithm.

[0004] The object of the present invention is achieved by the following approach: a coal seam hole wall crack capture method based on YOLO algorithm, which comprises the following steps:

[0005] Step 1: Collect high-precision coal seam hole wall crack images based on manual exploration of the hole wall;

[0006] Step 2: Enhance the crack features of the hole wall by preprocessing the collected images;

[0007] Step 3: Manually mark the cracks in the preprocessed image dataset and determine the location range of the cracks;

[0008] Step 4: Model training, use the YOLOV5 model to train the image dataset;

[0009] Step 5: Model testing: put the independent sample set that did not participate in the training into the model for testing, check the accuracy and loss value indicators, and ensure the model effect. If the independent sample that participated in the training failed to obtain the complete effect, continue to step 4 training and increase the number of training times. Otherwise, proceed to the next step;

[0010] Step 6: Model application: Apply the trained model to the new coal seam hole wall image to obtain the crack detection results.

[0011] As a further optimization, the specific operation of step 1 is: the mine staff first drills a hole at the location to be tested with a drill rig, and manually uses a peeping device to spy on the hole. The peeping data is saved and cropped into several hole wall crack images, from which high-resolution crack images are manually selected.

[0012] As a further optimization, the specific operations of step 2 are: cropping, binarization, and normalization of the collected images.

[0013] As a further optimization, the specific operation of step 3 is: use labelimg software to manually mark the cracks in the coal seam pore wall in the image.

[0014] As a further optimization, the specific operation of step 4 is: use the YOLOV5 model for training, which is based on the CNN convolutional neural network architecture and uses ReLU() as the activation function. Its function expression is:

[0015] f(x) = max(0, x), which means that when the input x is greater than 0, the output is equal to the input; when the input x is less than or equal to 0, the output is 0. In principle, it introduces nonlinear factors to the neural network by simply truncating the negative part to 0 and retaining the positive part;

[0016] And combined with the gradient descent algorithm, the formula is:

[0017]

[0018] W new is the updated weight, W old is the current weight α is the learning rate, It is the gradient of the loss function with respect to the weight, and then the weight is updated through back propagation to optimize the loss function; and during the training process, 60% is randomly selected as the training set, 20% as the validation set, and 20% as the test set;

[0019] The specific process of training is:

[0020] a. Split the training set and validation set images into batches, train the network with a small batch of data each iteration and then validate the network;

[0021] b. Use the cross entropy loss function to calculate the difference between the predicted result and the actual label;

[0022] The cross entropy loss function formula is:

[0023]

[0024] The true label is y and the predicted probability is The smaller the value of the cross entropy loss function, the more accurate the model's prediction;

[0025] c. Use the back-propagation algorithm to update the network weights to minimize the loss and improve the accuracy and verification rate;

[0026] d. Iterate the process until the loss function converges and the training is completed. If the loss function still does not converge, continue to iterate until the loss function converges.

[0027] As a further optimization, the specific operation of step 5 is: put the randomly selected 20% test set into the model for testing, propagate forward through the model, and obtain the final crack identification result; if the effect is not good, retrain and increase the number of training times, otherwise, proceed to step 6.

[0028] The present invention is based on the architecture of convolutional neural networks, which is a fusion of the fields of convolutional neural networks and mining engineering. It realizes the high-precision automatic recognition function of coal seam hole wall cracks, greatly reduces the manual workload, and also provides protection for the safety of coal mine personnel, improves the efficiency of coal mining, and provides solutions for route layout under mines.

[0029] Compared with the prior art, the coal seam hole wall crack capture method based on the YOLO algorithm of the present invention has the following beneficial effects:

[0030] 1. Automatic feature learning: The present invention is based on the YOLO algorithm, and its core CNN convolutional neural network architecture can automatically learn and extract image features. Through training on a large number of high-precision coal seam bore wall crack images, the model can deeply understand the various patterns of cracks, such as crack characteristics of different widths, lengths, and directions. For example, in the training process of 1,000 pictures of different coal seam bore wall cracks, the model can gradually identify subtle crack changes and distinguish the crack differences caused by different geological structures, thereby locating the crack position more accurately. Use convolution kernels of different sizes in the convolution layer to scan the image and capture crack feature information of different scales. Taking convolution kernels of different sizes such as 3x3 and 5x5 as examples, they can focus on small-scale subtle cracks and large-scale main crack features respectively, making the model's recognition of cracks more comprehensive and accurate.

[0031] 2. Rapidly process large amounts of data and reduce human errors: The present invention can process massive amounts of borehole wall image data in a short period of time. For example, on a computer equipped with an ordinary GPU (graphics processing unit), the algorithm of the present invention can process about 500 high-resolution (such as 5000x5000 pixels) borehole wall images per hour, while manual processing of the same number and resolution of images may take several days. Traditional manual visual inspection methods are easily affected by subjective factors when judging cracks, such as fatigue and experience differences of observers. Once the algorithm of the present invention is trained, the crack recognition results for the same image are highly consistent.

[0032] 3. On-site real-time operation: The CNN model of the present invention can be deployed in the monitoring equipment at the mine site to obtain and analyze the borehole wall images in real time. For example, at the borehole monitoring point in the coal mine, by installing a high-definition camera and an embedded computing device to run the model of the present invention, the borehole wall can be scanned and detected every few minutes (such as 5 minutes) to detect crack changes in a timely manner.

[0033] 4. Optimize mining strategies: By accurately detecting cracks in coal seam pore walls, coal mining companies can formulate more reasonable mining strategies based on the distribution of cracks. For example, in a mining area of ​​a coal mine, the mining direction and cutting parameters of the coal mining machine were adjusted based on the crack direction and density detected by the model of the present invention, thus avoiding the waste of coal resources caused by blind mining.

[0034] 5. Avoid mining accidents: Accurate identification of fissures can effectively prevent mining accidents caused by fissures, such as roof collapse. In the statistical analysis of historical accidents in multiple coal mines, it was found that there were an average of 3-5 mining accidents caused by fissures each year, which also avoided indirect losses such as waste of coal resources and equipment damage caused by accidents.

[0035] 6. Reduce costs: Reduce mining interruptions. Since the present invention can accurately detect cracks in the coal seam bore wall in advance, coal mining enterprises can take measures in advance to avoid mining interruptions caused by problems such as sudden expansion of cracks. For example, in a working face of a coal mine, after monitoring using the method of the present invention, the number of mining interruptions caused by cracks was reduced from 5 times a year to less than 1 time, reducing idle equipment and waste of manpower; traditional manual detection methods require a lot of manpower input, and long-term work can easily lead to fatigue and low efficiency. Taking a medium-sized coal mine as an example, if manual detection is used, 5 workers are required every day to be responsible for bore wall crack detection work, while using the method of the present invention only requires 1 technician to operate the equipment and maintain the model, which greatly saves labor costs.

[0036] 7. Strong scalability: Adaptable to different image types and angles, the trained CNN model of the present invention can adapt to different types of coal seam borehole wall images, whether dry, wet or covered with a certain amount of dust, and can better identify cracks. In the test of 100 borehole wall images collected under different environments, the model's accuracy rate for crack recognition in dry borehole wall images reached 92%, the accuracy rate for wet borehole wall images was 88%, and the accuracy rate for borehole wall images covered with a small amount of dust was 85%, all maintaining a high level. And it can be applied in a variety of geological conditions. The characteristics of coal seam borehole wall cracks under different geological conditions are different. The model of the present invention can be fine-tuned and trained by collecting a small number of samples in different geological areas to adapt to new geological conditions.

[0037] Comparison between the traditional method and the method of the present invention:

[0038]

[0039] BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The present invention is further described in detail below in conjunction with the accompanying drawings:

[0041] Figure 1 is a flow chart of the present invention;

[0042] Figure 2 It is the coal seam hole wall diagram of the present invention;

[0043] Figure 3 It is a coal seam hole wall fissure marking diagram of the present invention;

[0044] Figure 4 is a data processing flow chart of the present invention;

[0045] Figure 5 Flowchart of data training of the present invention. DETAILED DESCRIPTION

[0046] With the increasing development of convolutional neural networks (CNN), computers have shown great potential in image recognition. By manually collecting high-resolution images of borehole wall cracks, annotating the cracks on the images, and then using computers to clean and enhance the data, the processed images are trained, so that the computer has the function of identifying borehole wall cracks. Through the use of convolutional neural network technology, subtle and difficult-to-detect cracks can be detected more easily and accurately, so that the staff can analyze the data results more accurately. This information plays an indispensable role in the selection of ore excavation locations, the layout of mine tunnels, and safety management. In short, the capture of borehole wall cracks based on YOLOV5 can ensure the efficiency of mine production and the safety of underground workers to a certain extent. This is also the combination of mining engineering and convolutional neural networks. The mutual promotion of the two has achieved the sustainable development of each other's industry. . At present, the capture of coal seam hole wall cracks based on YOLOV5 is achieved by manually collecting high-resolution hole wall crack images, then cleaning and processing the collected images, and then building a convolutional neural network model to realize the training and testing of the data set, so as to finally enable the computer to have the ability to accurately capture coal seam hole wall cracks.

[0047] The present invention provides a coal seam hole wall crack capture method based on the YOLO algorithm, such as Figure 1 As shown, the following steps are included:

[0048] Step 1: Data preparation. The specific steps are as follows:

[0049] (1) Point selection and drilling: Mine staff carefully select representative locations in the coal seam as test points based on the mine's geological structure, mining plan and other factors. Drilling operations are performed at the selected points using a drill rig with appropriate power and precision. During the drilling process, the drilling angle, depth and other parameters are strictly controlled to ensure the integrity and stability of the hole wall.

[0050] (2) Borehole wall survey and image acquisition: A special measuring rod equipped with a high-resolution camera is slowly inserted into the drilled hole to conduct a comprehensive survey of the borehole wall at a stable and uniform speed. During the survey, the camera automatically collects borehole wall images at fixed time intervals or according to a predetermined shooting strategy to ensure that the entire borehole wall surface is covered and no areas where cracks may exist are missed. The collected images are transmitted in real time and stored in a large-capacity, high-reliability storage device for subsequent processing.

[0051] Step 2: Data processing, that is, processing the collected images. The functional flow of data processing is as follows: Figure 4 The specific steps are as follows:

[0052] (1) Image screening: The staff will check the large number of borehole wall images collected from the storage device one by one, and select those that clearly contain cracks and have clear images without obvious interference factors (such as severe shadows, blurred areas, etc.).

[0053] (2) Image cropping (if necessary): In cases where some cracks occupy a small proportion of the image and are difficult to observe and process, use professional image processing software (such as Photoshop, etc.) to crop the image. When cropping, take the crack as the center, adjust the size and shape of the cropped area appropriately, remove irrelevant background information around it, and make the crack occupy a more prominent position in the cropped image to facilitate subsequent analysis and processing. The cropped image is saved with a new file name, and a backup of the original image is retained for comparison and reference when necessary.

[0054] (3) Binarization and normalization processing

[0055] Binarization: Use the binarization function in the image processing algorithm library (such as OpenCV, etc.) to convert the color or grayscale hole wall image into a black and white binary image. By setting a suitable threshold (determined according to factors such as the brightness distribution of the image, the contrast between the crack and the background, etc.), the part with a pixel value greater than the threshold is determined as the crack part and assigned a value of white (for example, the pixel value is 255); the part with a pixel value less than or equal to the threshold is determined as the background part and assigned a value of black (the pixel value is 0). The processed image contains only two colors, which greatly simplifies the image information, highlights the outline and shape of the crack, and effectively avoids the interference of factors such as uneven illumination intensity on crack identification.

[0056] Normalization: Normalize the binary image and map the value of each pixel in the image to a decimal range between 0 and 1. The specific calculation method is: for the original value x of each pixel, use the formula The conversion is performed, where min is the minimum value of all pixel values ​​in the image and max is the maximum value. In the actual calculation process, each pixel in the image is traversed to accurately obtain the min and max values, and then each pixel is normalized according to the formula to ensure that the processed image data is within a unified numerical range, which is convenient for subsequent model training and calculation.

[0057] (4) Crack marking

[0058] Enter the directory where the image is saved and start the labelimg software. In the software interface, first change the label format to voc format to meet the requirements of model training.

[0059] The staff carefully observed each picture and used the mouse to select the location range of the cracks in the coal seam hole wall. For complex shapes or multiple cracks, multiple rectangular boxes were accurately drawn for annotation. During the annotation process, each crack was labeled with a clear and accurate name for subsequent identification and classification, and saved in a designated folder.

[0060] (5) Dataset division and format conversion

[0061] The labeled image dataset is divided into training set, validation set and test set at a ratio of 60%:20%:20%. In the division process, the diversity of images (such as the distribution of images of different locations and types of cracks) should be fully considered to ensure that the data in each subset is representative and avoid data bias from having adverse effects on model training and evaluation.

[0062] Use the script or tool you wrote to convert the VOC format labels to the YOLO format. During the conversion process, strictly follow the YOLO format specifications to accurately adjust the coordinate information, label category, etc. The converted data sets are saved in different folders with clear and unambiguous names to facilitate quick and accurate call of the corresponding data in the subsequent model training and testing process.

[0063] Step 3: Model training, such as Figure 5 As shown, the YOLOV5 model is used to train the image dataset.

[0064] The specific steps of model training are as follows:

[0065] (1) Model architecture and initialization: The YOLOV5 model is used for training. The model is based on the CNN convolutional neural network architecture and contains multiple convolutional layers, pooling layers, and fully connected layers. The convolutional layer is used to extract image features. It scans the image with convolution kernels of different sizes to capture crack feature information of different scales. The pooling layer is used to reduce the data dimension and the amount of calculation while maintaining the main information of the features. The fully connected layer integrates and maps the extracted features and finally outputs the crack category and location prediction results. When the model is initialized, random initial values ​​are assigned to the weights and bias parameters of each layer to provide a starting point for subsequent training.

[0066] (2) Loading training and validation sets: Load the training and validation set images from the divided dataset into the model training environment. During the loading process, the image data is divided into batches, and each batch contains a certain number of images (such as 32, 64, etc., determined according to hardware resources and dataset size). At the same time, the corresponding annotation information of the image (such as the location and category of the crack) is loaded together to ensure that the model can obtain correct verification information during the training process.

[0067] (3) Model training iteration process

[0068] a. Forward propagation: In each iteration, the forward propagation process is performed first. A small batch of training set image data is input into the model. After a series of operations of convolutional layer, pooling layer and fully connected layer, the model outputs the prediction results of the cracks in each image, including the location of the cracks (such as the coordinates of the bounding box) and the category probability.

[0069] b. Loss calculation: The cross entropy loss function is used to calculate the difference between the predicted result and the actual label.

[0070] Assume that the true category of sample i is y i (y i is a vector indicating that the sample belongs to a certain category), and the category probability distribution predicted by the model is ( represents the probability that sample i is predicted as category j), then the cross entropy loss function L ce for: Where C is the total number of categories.

[0071] Position loss function (for example, using mean square error for position prediction):

[0072] Assume that the predicted bounding box coordinates are The coordinates of the true bounding box are (x, y, w, h), then the position mean square error loss L loc for:

[0073]

[0074] Total loss function (weighted sum of position loss and category loss): Let λ loc is the weight of the position loss, λ ce is the weight of the category loss, then the total loss function L is:

[0075] L=λ loc L loc +λ ce L ce

[0076] c. Back propagation and weight update: Based on the calculated loss value, the back propagation algorithm is used to calculate the gradient of the loss function to the weights of each layer of the model. Starting from the output layer, the gradient is back propagated along the network structure, and the weight parameters of each layer are adjusted according to the gradient information. During the weight update process, the weight is updated using the gradient descent algorithm. By continuously adjusting the weights, the model gradually learns the feature representation that can accurately identify the cracks in the coal seam pore wall, improving the accuracy and verification rate of the prediction.

[0077] d. Verification and convergence judgment: After each training batch, the model is verified using the validation set. The changing trend of the loss function during the training process is observed. If the value of the loss function gradually decreases and tends to be stable (i.e. converges), it is considered that this training has achieved good results and the training can be terminated; if the loss function has not converged, the next iteration is continued, and the above forward propagation, loss calculation, back propagation and weight update process are repeated until the loss function converges or the predetermined number of training rounds (epochs) is reached.

[0078] Step 4: Model testing, apply the newly generated weight file, and put the independent sample set that did not participate in the training into the model for testing. The specific steps are as follows:

[0079] (1) Test set preparation and loading: Take out an independent sample set that does not participate in training (i.e., test set) from the dataset to ensure that the test set is completely independent of the training set and validation set, and there is no data overlap.

[0080] (2) Model prediction and result evaluation: The test set images are input into the trained model batch by batch. The model performs forward propagation and outputs the prediction results of cracks in each image. For each prediction result, it is compared with the corresponding real annotation information and the evaluation indicators such as precision, recall, mean average accuracy (mAP) are calculated. Precision indicates the ratio of the number of samples predicted by the model as positive examples (i.e., cracks exist) and actually positive examples to the number of samples predicted by the model as positive examples. The calculation formula is:

[0081]

[0082] TP (True Positive) represents true positive examples, that is, the number of samples predicted by the model as positive examples (there are cracks) and are actually positive examples; FP (False Positive) represents false positive examples, that is, the number of samples predicted by the model as positive examples but are actually negative examples (there are no cracks);

[0083] The recall rate indicates the ratio of the number of samples predicted by the model as positive and actually positive to the number of samples that are actually positive. The calculation formula is:

[0084]

[0085] Among them, FN (False Negative) represents false negative examples, that is, the number of samples that the model predicts to be negative examples but are actually positive examples;

[0086] The average accuracy rate comprehensively considers the accuracy rates of different categories and different confidence thresholds. The relevant calculation formula is:

[0087] First, for each category C, calculate the precision P(r) under different recall rates r.

[0088] Then calculate the average accuracy AP for each category c , the formula is:

[0089] AP c =∑(p(r)×Δr)

[0090] Δr is the step size of the recall rate change.

[0091] (3) Effect judgment and decision-making: The effect of the model on the test set is judged based on the calculated evaluation indicators. If the model's precision, recall rate and other indicators meet the expected requirements, it indicates that the model has good performance and can accurately capture the cracks in the coal seam bore wall. The next step can be taken to apply the model to new coal seam bore wall images. If the model effect is not good, such as low precision or recall rate does not meet the requirements, the model training process needs to be reviewed. It may be necessary to adjust the model's hyperparameters (such as learning rate, number of iterations, etc.), increase the amount of training data, optimize the data preprocessing method, etc., and then retrain and appropriately increase the number of training times and test again until the model performance meets the actual application requirements.

[0092] Step 5: Model application: Apply the trained model to the new coal seam hole wall crack image to obtain the crack detection results. The specific steps of model application are as follows:

[0093] (1) New image acquisition and preparation: In actual mine operations, when new coal seam borehole walls need to be inspected for cracks, new borehole wall images are acquired according to the method in step 1. Ensure that the image acquisition quality and resolution meet the requirements. After acquisition, perform data preprocessing operations similar to step 2 on the image, including necessary cropping, binarization, and normalization, so that the image data format and features are consistent with those when the model was trained.

[0094] (2) Crack detection and result output: The preprocessed new coal seam borehole wall image is input into the trained YOLOV5 model. The model analyzes and predicts the image and outputs the location (expressed in bounding box coordinates) and category information of the crack. The detection results are visualized in an intuitive and clear way, such as drawing the bounding box of the crack on the image and annotating the category name and confidence score (indicating the model's confidence in the prediction result).

Claims

1. A method for capturing cracks in coal seam pore walls based on the YOLO algorithm, characterized in that: It includes the following steps: Step 1: Collect high-precision coal seam hole wall crack images based on manual exploration of the hole wall; Step 2: Enhance the crack features of the hole wall by preprocessing the collected images; Step 3: Manually mark the cracks in the preprocessed image dataset and determine the location range of the cracks; Step 4: Model training, use the YOLOV5 model to train the image dataset; Step 5: Model testing: put the independent sample set that did not participate in the training into the model for testing, check the accuracy and loss value indicators, and ensure the model effect. If the independent sample that participated in the training failed to obtain the complete effect, continue to step 4 training and increase the number of training times. Otherwise, proceed to the next step; Step 6: Model application: Apply the trained model to the new coal seam hole wall image to obtain the crack detection results.

2. A coal seam hole wall crack capture method based on YOLO algorithm as claimed in claim 1, characterized in that: The specific operation of step 1 is: the mine staff first drills a hole at the location to be tested with a drill rig, and manually uses a spying device to spy on the hole. The spying data is saved and cropped into several hole wall crack images, from which high-resolution crack images are manually selected.

3. A coal seam hole wall crack capture method based on YOLO algorithm as claimed in claim 1, characterized in that: The specific operations of step 2 are: cropping, binarization, and normalization of the collected images.

4. The method for capturing cracks in coal seam pore walls based on the YOLO algorithm according to claim 1, characterized in that: The specific operation of step 3 is: use labelimg software to manually mark the cracks in the coal seam pore wall in the image.

5. The method for capturing cracks in coal seam pore walls based on the YOLO algorithm according to claim 1, characterized in that: The specific operation of step 4 is: use the YOLOV5 model for training. The model is based on the CNN convolutional neural network architecture and uses ReLU() as the activation function. Its function expression is: f(x) = max(0, x), which means that when the input x is greater than 0, the output is equal to the input; when the input x is less than or equal to 0, the output is 0. In principle, it introduces nonlinear factors to the neural network by simply truncating the negative part to 0 and retaining the positive part; And combined with the gradient descent algorithm, the formula is: W new is the updated weight, W old is the current weight α is the learning rate, It is the gradient of the loss function with respect to the weight, and then the weight is updated through back propagation to optimize the loss function; and during the training process, 60% is randomly selected as the training set, 20% as the validation set, and 20% as the test set; The specific process of training is: a. Split the training set and validation set images into batches, train the network with a small batch of data each iteration and then validate the network; b. Use the cross entropy loss function to calculate the difference between the predicted result and the actual label; The cross entropy loss function formula is: The true label is y and the predicted probability is The smaller the value of the cross entropy loss function, the more accurate the model's prediction; c. Use the back-propagation algorithm to update the network weights to minimize the loss and improve the accuracy and verification rate; d. Iterate the process until the loss function converges and the training is completed. If the loss function still does not converge, continue to iterate until the loss function converges.

6. A method for capturing cracks in coal seam pore walls based on the YOLO algorithm as claimed in claim 1, characterized in that: The specific operation of step 5 is: put the randomly selected 20% test set into the model for testing, propagate forward through the model, and obtain the final crack identification result; if the effect is not good, retrain and increase the number of training times, otherwise, proceed to step 6.

Images