Coal rock fracture image recognition method and system based on deep learning and post-processing

Abstract

The invention discloses a coal rock fracture image recognition method and system based on deep learning and post-processing, a coal rock fracture image recognition model is adopted to recognize and classify spliced coal rock fracture images, the model comprises an improved YOLOv5 target detection network, an image segmentation network and a classifier which are connected in sequence, after the image segmentation network outputs the binary image, parallax depth calculation is carried out on cracks and non-cracks in the crack images at different angles, and the cracks are determined based on the parallax depth; after the fracture is determined, carrying out image feature numeralization on the binarized image, and extracting fracture image feature parameters; the fractal dimension of the fracture is calculated, whether the fracture can be represented or not is preliminarily judged based on the fractal dimension, if yes, the fracture image feature parameters are input into a classifier, the type of the coal rock fracture is output, and if not, the specific type of the fracture is output. And the parallax depth, the fractal dimension and the image features are integrated for recognition and classification, so that the problems of misrecognition and missed recognition are effectively reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal-rock fracture recognition, and in particular relates to a coal-rock fracture image recognition method and system based on deep learning and post-processing. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] The evolution of the distribution and structure of cracks in coal rock is related to the safe mining and efficient utilization of coal resources. The development of cracks will lead to anisotropy of coal seam permeability, affect the gas seepage advantage of coal seams, and affect the production efficiency of gas extraction and coalbed methane wells. In addition, the development of coal rock cracks will lead to a decrease in the bearing capacity of the coal rock mass itself, resulting in a decrease in the stability of the coal rock mass and causing coal mine safety accidents. Rapid positioning, identification and classification of coal rock cracks are of great significance for roadway support and gas extraction.

[0004] With the rapid development of deep learning technology and digital image processing technology, image recognition algorithms and classification algorithms have been widely used in the recognition and classification of coal rock fractures. However, the existing image recognition algorithms have low recognition accuracy when facing low light, noise, blur, and complex fracture types in coal mines. Especially when dealing with coal rock fractures with complex shapes and different types, misidentification and missed recognition are prone to occur. In addition, most of the existing target recognition models are constructed using a single algorithm, which makes the precision and accuracy of identifying multiple types of fractures low. At the same time, in the process of coal mining, fractures need to be detected in real time and accurately, which puts higher requirements on image preprocessing and recognition algorithms. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a coal rock fracture image recognition method and system based on deep learning and post-processing, which combines the target detection algorithm with the image segmentation algorithm, and performs post-processing after the fracture image is segmented. It integrates multiple information, namely parallax depth, fractal dimension and image features to perform fracture identification and classification, which can effectively reduce the problems of misidentification and missed identification, and improve the accuracy of fracture identification and classification in complex environments.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a coal rock fracture image recognition method based on deep learning and post-processing, comprising: Acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles; The pre-processed coal-rock fracture images at different angles are spliced ​​to obtain spliced ​​coal-rock fracture images, and the spliced ​​coal-rock fracture images are input into a trained coal-rock fracture image recognition model for fracture recognition and classification to obtain fracture classification results; the coal-rock fracture image recognition model includes an improved YOLOv5 target detection network, an image segmentation network and a classifier connected in sequence; Specifically, the image processing steps in the coal rock fracture image recognition model are as follows: input the spliced ​​coal rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; input the fracture annotation image into the image segmentation network for image segmentation, and output the segmented binary image; perform disparity depth calculation on the fractures and non-fractures in the pre-processed coal rock fracture images at different angles, and determine the fractures based on the disparity depth; after determining the fractures, perform image feature digitization on the binary image, and extract fracture image feature parameters; calculate the fractal dimension of the fracture based on the binary image, and preliminarily judge whether the fracture can be characterized based on the fractal dimension. If so, input the fracture image feature parameters into the classifier, and output the type of coal rock fracture; otherwise, output the specific type of fracture. Further technical solution, the preprocessing process of the coal rock fracture image is specifically as follows: Adjust the brightness, contrast and grayscale of the image; Enhance image details and perform the first global noise reduction; Perform threshold segmentation on the image; Use Gaussian filtering to perform secondary noise reduction on cracks and stubborn noise points in the image; Finally, the piecewise linear transformation method is used to enhance the contrast of the coal rock fracture image after noise reduction to highlight the fracture characteristics.

[0007] A further technical solution is that the improved YOLOv5 target detection network introduces a deformable convolutional network in its feature extraction network, and adopts a weighted bidirectional feature pyramid network to replace the feature pyramid network in its feature fusion network.

[0008] A further technical solution is that in the variable convolution operator of the deformable convolution network, by adding a learnable offset parameter to each element in the convolution kernel, the original fixed convolution kernel has the ability to adapt to the shape of the object. The formula is as follows:

[0009] in, Indicates at location The response on the feature map, Represents the target position, It is represented as the position index of an element in the convolution kernel, R is represented as the area covered by the convolution kernel, W is represented as the weight, and X is represented as the input feature map. Indicates the offset.

[0010] A further technical solution is to calculate the disparity depth of the cracks and non-cracks in the pre-processed coal-rock crack images at different angles, specifically: find matching feature points in the coal-rock crack images at different angles, calculate the disparity on the image plane for the matching feature points, and calculate the disparity depth of the feature points using the disparity, the internal parameters of the camera, and the external parameters. The formula is as follows:

[0011] in, Expressed as parallax depth, Expressed as the baseline distance, Expressed as focal length, Expressed as parallax.

[0012] A further technical solution is that the fractal dimension is calculated using a box counting method, specifically: boxes of different sizes are selected to cover the crack network of the binary image, the number of boxes and the corresponding box side lengths are plotted in a double logarithmic coordinate system, and the plotted points are linearly fitted to obtain the fractal dimension of the crack.

[0013] A further technical solution is to preliminarily judge whether the crack can be characterized based on the fractal dimension: When the fractal dimension is greater than the set threshold, it is judged as no, the specific type of crack is output, and the structural features of the specific type of crack are extracted; When the fractal dimension is within the set range, it is judged as yes, and the fracture image feature parameters are input into the classifier, the classifier outputs the type of coal rock fracture, and extracts the structural features of the corresponding type.

[0014] In a second aspect, the present invention provides a coal-rock fracture image recognition system based on deep learning and post-processing, comprising: An image acquisition module is configured to: acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles; A fracture recognition and classification module is configured to: splice the pre-processed coal-rock fracture images at different angles to obtain a spliced ​​coal-rock fracture image, and input the spliced ​​coal-rock fracture image into a trained coal-rock fracture image recognition model to perform fracture recognition and classification to obtain a fracture classification result; the coal-rock fracture image recognition model includes an improved YOLOv5 target detection network, an image segmentation network, and a classifier connected in sequence; Specifically, the image processing steps in the coal rock fracture image recognition model are as follows: input the spliced ​​coal rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; input the fracture annotation image into the image segmentation network for image segmentation, and output the segmented binary image; perform disparity depth calculation on the fractures and non-fractures in the pre-processed coal rock fracture images at different angles, and determine the fractures based on the disparity depth; after determining the fractures, perform image feature digitization on the binary image, and extract fracture image feature parameters; calculate the fractal dimension of the fracture based on the binary image, and preliminarily judge whether the fracture can be characterized based on the fractal dimension. If so, input the fracture image feature parameters into the classifier, and output the type of coal rock fracture; otherwise, output the specific type of fracture.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the coal rock fracture image recognition method based on deep learning and post-processing as described in the first aspect.

[0016] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for recognizing coal and rock fracture images based on deep learning and post-processing as described in the first aspect are implemented.

[0017] One or more of the above technical solutions have the following beneficial effects: The present invention obtains coal rock fracture images from different angles and stitches them together, which can comprehensively present the geometric shape and distribution of the fractures, improve the processing capability of the coal rock fracture image recognition model for fractures, and thus improve the accuracy of fracture recognition and classification; an improved YOLOv5 target detection network is introduced into the coal rock fracture image recognition model for fracture boundary positioning, and then the fracture area is accurately segmented in combination with the image segmentation network, which helps to accurately extract the boundaries and regional features of the fractures and improve the accuracy of fracture classification.

[0018] Before classifying the identified cracks, the present invention designs two innovative judgment steps. First, the crack depth is obtained by calculating the parallax depth, and the parallax depth between the crack and the non-crack part is used to judge whether the identified object is a crack rather than other objects such as stains. Secondly, after it is determined to be a crack, the fractal dimension of the crack is used to analyze the complexity of the crack, and the characterization of the crack is judged, which effectively distinguishes complex cracks from simple cracks, avoids unnecessary further analysis of overly complex cracks, and inputs the crack image feature parameters that can be characterized into the classifier for classification, thereby further improving the accuracy and reliability of the classifier.

[0019] The coal rock fracture image recognition model in the present invention integrates the advantages of multiple deep learning algorithms and combines the target detection algorithm with the image segmentation algorithm. The target detection algorithm can effectively locate the boundaries of coal rock fractures and improve the existing YOLOv5 target detection network to provide preliminary positioning results for subsequent fine segmentation. Image segmentation is good at capturing the details of the fracture edge and uses the SegNet image segmentation algorithm to complete more refined coal rock fracture extraction tasks, thereby improving the accuracy of fracture classification.

[0020] The real-time wireless monitoring system for coal and rock fractures based on the coal and rock fracture image recognition model has the characteristics of high accuracy, good robustness and fast calculation speed, which helps to realize the intelligent identification, recording and sorting of coal and rock fractures and improve production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0022] Figure 1 is a flow chart of a method for identifying coal rock fracture images according to an embodiment of the present invention; Figure 2 are the linear cracks and their related parameters in the embodiment of the present invention; Figure 3 It is a structure diagram of the target detection network and image segmentation network of the model in the embodiment of the present invention; Figure 4 It is a structural schematic diagram of a real-time wireless monitoring system for coal-rock fissures based on a coal-rock fissure image recognition model in an embodiment of the present invention. DETAILED DESCRIPTION

[0023] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0024] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0025] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0026] Embodiment 1 like Figure 1 As shown, this embodiment discloses a coal rock fracture image recognition method based on deep learning and post-processing, and the method includes the following steps: S1: Acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles.

[0027] In this embodiment, two cameras are used to take pictures of coal and rock, and two images of the same object are collected at different angles. The two cameras are arranged on the same plane in front of the coal and rock to be photographed, and the images are collected at different angles. The angle difference of the camera optical axis is preferably between 25° and 35°.

[0028] A coal-rock fracture image database containing more than 300 images was established. The underground light is insufficient, the coal-rock body itself is dark in color, and the coal-rock fracture types are complex and have different shapes. In addition, the fracture morphology and occurrence characteristics in the image are not complete. Therefore, professional knowledge is required to identify coal-rock images, select images with typical fracture characteristics for training, and ensure the accuracy of the coal-rock fracture recognition model in distinguishing fracture characteristics through image analysis. After the images are screened, they are preprocessed.

[0029] The specific preprocessing process of coal rock fracture images is as follows: S101: adjust the brightness, contrast and grayscale of the image, increase the brightness of the image, and preliminarily remove useless color information of the image to make the cracks more obvious; S102: Enhance image details and improve image resolution through Topaz Gigapixel AI; S103: Use Topaz DeNoise AI software based on the principle of convolutional neural network to perform global noise reduction on the image, reduce noise points, and improve the clarity of cracks; S104: Perform threshold segmentation processing on the image to filter out all cracks and a small part of stubborn noise points; the threshold segmentation principle formula is as follows: (1) in, It's pixels The adaptive threshold is the local neighborhood window, C is the constant adjustment value, and are the height and width of the image, is the coordinate of a pixel on the image. is the grayscale value.

[0030] S105: performing secondary noise reduction on cracks and stubborn noise points in the image using Gaussian filtering method; S106: The crack image after noise reduction tends to be smooth, so the piecewise linear transformation method is finally used to enhance the contrast of the coal rock crack image after noise reduction to highlight the characteristics of the crack area. The piecewise linear transformation formula is as follows: (2) in, is the pixel value of the original image, is the pixel value of the image after piecewise linear transformation, a and b are the pixel nodes of the original image, c and d are the pixel nodes of the image after piecewise linear transformation, , are the maximum pixel values ​​of the original image and the image after piecewise linear transformation, respectively. Labelimg is used to annotate cracks in preprocessed images.

[0031] S2: The pre-processed coal rock fracture images at different angles are stitched together to obtain a stitched coal rock fracture image, and the stitched coal rock fracture image is input into a trained coal rock fracture image recognition model to identify and classify the fractures to obtain a fracture recognition result.

[0032] In this embodiment, two images of the same photographed coal rock collected at different angles are spliced ​​to obtain a spliced ​​coal rock fracture image, and the spliced ​​coal rock fracture image is input into a coal rock fracture image recognition model for fracture recognition and classification. The coal rock fracture image recognition model includes a target detection network, an image segmentation network, and a classifier connected in sequence. The target detection network is an improved YOLOv5 target detection network, and the image segmentation network is a SegNet network. It can be seen that the coal rock fracture image recognition model framework is an integration of a target detection algorithm and an image segmentation algorithm.

[0033] S201: Input the spliced ​​coal rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; that is, the detected fracture boundary box is superimposed on the spliced ​​original image, so as to mark the specific position and range of the fracture.

[0034] The boundaries of coal and rock fractures are located based on the improved YOLOv5 target detection network. The feature extraction effect of coal and rock fractures is enhanced by reducing the segmentation targets, providing preliminary positioning results for subsequent fine segmentation.

[0035] The YOLOv5 target detection network includes an input layer Input, a feature extraction network Backbone, a feature fusion network Neck, and a detection head YOLO Head, which are connected in sequence. The input layer Input part inputs the fractured coal rock image with fracture development characteristics into the image recognition model; the feature extraction network Backbone part adopts the Focus and C3 structures. Coal rock fracture images generally have dense coal rocks, complex backgrounds, irregular shapes of detection objects, and small targets. For the recognition task of coal rock fracture images, the YOLOv5 network is improved to obtain an improved YOLOv5 target detection network. Specifically: 1) In the original YOLOv5 model, the feature extraction network Backbone is mainly composed of traditional convolution layers, BN layers and activation functions. When extracting features, since the size mutations and contours of coal and rock fractures are generally different, it is difficult for a fixed-size convolution kernel to accurately extract the boundary information of the object, thus affecting the model's ability to extract fracture features. Therefore, a deformable convolution network is introduced in Backbone. In the variable convolution operator, by adding a learnable offset parameter offset to each element in the convolution kernel, the original fixed convolution kernel has the ability to adapt to the shape of the object. The calculation formula is as follows: (3) in, Indicates at location The response on the feature map, Represents the target position, It is represented as the position index of an element in the convolution kernel, R is represented as the area covered by the convolution kernel, W is represented as the weight, and X is represented as the input feature map. Represents the offset. Since the position of the variable convolution sampling is irregular, the offset is generally a decimal, so formula (4) is implemented using bilinear interpolation as follows: (4) Among them, X represents the input feature map, represents any position in the area corresponding to the deformable convolution, It is a feature map The pixel value corresponding to a sampling point is Represents a two-dimensional bilinear interpolation kernel.

[0036] 2) For crack identification in a dense and complex background of coal and rock, how to effectively combine and process multi-scale features and fuse them is a key problem. The size of coal and rock cracks is small, usually accounting for less than 10% of the original image. The original YOLOv5 uses the FPN+PAN structure in the feature fusion network Neck to sample feature images of different sizes and process the feature images into the same size. However, different input features have different resolutions, so when performing up and down sampling and tensor splicing, the weights of the output fusion features are different. This requires optimizing the YOLOv5 network architecture and introducing learnable weights to distinguish the importance of different input features, thereby enhancing the learning ability for small targets.

[0037] The weighted bidirectional feature pyramid network (BiFPN) is used to replace the feature pyramid network FPN to adapt to the backbone network of small target detection, and efficient weighted feature fusion is achieved by using bidirectional cross connection and fast normalization. The fast normalization fusion is shown in formula (5): (5) in, It is represented as the output feature after fusion, and E represents a very small additional value (in order to keep numerical stability), Represented as all feature maps involved in fusion, Indicates the input A feature map. , represents the total number of feature maps, Represents the weight of the input feature, and the activation function makes , the general weight is in the range of 0 to 1, and the calculation formula of a feature layer of BiFPN is shown in formula (6) and formula (7): (6) (7) in, Indicates the top-down path The intermediate features of the layer, Indicates bottom-up The output features of the layer, Represented as the top-down path The input features of the layer, Resize() means adjusting the spatial size of the feature map, Represented as the top-down path The input features of the layer, is a very small additional value used to avoid the denominator being 0. In the feature fusion stage, the model uses depthwise separable convolutions and adds batch normalization and activation functions after each convolution to fuse multi-scale features and improve computational efficiency.

[0038] 3) In target detection, the boundary loss function used by the YOLOv5 network in the detection head is IoU, which has limitations in identifying irregularly shaped coal and rock cracks. The improved YOLOv5 target detection network uses Inner-CIoU loss to judge the target boundary. During the model training process, when dealing with high IoU coal and rock cracks, more attention is paid to fine-tuning the position of the prediction box to improve the positioning accuracy. When dealing with low IoU coal and rock cracks, a larger scale auxiliary box is used to effectively cover the target area and improve the generalization ability of the model. In addition, Inner-CIoU focuses on the overlapping part inside the bounding box, improves the accuracy of bounding box regression in target detection, and can improve the detection accuracy and efficiency of occluded or overlapping coal and rock targets. The Inner-CIoU formula is as follows: (8) in, for Loss function, for value, Represents the intersection-over-union ratio, ranging from 0 to 1.

[0039] It should be noted that, due to the above improvements, the network's coal and rock fracture image positioning capability can be enhanced with a small increase in the amount of calculation.

[0040] For images where coal-rock fractures are detected, the improved YOLOv5 target detection network outputs the fracture bounding box range, i.e., the fracture annotation image, and crops the image before proceeding to the next step. The content of the bounding box includes the offset, width, and height of the center point relative to the upper left corner of the grid unit. For images where coal-rock fractures are not detected, the number of images where fractures are not detected is calculated and the information is stored.

[0041] S202: Inputting the crack-annotated image into an image segmentation network for image segmentation, and outputting a segmented binary image; The semantic information of the coal-rock fracture image after detection and cropping by the target detection network is relatively simple, and the edge details of the fracture are difficult to capture. The high-level semantics of the image are missing, and the structure of the fracture cannot be further studied. Therefore, the SegNet image segmentation algorithm is used to complete a more refined coal-rock fracture extraction task. SegNet is a fully convolutional neural network based on convolutional neural network (CNN). Compared with other image segmentation networks, convolutional neural networks have higher accuracy and flexibility.

[0042] The image segmentation network, namely SegNet, consists of two parts: an encoder and a decoder. The encoder is based on the VGG-16 network and contains several convolutional layers and maximum pooling layers. Each convolutional layer performs a two-dimensional convolution operation to extract local features of the image, which is used to extract the deep features of the input coal and rock images. The decoder restores the spatial resolution of the image through a step-by-step upsampling operation and combines the feature map output by the encoder to achieve pixel-by-pixel segmentation. The specific steps are: 1) Input the object detection network to locate the cropped image.

[0043] 2) Through the convolution and pooling operations of the encoder, the deep features of the coal rock image are extracted, the image size is gradually reduced, and key information such as cracks and interfaces in the coal rock is captured. The convolution layer is used to extract local features of the coal rock image. The convolution operation formula is as follows: (9) in, Denoted as the input image, Represented as the convolution kernel, is the integration variable, Represents convolution.

[0044] The maximum pooling layer is used to reduce the spatial dimension of the feature map and retain important features. The size of the pooling window is 2×2, and the formula for the maximum pooling operation is: (10) in, is the output after pooling, is the pixel value in the input feature map.

[0045] 3) The decoder uses nonlinear upsampling to restore the spatial resolution of the image. SegNet records the pooling index and uses these indexes to achieve accurate de-pooling in the decoding stage. The formula is as follows: (11) in, is the pixel value after upsampling, is the maximum value retained by the pooling operation in the input feature map, is the maximum position index recorded during the maximum pooling, Represented as the coordinates of an element in the input feature map.

[0046] 4) Output the segmented binary image, in which each pixel is marked as crack and background. This can ultimately confirm whether the coal rock has cracks and detect the exact location and structure of the cracks.

[0047] S203: performing parallax depth calculation on the cracks and non-cracks in the pre-processed coal-rock crack images at different angles, and determining the cracks based on the parallax depths; after determining the cracks, performing image feature digitization on the binary images, and extracting crack image feature parameters; S20301: Perform parallax depth calculation on the fracture and non-fracture parts of the pre-processed coal rock fracture images at different angles.

[0048] Specifically, matching feature points are found in the images of two perspectives (i.e., coal rock fracture images at different angles). For this pair of matching feature points, their lateral directions (i.e., parallax) on the image plane are calculated. The parallax depth of the feature points is calculated using the parallax, the camera's internal parameters (focal length, principal point, etc.), and the external parameters (camera position and direction). The formula is as follows: (12) in, is the parallax depth, is the baseline distance (i.e. the distance between the optical centers of the two cameras), is the focal length, The difference between the parallax depth of the crack and the coal rock surface is the crack depth.

[0049] The present invention can obtain the crack depth by calculating the parallax depth, so that the crack image is post-processed to obtain and record more crack features; it can also detect the accuracy of crack recognition. The parallax depths of the crack part and the non-crack part (i.e., the coal rock surface) are different. Through this step, it can be determined that the identified crack is not other objects such as stains, and the crack can be classified after it is determined.

[0050] S20302: Use a gray-level co-occurrence matrix to digitize the image features of the binary image and extract crack image feature parameters.

[0051] The gray-level co-occurrence matrix is ​​used to digitize the image features of the crack high-level semantics (i.e., binary image) after image segmentation processing as the input data type for crack classification.

[0052] The gray-level co-occurrence matrix is ​​the combination probability between the data gray levels. The matrix composed of .

[0053] in, represents the grayscale direction, represents the gray level, represents the row number of the gray-level co-occurrence matrix, Represents the column number of the gray-level co-occurrence matrix. The gray level is point As the starting point, at a distance of , direction is When the gray value is point probability. The calculation formula is as follows: (13) in, As the starting point, is the point where the gray value appears, Choose from 4 directions: 0°, 45°, 90°, 135°. For distance, .

[0054] Through the above steps, the crack image characteristic parameters, namely contrast, homogeneity and entropy, are extracted.

[0055] S204: Calculate the fractal dimension of the crack based on the binary image, and preliminarily determine whether the crack can be characterized based on the fractal dimension. If so, input the characteristic parameters of the crack image into the classifier and output the type of coal rock crack. Otherwise, output the specific type of crack.

[0056] The box counting method is used to calculate the fractal dimension of the crack. Boxes of different sizes are selected to cover the crack network of the binary image. The number of boxes and the corresponding box side lengths are plotted in a double logarithmic coordinate system. The plotted points are linearly fitted to obtain the fractal dimension of the crack.

[0057] The specific steps of the box counting method are as follows: select a series of "boxes" of different sizes and overlay them on the crack network. For each grid size, count how many grids contain at least part of the crack. Plot the number of boxes and the corresponding box side lengths in a double logarithmic coordinate system. Perform a linear fit on the plotted points, and the slope of the resulting straight line is the fractal dimension of the crack. The fractal dimension is a non-integer quantity that describes how a fractal object fills in space. For the crack network in the image, the fractal dimension is usually between 1 and 2, and its value reflects the density and complexity of the crack network.

[0058] Based on the obtained fractal dimension, it is judged whether the crack can be characterized, specifically: When the fractal dimension of the fracture is greater than the set threshold (the threshold in this embodiment is set to 1.5), the fracture network is very dense, there are many connections between the fractures, and they cover a larger space. The fracture is classified as a broken zone. At this time, further characterization is difficult, and it is judged as no, and the specific type of the fracture, namely the broken zone, is output, and its key structural features are extracted for the broken zone. In this embodiment, for the broken zone, the position of the broken area is determined according to the position of the pixel points of the broken area on the coal and rock image. In order to accurately describe the degree of fragmentation, the structural feature of the fragmentation rate is defined. The fragmentation rate refers to the ratio of the number of broken pixels in the fractured area to the total number of pixels in the area. The breakage rate can be expressed as: (14) Where B represents the breakage rate, is the total number of image pixels, and D is the number of fractured pixels in the fractured area.

[0059] When the crack dimension is within the set range (the range in this embodiment is set to [1, 1.5], that is, when it is greater than 1 and less than 1.5), it can be further characterized, then it is judged as yes, and then the crack image feature parameters are input into the classifier to output the type of coal rock cracks. The classifier adopts a supervised learning model. The classifier input is the crack image feature parameters, namely contrast, homogeneity and entropy parameters. The classifier outputs the type of coal rock cracks, including two types: curved cracks and linear cracks, and extracts the structural features of the corresponding types. The classifier can use a commonly used supervised machine learning classification model, which is not specifically limited here.

[0060] The coal and rock fracture image recognition model is trained in advance using labeled data and applied to fracture identification and classification tasks.

[0061] For the classified curved cracks and linear cracks, their key structural features are extracted. Specifically, for linear cracks, their position, length and apparent inclination angle are calculated. Figure 2 is a linear crack related parameter, and the linear crack position can be calculated by the following formula: (15) (16) in, L 1 and L 2 represent the starting and ending positions of the cracks, respectively. H is the length of coal rock, and are the starting and ending coordinates of the linear crack, Indicates the number of vertical pixels of the coal rock image.

[0062] The apparent inclination angle can be calculated by the following formula: (17) in, is the coal rock width, and Respectively Dot and The horizontal coordinate of the point.

[0063] True length of linear crack It can be expressed as: (18) in, L 1 and L 2 represent the starting and ending positions of the cracks, respectively. is the apparent inclination angle.

[0064] For curved cracks, calculate their length and curvature (i.e. overall arc). The length can be estimated using the following formula: (19) in, is the slope of a point on the curve, and are the starting and ending coordinates of the crack respectively.

[0065] The overall curvature of the curved fracture It can be calculated by integrating the local radians, as follows: (20) in, is the overall arc, is the slope of a point on the curve, and are the starting and ending coordinates of the crack respectively.

[0066] The present invention determines whether a crack can be characterized based on the fractal dimension of the crack, can make more accurate classification under different complexity levels of the crack, effectively distinguish complex cracks from simple cracks, and avoid unnecessary further analysis of overly complex cracks; it can also reduce the amount of calculation when the classifier performs crack classification and improve the processing efficiency of the model. Especially in the processing of large-scale coal and rock crack images, this judgment and simplification method can effectively save computing resources and time.

[0067] The above S202 and S203 post-process the crack image after positioning and annotating by the improved YOLOv5 target detection network, further determine the identified cracks, and classify them to obtain corresponding crack types; for the corresponding crack types, extract their key structural features and store them.

[0068] The number of images without detected cracks, the types of cracks in detected cracks, and the structural features are stored in the local file system, such as CSV files, JSON files, etc. In order to ensure the security and reliability of coal rock crack data, a data backup mechanism is established. This storage mechanism provides important data support for subsequent crack identification, analysis, and model training.

[0069] Embodiment 2 This embodiment provides a coal-rock fracture image recognition system based on deep learning and post-processing, including: An image acquisition module is configured to: acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles; A fracture recognition and classification module is configured to: splice the pre-processed coal-rock fracture images at different angles to obtain a spliced ​​coal-rock fracture image, and input the spliced ​​coal-rock fracture image into a trained coal-rock fracture image recognition model to perform fracture recognition and classification to obtain a fracture classification result; the coal-rock fracture image recognition model includes an improved YOLOv5 target detection network, an image segmentation network, and a classifier connected in sequence; Specifically, the image processing steps in the coal rock fracture image recognition model are as follows: input the spliced ​​coal rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; input the fracture annotation image into the image segmentation network for image segmentation, and output the segmented binary image; perform disparity depth calculation on the fractures and non-fractures in the pre-processed coal rock fracture images at different angles, and determine the fractures based on the disparity depth; after determining the fractures, perform image feature digitization on the binary image, and extract fracture image feature parameters; calculate the fractal dimension of the fracture based on the binary image, and preliminarily judge whether the fracture can be characterized based on the fractal dimension. If so, input the fracture image feature parameters into the classifier, and output the type of coal rock fracture; otherwise, output the specific type of fracture.

[0070] like Figure 4 As shown, based on the above coal rock fracture image recognition system, a coal rock fracture real-time wireless monitoring system is established. First, the coal rock fracture image recognition model is trained on the server, and then the trained model is deployed on the Nvidia JetsonXavier development board. The camera is used to collect on-site pictures in real time, and the pictures are transmitted to JetsonXavier wirelessly. Jetson Xavier uses the trained coal rock fracture image recognition model to detect whether there are coal rocks with developed fractures in the picture, and classifies the identified fractures. Jetson Xavier sends the classification results to the embedded terminal STM32 through the serial port for fracture data storage and sorting.

[0071] Embodiment 3 The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0072] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0073] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0074] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0075] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0076] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A coal-rock fracture image recognition method based on deep learning and post-processing, characterized in that: include: Acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles; The pre-processed coal-rock fracture images at different angles are spliced ​​to obtain spliced ​​coal-rock fracture images, and the spliced ​​coal-rock fracture images are input into a trained coal-rock fracture image recognition model for fracture recognition and classification to obtain fracture classification results; the coal-rock fracture image recognition model includes an improved YOLOv5 target detection network, an image segmentation network and a classifier connected in sequence; Specifically, the image processing steps in the coal-rock fracture image recognition model are as follows: input the spliced ​​coal-rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; Inputting the crack annotated image into an image segmentation network for image segmentation, and outputting a segmented binary image; performing parallax depth calculation on cracks and non-cracks in the pre-processed coal rock crack images at different angles, and determining the cracks based on the parallax depth; After the cracks are determined, the image features of the binary image are digitized to extract crack image feature parameters; The fractal dimension of the crack is calculated based on the binary image, and it is preliminarily determined whether the crack can be characterized based on the fractal dimension. If so, the characteristic parameters of the crack image are input into the classifier to output the type of coal rock crack. Otherwise, the specific type of crack is output.

2. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 1, characterized in that: The preprocessing process of the coal rock fracture image is specifically as follows: Adjust the brightness, contrast and grayscale of the image; Enhance image details and perform the first global noise reduction; Perform threshold segmentation on the image; Use Gaussian filtering to perform secondary noise reduction on cracks and stubborn noise points in the image; Finally, the piecewise linear transformation method is used to enhance the contrast of the coal rock fracture image after noise reduction to highlight the fracture characteristics.

3. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 1, characterized in that: The improved YOLOv5 target detection network introduces a deformable convolutional network in its feature extraction network. In its feature fusion network, a weighted bidirectional feature pyramid network is used to replace the feature pyramid network.

4. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 3, characterized in that: In the variable convolution operator of the deformable convolutional network, a learnable offset parameter is added to each element in the convolution kernel, so that the original fixed convolution kernel has the ability to adapt to the shape of the object. The formula is as follows: in, Indicates at location The response on the feature map, Represents the target position, It is represented as the position index of an element in the convolution kernel, R is represented as the area covered by the convolution kernel, W is represented as the weight, and X is represented as the input feature map. Indicates the offset.

5. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 1, characterized in that: The disparity depth of the cracks and non-cracks in the preprocessed coal-rock crack images at different angles is calculated, specifically: matching feature points are found in the coal-rock crack images at different angles, the disparity on the image plane is calculated for the matching feature points, and the disparity depth of the feature points is calculated using the disparity, the internal parameters of the camera, and the external parameters. The formula is as follows: in, Expressed as parallax depth, Expressed as the baseline distance, Expressed as focal length, Expressed as parallax.

6. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 1, characterized in that: The fractal dimension is calculated by a box counting method, specifically, by selecting boxes of different sizes to cover the crack network of the binary image, plotting the number of boxes and the corresponding box side lengths in a double logarithmic coordinate system, performing linear fitting on the plotted points, and obtaining the fractal dimension of the crack.

7. The coal-rock fracture image recognition method based on deep learning and post-processing according to claim 1, characterized in that: Based on the fractal dimension, it is preliminarily judged whether the crack can be characterized specifically as follows: When the fractal dimension is greater than the set threshold, it is judged as no, the specific type of crack is output, and the structural features of the specific type of crack are extracted; When the fractal dimension is within the set range, it is judged as yes, and the fracture image feature parameters are input into the classifier, the classifier outputs the type of coal rock fracture, and extracts the structural features of the corresponding type.

8. A coal-rock fracture image recognition system based on deep learning and post-processing, characterized in that: include: An image acquisition module is configured to: acquire coal-rock fracture images at different angles and perform preprocessing to obtain preprocessed coal-rock fracture images at different angles; A fracture recognition and classification module is configured to: splice the pre-processed coal-rock fracture images at different angles to obtain a spliced ​​coal-rock fracture image, and input the spliced ​​coal-rock fracture image into a trained coal-rock fracture image recognition model to perform fracture recognition and classification to obtain a fracture classification result; the coal-rock fracture image recognition model includes an improved YOLOv5 target detection network, an image segmentation network, and a classifier connected in sequence; Specifically, the image processing steps in the coal-rock fracture image recognition model are as follows: input the spliced ​​coal-rock fracture image into the improved YOLOv5 target detection network to locate the fracture boundary, and output the fracture annotation image; Inputting the crack annotated image into an image segmentation network for image segmentation, and outputting a segmented binary image; performing parallax depth calculation on cracks and non-cracks in the pre-processed coal rock crack images at different angles, and determining the cracks based on the parallax depth; After the cracks are determined, the image features of the binary image are digitized to extract crack image feature parameters; The fractal dimension of the crack is calculated based on the binary image, and it is preliminarily determined whether the crack can be characterized based on the fractal dimension. If so, the characteristic parameters of the crack image are input into the classifier to output the type of coal rock crack. Otherwise, the specific type of crack is output.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the coal-rock fracture image recognition method based on deep learning and post-processing as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the coal-rock fracture image recognition method based on deep learning and post-processing as described in any one of claims 1 to 7 are implemented.

Images