Coal gangue X-ray image concave defect detection and segmentation method based on deep learning

Abstract

The invention relates to the technical field of coal and gangue separation, in particular to a coal and gangue X-ray image concave defect detection and segmentation method based on deep learning. According to the deep learning-based coal gangue X-ray image concave defect detection and segmentation method provided by the invention, through an innovative thought of concave defect region labeling, anchor point edge labeling in a traditional method is avoided, and the labeling efficiency is remarkably improved. Besides, by utilizing the strong feature extraction and representation capability of the deep learning model, the method can effectively segment overlapped and adhered targets from the X-ray image, accurately position each target, and provide reliable technical support for subsequent blowing separation of the coal gangue.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of coal gangue sorting, in particular to a coal gangue X-ray image concave defect detection and segmentation method based on deep learning. BACKGROUND

[0002] Coal gangue is a waste rock associated with coal mining, which contains a large amount of impurities and usually needs to be separated by photoelectric sorting technology. With the continuous improvement of automation and intelligent level of coal mining, intelligent photoelectric sorting technology of coal gangue has been widely applied, and X-ray imaging technology as an effective detection means is widely used in the process of coal gangue sorting. However, in practical application, the coal gangue targets in the X-ray image often appear overlapping and sticking phenomenon, which makes the traditional image processing and segmentation method face great challenges.

[0003] Traditional image segmentation methods, such as watershed algorithm and concave point detection and matching segmentation algorithm, are aimed at multi-target overlapping and multi-scale morphology of coal gangue, and the gray difference of the target is weak, which leads to the fact that these methods cannot effectively cope with complex target segmentation tasks. The concave point detection and matching segmentation algorithm is prone to over or under detection of concave points when dealing with multi-target overlapping and sticking, resulting in false segmentation. Although the watershed algorithm can handle target segmentation, it relies on accurate calculation of the extreme value region of each target, which is prone to false segmentation for multi-scale, irregularly shaped coal gangue, especially for complex morphologies such as 8-shaped coal gangue, resulting in unstable results. In summary, these traditional algorithms are also susceptible to image target shape, size and other disturbances, resulting in unsatisfactory segmentation results, and low efficiency in efficiently processing multi-target scenes.

[0004] Image segmentation methods based on deep learning have made significant progress in recent years, but these methods usually rely on edge anchor point labeling for all targets and circle along the edge of the target one by one to label and segment the target. This labeling method requires high labeling personnel and has a large workload, and there is a serious problem of low efficiency, so it needs to be solved urgently. SUMMARY

[0005] In order to avoid and overcome the technical problems existing in the prior art, the present application provides a coal gangue X-ray image concave defect detection and segmentation method based on deep learning. The present application can effectively improve the accuracy of target segmentation.

[0006] To achieve the above purpose, the present application provides the following technical scheme:

[0007] A coal gangue X-ray image concave defect detection and segmentation method based on deep learning, comprising the following steps:

[0008] S1, obtain a coal gangue X-ray image, mark out a concave defect area of an overlapping and adhesion part in the coal gangue X-ray image, and find out a pair of concave defect areas in the coal gangue X-ray image to form a training image;

[0009] S2, train a deep learning model by using the training image to obtain a prediction model capable of predicting a pair of concave defect areas in a coal gangue X-ray image;

[0010] S3, input a coal gangue X-ray image to be detected into the prediction model, and output a to-be-segmented image in which a pair of concave defect areas are marked out;

[0011] S4, calculate a shortest path between a pair of concave defect areas of a same overlapping and adhesion part in the to-be-segmented image, and segment the overlapping and adhesion part by taking the shortest path as a segmentation line.

[0012] As a further scheme of the present application, the marking step of the concave defect area is as follows:

[0013] S11, determine an outline of each target of the overlapping and adhesion part which does not overlap and adhere, and connect the outlines of the overlapping and adhesion part in sequence to form a closed loop;

[0014] S12, draw a convex hull which partially coincides with the closed loop on the outside of the closed loop to constitute a convex hull surrounding the outlines, and form a plurality of areas which are not connected to each other between the closed loop and the convex hull, and the areas constitute concave defect candidate areas, and the concave defect candidate areas are all non-convex hull structures;

[0015] S13, if there is a concave defect area in the concave defect candidate area, the concave defect area is an area with a concave defect area greater than an area threshold value and a concave defect angle less than an angle threshold value.

[0016] As a further scheme of the present application, the obtaining process of the prediction model is as follows:

[0017] S21, the training image adopts a dual-energy X-ray high-energy image or a binary image generated by a background threshold method, the image size of the training image is unified to a fixed resolution, and normalization processing is performed, and the rectangular frame coordinates of the marked pair of concave defects are converted into a target detection format;

[0018] S22, a deep learning model is selected, and a channel and spatial attention mechanism is activated in the deep learning model architecture, and the output layer is set as a single-class detection task, that is, only a pair of concave defect areas are identified;

[0019] S23, the deep learning model is trained by using the training image to obtain a corresponding prediction model.

[0020] As a further scheme of the present application, the obtaining process of the to-be-segmented image is as follows:

[0021] S31, processing the to-be-detected image according to the content of step S21 to generate a preprocessed image;

[0022] S32, inputting the preprocessed image into a prediction model to output a set of paired concave defect region frames;

[0023] S33, applying a non-maximum suppression algorithm, setting an overlap area ratio threshold IoU = 0.5, and retaining the paired concave defect region frame with the highest confidence and non-collision IoU in each adhesion region;

[0024] S34, outputting the to-be-segmented image with labeled frames and coordinate information to a segmentation module.

[0025] As a further scheme of the application, the calculation steps of the shortest path are as follows:

[0026] S4A1, obtaining paired concave defect regions in the current convex hull;

[0027] S4A2, obtaining the two points with the closest Euclidean distance in the paired concave defect regions, and connecting the two points by a straight line, which constitutes the segmentation line of the segmentation target, and the segmentation line is used to generate an independent target mask or contour coordinate.

[0028] As a further scheme of the application, the calculation steps of the shortest path are as follows:

[0029] S4B1, obtaining paired concave defect regions in the current convex hull;

[0030] S4B2, using Dijkstra algorithm or A* algorithm to calculate the shortest path between one concave defect region and another concave defect region, and the shortest path constitutes the segmentation line of the segmentation target.

[0031] As a further scheme of the application, the normalization processing is to scale the pixel value of the image to [0, 1].

[0032] As a further scheme of the application, the deep learning model used is YOLO or Faster R-CNN.

[0033] As a further scheme of the application, the labeling process in step S1 needs to be verified by combining the original X-ray grayscale image and its binary image:

[0034] In the grayscale image, the suspected adhesion region needs to be confirmed whether it is actually adhesion through the binary image;

[0035] When labeling in the binary image, the independent target needs to be excluded by referring to the grayscale image.

[0036] As a further scheme of the application, when there are multiple concave defects in the detection frame:

[0037] The two concave defects with the largest area are preferentially selected for segmentation;

[0038] After segmentation, a connected region analysis is performed, and noise segments with an area less than a preset threshold are removed.

[0039] Compared with the prior art, the present application has the following advantages:

[0040] The present application provides a coal gangue X-ray image concave defect detection and segmentation method based on deep learning. The method avoids the anchor point edge labeling in the traditional method through the innovative idea of concave defect region labeling, significantly improving the labeling efficiency. In addition, by utilizing the powerful feature extraction and representation ability of the deep learning model, the method can effectively segment the overlapping and adhered targets from the X-ray image, accurately locate each target, and provide reliable technical support for the subsequent blowing separation of coal gangue. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 The present application provides a method flowchart.

[0042] Figure 2 The present application provides a labeling method comparison diagram.

[0043] Figure 3 The present application provides a gray-scale image and binary image collaborative labeling flowchart.

[0044] Figure 4 The present application provides a binary image-based adhesion confirmation determination diagram.

[0045] Figure 5 The present application provides a multi-concave defect detection frame screening logic diagram.

[0046] Figure 6 The present application provides a concave defect labeling frame size optimization reference diagram.

[0047] Figure 7 The present application provides a concave defect geometric feature extraction flowchart.

[0048] Figure 8 The present application provides a concave defect angle and area threshold determination method diagram.

[0049] Figure 9 The present application provides a single image labeling misjudgment case analysis diagram.

[0050] Figure 10 The present application provides a concave defect-based segmentation line generation algorithm diagram. DETAILED DESCRIPTION

[0051] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort belong to the scope of protection of the present application.

[0052] I. SUMMARY OF TECHNICAL SOLUTIONS

[0053] The present application provides a coal gangue X-ray image concave defect detection and segmentation method based on deep learning. By combining a deep learning model and a concave defect region labeling method, the limitations of traditional methods in multi-target overlapping and adhering coal gangue segmentation are broken through, including the following main contents.

[0054] 1. Raw image acquisition and preprocessing

[0055] A large number of on-site collected dual-energy X-ray transmission images are obtained, in which the coal gangue particle size range is as large as possible to ensure the generalization of the model. The original dual-energy X-ray transmission image is denoised to remove background noise. Image enhancement techniques such as rotation, scaling, translation, etc. are used to increase the diversity of data and ensure the generalization ability of the model. The image is converted to a uniform input size to ensure the consistency of the subsequent deep learning model input.

[0056] 2. Concave defect region labeling and dataset construction

[0057] The concave defect region in the image is labeled to provide training data for the deep learning model. As shown in Figure 2 , the high-energy image (because the high-energy image is brighter and clearer) or the binary image of the high-energy image of the dual-energy X-ray transmission image is labeled; only the paired concave defects 002 at the overlapping and adhering target are labeled. Training set, validation set and test set are constructed to ensure comprehensive coverage of samples of different adhesion types and different gray scale contrasts. The concave defect region is labeled by a rectangular box 0011, rather than the traditional anchor edge labeling 0021, which significantly improves the labeling efficiency. Based on the labeled data, a training dataset is constructed, including input images and corresponding concave defect region labels.

[0058] In the image labeling process, only 1 label needs to be given, i.e. the model is a 1-classification target detection. The labeled region needs to include at least two background regions, and both background regions are the regions where the concave defects are located, i.e. paired concave defect regions. As shown in Figure 3 , when labeling the X-ray image 003, it needs to be determined whether it is adhered by combining its binary image 004, and there are paired concave defect regions. As shown in Figure 4As shown, when labeling the region of the binarized image 006, it is necessary to determine whether the target is a sticking target in combination with the X-ray image 005. The reason is that it is difficult to determine whether the target is sticking after binarization through a single binarized image 006; for example, the target region of a certain binarized image 008 may be a sticking target or an independent target, but it is observed from the X-ray image that it is more like a sticking target. Figure 5 As shown, when labeling through a single X-ray image 007, it is difficult to determine whether the target is a sticking target that needs to be labeled. For example, the target region of a certain binarized image 008 may be a sticking target or an independent target, but it is observed from the X-ray image that it is more like a sticking target.

[0059] In addition, as shown, Figure 6 When labeling the X-ray image 009, it is necessary to determine the range of the concave defect in combination with the binarized image 0010 to select the appropriate size of the labeling box.

[0060] In the X-ray image of coal gangue, due to the spatial overlap and sticking of multiple targets (coal and gangue, etc.), the outline is no longer smooth and closed, but appears as an irregular structure with concave and recessed. We call this structure a concave defect. That is, the concave defect refers to the local morphological structure formed in the sticking region after the spatial contact or overlap of two or more sticking targets in the X-ray image of coal gangue, which presents a significant inward shrinkage or recess relative to the outline of the respective target body.

[0061] 2.1, morphological characteristics

[0062] (1) The concave defect usually presents a local sharp inward shrinkage of the outline, and the curvature direction of the outline points to the inside of the target;

[0063] (2) Forming a non-convex closure structure with the outer boundary of the target;

[0064] (3) Located at the sticking boundary between two or more targets.

[0065] 2.2, geometric feature determination

[0066] Taking a single target outline as input, the shape is analyzed, and the concave defect can be quantified and determined in the following ways:

[0067] Defect determination based on convex hull:

[0068] (1) Construct the convex hull of the target outline;

[0069] (2) Compare the actual outline with its convex hull, and the difference area between all convex hulls and the actual outline is the concave defect candidate area;

[0070] (3) If the area meets certain angle and area thresholds, it is determined to be an effective concave defect.

[0071] Drawing of the circumscribed convex hull, as shown in Figure 7As shown, including the following steps:

[0072] ① Binaryzation of image a (such as background threshold method); ② Edge detection of image a (such as Canny edge detection); ③ Calculate the circumscribed convex hull of the edge of image a; ④ Fill the circumscribed convex hull c to obtain image d; ⑤ Subtract image b from image d to obtain all concave defects e; ⑥ Set the pixel area threshold to delete the concave defects to obtain f, separate the connected domain of f to obtain independent concave defects q1, q2, q3, q4.

[0073] Wherein, the circumscribed convex hull of the edge is calculated, and the convex hull principle is to calculate the minimum convex polygon envelope of a point set, which contains all points in the point set, just like wrapping all points with a rubber band.

[0074] Let the contour be C and the convex hull be H, then the concave defect area is: D = H-C. As shown in Figure 8 , if the concave defect area Area(D) > Ta and the concave defect angle DefectAngle < Tθ, it is determined as a concave defect.

[0075] Area judgment: If the difference area is greater than Ta, it is a potential concave defect, for example, Ta = 50-300 pixels.

[0076] Angle judgment: If the angle in the difference area is less than Tθ, it indicates that there is a sharp concave corner, Tθ = 100°-130°.

[0077] As shown in Figure 7 , the angle judgment method is: ① Perform edge detection on image f to obtain image i, and perform edge detection on image d to obtain image h; ② Subtract image h from image i to obtain image j, which is the edge of the concave defect; ③ Traverse the edge pixel points, for example, select three points, x (front point), y (middle point), and z (rear point), which are set to be 2 pixels apart from y point, then the lines connecting x and y, and y and z will form an angle.

[0078] It can directly mark the gray-scale image, such as (a) in Figure 9 , the problem is that sometimes it is not clear which targets are adhered. Because it is not segmented, if it seems to be adhered on the gray-scale image, but it is not adhered in the binary image Figure 9 (b), then it is not adhered.

[0079] It is also possible to directly label the binary image, but there are problems. Sometimes it is not clear from the binary image alone which are the connected regions (because some coal gangue shapes look like overlapping connected targets, that is, there may also be concave defects in individual targets), or the detection frame is not accurately labeled. Therefore, labeling to some extent needs to be combined with the original grayscale image and the binary image (the final training can be based on the labeled original grayscale image or the binary image).

[0080] 3. Deep learning model training

[0081] A deep learning detection model is trained to identify pairs of concave defect regions in coal gangue X-ray images, providing a basis for subsequent segmentation. A target detection model (such as YOLO, FasterR-CNN, etc.) is used to train the labeled data set, with the goal of detecting pairs of concave defect regions. During training, cross-entropy loss function or IoU (Intersection over Union) and other indicators are used to optimize the model, so that it can accurately detect the concave defect regions in the image. Through multiple rounds of training and hyperparameter adjustment, the detection accuracy of the model for complex morphological targets in coal gangue images is improved. (The neural network model can be selected from but not limited to YOLOv5, YOLOX, RetinaNet, FasterR-CNN, CascadeR-CNN, DETR, DINO, and other existing deep target detection models. In particular, it is recommended to use a model with small target detection capability and structural attention mechanism to improve the detection accuracy of concave defect regions.)

[0082] 4. Concave defect region detection

[0083] Input the image to be detected, and the model outputs multiple candidate frames and corresponding confidence; apply the non-maximum suppression (NMS) algorithm to remove overlapping redundant frames; keep the high-confidence concave defect frame coordinate information as the basis for subsequent segmentation.

[0084] 5. Concave defect-based target segmentation

[0085] As Figure 10 , use the coordinates of the two detected concave defect regions to determine the segmentation line of the connected region; ① A straight line connecting the two nearest points on the two concave defects can be used (obtain the bounding box of the two detected concave defect regions; in each edge pixel point of the concave defect region, traverse all point pairs, calculate the Euclidean distance, and determine the shortest pixel pair; connect the shortest pixel pair between the two concave defect regions to construct a segmentation line; use the segmentation line to cut or mask the original connected target region, achieving target separation.) ② or based on the shortest path algorithm of the edge gradient direction between the two concave defects to perform target segmentation. Finally, the segmentation result is output as two independent target regions.

[0086] 6. Post-processing and output

[0087] Region connectivity analysis is performed on the segmentation results to eliminate small noise regions. The segmented image or contour coordinates are output for subsequent recognition, sorting, classification, etc. The system can be integrated into an industrial control system for automated operation.

[0088] II. Specific implementation steps

[0089] 1. Raw image acquisition and preprocessing

[0090] Image source: collected by a dual-energy X-ray transmission device in a coal mine site, with a particle size coverage of 10-200mm (to ensure model generalization).

[0091] Preprocessing process:

[0092] De-noising: Gaussian filter (σ=1.5) to remove background noise.

[0093] Enhancement: random rotation (±15°), scaling (0.8-1.2 times), and translation (±10%) to expand the data set.

[0094] Binaryzation: background threshold method is used to generate binary images (e.g. (b) in Figure 7 , to avoid edge loss by background threshold method (e.g. (c) in Figure 7 ).

[0095] Size normalization: scaled to 640x640 pixels, and grayscale image normalized to [0,1].

[0096] 2. Concave defect region labeling and dataset construction

[0097] Labeling rules:

[0098] Dual-image collaborative labeling ( Figure 3 ):

[0099] In the gray image suspected adhesion area (003 in Figure 3 ), the binary image (004 in Figure 3 ) needs to be checked to confirm the actual adhesion.

[0100] In the binary image labeling (006 in Figure 4 ), the gray image (005 in Figure 4 ) needs to be referred to to exclude independent target mislabeling.

[0101] Concave defect determination:

[0102] Calculate the convex hull difference region of the contour: D=H-C.

[0103] Only label the region that meets Area(D)>50 pixels and DefectAngle<120° (006 in Figure 8θ in the image.

[0104] Dataset split:

[0105] Train:Validation:Test = 7:2:1.

[0106] Cover chain adhesion, 8-shaped adhesion, and other complex scenarios Figure 5 and Figure 6 ).

[0107] 3. Deep learning model training

[0108] Model used: YOLOv8s-small-obj (small target detection enhancement).

[0109] Attention mechanism: CBAM module embedded (improve concave defect feature extraction). Output: single-class detection (only "paired concave defects").

[0110] 4. Concave defect area detection

[0111] Detection process:

[0112] Input preprocessed image, model output candidate box (confidence + cx + cy + w + h); filter confidence <0.7 box;

[0113] NMS algorithm to remove redundancy (IoU threshold = 0.5);

[0114] Multi-frame processing: if single adhesion area >2 boxes, keep the two largest areas Figure 5 ); Performance requirements: recall rate ≥85%, time delay ≤50ms.

[0115] 5. Target segmentation based on concave defects

[0116] Segmentation algorithm selection:

[0117]

[0118] Straight line connection step:

[0119] Extract the edge pixel point set of the two concave defect regions.

[0120] Calculate the Euclidean distance of the nearest point pair (P1, P2);

[0121] Connect P1-P2 to form a segmentation line Figure 10 ).

[0122] Shortest path step:

[0123] Calculate the gradient amplitude of the image as an energy graph;

[0124] Take the center of the concave defect as the starting / ending point, and use Dijkstra's algorithm to solve the minimum energy pathFigure 10 ).

[0125] 6. Post-processing and output

[0126] Noise filtering: remove fragments with area < 100 pixels after segmentation;

[0127] Output format:

[0128] Segmentation mask map: for visualization;

[0129] Contour coordinate JSON: drive sorting robot arm.

[0130] The above description is only the preferred specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can make equivalent replacements or changes within the technical range disclosed by the present application according to the technical solution and the inventive concept of the present application, which should be covered within the protection scope of the present application.

Claims

1. A method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning, characterized in that, Includes the following steps: S1. Obtain X-ray images of coal gangue, mark the concave defect areas where they overlap and adhere, and find the paired concave defect areas to form training images. S2. Use training images to train the deep learning model to obtain a predictive model that can predict paired concave defect regions in coal gangue X-ray images. S3. Input the X-ray image of the coal gangue to be detected into the prediction model, and output the image to be segmented with the paired concave defect areas marked. S4. Calculate the shortest path between pairs of concave defect regions at the same overlapping and adhesion point in the image to be segmented, and use the shortest path as the segmentation line to segment the overlapping and adhesion point.

2. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 1, characterized in that, The steps for marking concave defect areas are as follows: S11. Determine the non-overlapping and non-adhesive contours of each target at the overlapping and adhesion points, and connect the contours at the overlapping and adhesion points in sequence to form a closed loop. S12. Draw a convex hull that coincides with the closed loop on the outside of the closed loop to form a convex hull that surrounds each contour, and make multiple unconnected regions between the closed loop and the convex hull. These regions constitute concave defect candidate regions, and all concave defect candidate regions are non-convex closure structures. S13. If there is a region in the concave defect candidate region where the area of ​​the concave defect is greater than the area threshold and the included angle of the concave defect is less than the included angle threshold, then this region is the concave defect region.

3. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 2, characterized in that, The process of obtaining the prediction model is as follows: S21. The training images are binarized images generated by dual-energy X-ray high-energy images or background thresholding method. The image size of the training images is unified to a fixed resolution and normalized. The coordinates of the rectangular boxes of the paired concave defects are converted into the target detection format. S22. Select a deep learning model and activate the channel and spatial attention mechanisms in the deep learning model architecture. Set the output layer as a single-class detection task, that is, only identify paired concave defect regions. S23. Use the training images to train the deep learning model to obtain the corresponding prediction model.

4. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 3, characterized in that, The process of obtaining the image to be segmented is as follows: S31. Process the image to be detected according to the content of step S21 to generate a preprocessed image; S32. Input the preprocessed image into the prediction model and output a set of paired concave defect region boxes; S33. Apply the non-maximum suppression algorithm and set the overlap area ratio threshold IoU = 0.5; retain the pair of concave defect regions with the highest confidence and no IoU conflict within each adhesion region; S34. Output the image to be segmented with the labeled box and coordinate information to the segmentation module.

5. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 4, characterized in that, The steps for calculating the shortest path are as follows: S4A1: Obtain the paired concave defect regions in the current convex hull; S4A2. Obtain the two points with the closest Euclidean distance in the paired concave defect regions, and connect these two points with a straight line. This straight line constitutes the dividing line of the segmentation target.

6. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 4, characterized in that, The steps for calculating the shortest path are as follows: S4B1: Obtain the paired concave defect regions in the current convex hull; S4B2. Use Dijkstra's algorithm or A* algorithm to calculate the shortest path between one concave defect region and another concave defect region. This shortest path constitutes the dividing line of the segmentation target.

7. A method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning, as described in claim 5 or 6, characterized in that, The normalization process used scales the pixel values ​​of the image to [0,1].

8. The method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 7, characterized in that, The deep learning models used are YOLO and Faster R-CNN.

9. A method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning, as described in claim 8, is characterized in that... The annotation process in step S1 requires double verification using both the original X-ray grayscale image and its binarized image. In grayscale images, areas suspected of being stuck together need to be confirmed by binarizing the image to determine whether they are actually stuck together. When labeling in a binary image, it is necessary to refer to the grayscale image to eliminate mislabeled independent targets.

10. A method for detecting and segmenting concave defects in X-ray images of coal gangue based on deep learning according to claim 9, characterized in that, When multiple concave defects exist within the detection frame: Prioritize dividing the two concave defects with the largest areas; After segmentation, perform connected component analysis and remove noise segments with an area smaller than a preset threshold.

Images