Intelligent sorting method and system for coal gangue based on image deep learning

Abstract

The present invention discloses an intelligent sorting method and system for gangue based on image deep learning, which belongs to the field of coal rock identification technology, and obtains a global image and a local image of gangue; preprocesses the global image and the local image to obtain a preprocessed global image and a local image; fuses the preprocessed local image to the global image to obtain a fused image; inputs the fused image to a target detection model based on a color feature attention mechanism, performs detection and identification of coal and gangue, and obtains an identification result. The image fusion technology is adopted to fuse the global image and the local image of gangue, so as to provide detailed gangue image information, and the deep learning target detection model based on the color feature attention mechanism is adopted to accurately identify coal and gangue, and the color feature is used to improve the recognition accuracy, ensuring that the sorting process is more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of coal-rock identification, and in particular relates to a method and system for intelligently sorting coal gangue based on image deep learning. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Gangue sorting plays an important role in the mining industry. By effectively separating coal and gangue, it can avoid the waste of coal resources and reduce the damage of gangue to the environment, especially the pollution that may be generated during the stacking and landfilling process. At the same time, sorting technology can also improve the purity of coal products, improve fuel quality, and provide support for clean production and sustainable development of the mining industry.

[0004] Traditional coal gangue sorting mainly relies on manual sorting and identification methods based on physical characteristics such as density and hardness. Manual sorting is inefficient, easily affected by worker fatigue and subjective judgment, and has large errors. The density identification method uses a liquid medium to separate coal and gangue based on the density difference between them, but there are problems such as waste of water resources and rising costs. The hardness identification method is based on the difference in hardness between the two and is often used for selective crushing underground, but it has high requirements for crushing force and particle size. Other identification methods, such as those based on chemical composition, magnetic conductivity, acoustic signals and geometric characteristics, have achieved certain results in experiments, but their promotion in practical applications is limited. In addition, traditional sorting methods are accompanied by high energy consumption and environmental burdens, which are difficult to meet the requirements of modern mining for environmental protection and efficient production.

[0005] Inspired by the manual gangue picking method, the researchers combined automated equipment with image recognition technology, using industrial cameras as the "eyes" of the equipment to obtain images of gangue, and using computers as the "brain" of the equipment to process the images, extract grayscale, texture and other feature information, and realize automatic identification of gangue. This method not only effectively reduces labor intensity, but also greatly improves production efficiency.

[0006] However, there is still a lack of image recognition models that can effectively deal with complex types of coal gangue, and the recognition accuracy of the models is not high. At the same time, integrated intelligent equipment suitable for large-scale production also needs to be optimized. In addition, the intelligent sorting system for coal gangue still needs to be further improved. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a coal gangue intelligent sorting method and system based on image deep learning, which can orderly collect coal gangue images and accurately identify coal and gangue, and then realize efficient coal gangue sorting according to the identification results, thereby significantly improving the sorting accuracy and efficiency.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] In a first aspect, the present invention provides a method for intelligently sorting coal gangue based on image deep learning, comprising:

[0010] Obtain global and local images of coal gangue;

[0011] Preprocessing the global image and the local image to obtain preprocessed global image and local image;

[0012] The preprocessed local image is fused into the global image to obtain a fused image;

[0013] The fused image is input into a target detection model based on a color feature attention mechanism to detect and identify coal and gangue to obtain a recognition result.

[0014] According to a further technical solution, the preprocessing includes denoising, image enhancement and image standardization.

[0015] A further technical solution is to fuse the preprocessed local image into the global image by: inputting the global image and the local image, using the local image as a guide image to perform multi-scale filtering on the global image, using multi-scale filtering in the local area, determining the filter kernel according to the local image, and outputting the filtered global image as a fused image.

[0016] A further technical solution is to perform multi-scale decomposition and post-filtering on the input global image and local image when performing multi-scale filtering. The filtering is expressed as:

[0017]

[0018] in, For the The pixel value of the output image of level, For the local image The pixel value of level, and For each local window The weights and biases within is the position index of the current pixel, Represented as a local area of ​​the image, It is The set of all pixels or points in a local area.

[0019] A further technical solution is to calculate the weight and offset in each local window, which is specifically expressed as:

[0020] ,

[0021] in, For local images and global image exist covariance within the level window, For local images The variance within the level window, is the regularization parameter, For the global image In the window The mean value within For local images In the window The mean within .

[0022] A further technical solution is that the target detection model is a YOLOv5 target detection model that adds color feature extraction and attention mechanism, including a feature extractor, a feature fusion layer and a detection head connected in sequence, the feature extractor adopts a CSPDarknet structure, and an attention mechanism based on color features is introduced to improve the CSPDarknet structure.

[0023] A further technical solution is to use a color histogram to extract color information from the color channel of the fused image before the fused image is input into the target detection model, and generate a specific color feature map as the input of the attention mechanism, which is expressed as follows:

[0024]

[0025]

[0026]

[0027] in, , and The red, green, and blue channel values ​​are The number of pixels, is the indicator function, , and Represents pixels The values ​​of the red, green, and blue channels at ;

[0028] Map the histogram information of each channel to generate a specific color feature map:

[0029]

[0030] in, is a weighted mapping function used to associate color histogram information with pixel locations, Represented as pixel position The red channel intensity at Histogram mapping function through the red channel The result after processing, Represented as pixel position The green channel intensity at Histogram mapping function through the green channel The result after processing, Represented as pixel position The blue channel intensity at Histogram mapping function through the blue channel The result after processing.

[0031] In a second aspect, the present invention provides an intelligent gangue sorting system based on image deep learning, comprising:

[0032] An image acquisition module is configured to: acquire a global image and a local image of the coal gangue;

[0033] An image preprocessing module is configured to: preprocess the global image and the local image to obtain a preprocessed global image and a local image;

[0034] An image fusion module is configured to: fuse the preprocessed local image into the global image to obtain a fused image;

[0035] The target detection module is configured to: input the fused image into a target detection model based on a color feature attention mechanism, detect and identify coal and gangue, and obtain a recognition result.

[0036] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the method for intelligent sorting of coal gangue based on image deep learning as described in the first aspect.

[0037] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the method for intelligent sorting of coal gangue based on image deep learning as described in the first aspect are implemented.

[0038] One or more of the above technical solutions have the following beneficial effects:

[0039] The present invention adopts image fusion technology to fuse the global image of coal gangue with the local image, so as to provide coal gangue image information with rich details, providing a data basis for subsequent target detection. Fusion based on multi-scale methods can highlight local details while retaining the global structure, making the fused image more visually rich. At the same time, the multi-scale method can effectively reduce the artifacts or detail loss problems that may be caused by single-scale filtering when processing complex textures or areas with rich details, and improve the naturalness and detail fidelity of the image by decomposing the image information and processing it at multiple scales.

[0040] The present invention adopts a deep learning target detection model based on the color feature attention mechanism, so that it can more effectively utilize color information, accurately identify coal and gangue, and use color features to improve the recognition accuracy, ensuring a more accurate sorting process.

[0041] The present invention realizes full-process automated sorting through automated operations of modules such as material sorting, image acquisition, preprocessing, image fusion, target detection, and intelligent sorting, thereby reducing manual intervention and improving production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 is a flow chart of a method for intelligently sorting coal gangue according to an embodiment of the present invention;

[0044] Figure 2 It is a structural diagram of the intelligent gangue sorting system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0047] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0048] Embodiment 1

[0049] like Figure 1 As shown, this embodiment discloses a coal gangue intelligent sorting method based on image deep learning, which includes the following steps:

[0050] S1: Obtain global and local images of coal gangue;

[0051] In this embodiment, the gangue is arranged in order by a conveyor belt equipped with a vibration device. When the material passes through the conveyor belt, two cameras with different resolutions are used to take multiple global images and local images respectively to collect high-quality image data. The global image is used to obtain an overall view of the gangue on the conveyor belt, while the local image is used to capture the details of the surface structure of the gangue to ensure that clearer local features are captured. Multiple local images can be taken, and the resolution of the local image is higher and clearer, and more details can be displayed.

[0052] A conveyor belt equipped with a vibrating device is used to ensure that the material does not accumulate during the conveying process. The vibrating device generates continuous or intermittent vibrations to evenly spread the material on the conveyor belt, preventing the material from concentrating in one place or forming a blockage. This design can not only effectively improve the efficiency of material transportation, but also ensure that the material is evenly distributed on the conveyor belt, thus providing a stable material flow for subsequent sorting, classification or processing operations.

[0053] S2: preprocessing the global image and the local image to obtain a preprocessed global image and a local image;

[0054] In this embodiment, preprocessing includes denoising, image enhancement and image standardization. Images are often affected by noise during acquisition and transmission, resulting in reduced image quality. By denoising the image, the noise in the image is reduced or eliminated, thereby improving the image quality. Image enhancement includes improving features such as image contrast, details and brightness, with the aim of improving the image quality and visual effects, so as to better display and identify the content in the image. The purpose of image standardization is to eliminate the interference of factors such as image scale in the image on subsequent processing.

[0055] S3: Fusing the preprocessed local image into the global image to obtain a fused image;

[0056] In this embodiment, a global image and a local image are input, and the local image is used as a guide image to perform multi-scale filtering on the global image. By using multi-scale filtering in a local area (i.e., the covered part of the local image corresponding to the global image), the filter kernel is determined according to the local image of the gangue, so that the output image can smooth the global image while maintaining the details of the local image, and the filtered global image is output as a fused image. Multi-scale filtering can highlight local details while retaining the global structure, making the fused image more visually rich. At the same time, the multi-scale method can effectively reduce the artifacts or detail loss problems that may be caused by single-scale filtering when processing complex textures or areas with rich details, and improve the naturalness and detail fidelity of the image by decomposing the image information and processing it at multiple scales.

[0057] The specific steps are as follows:

[0058] 1) Input images, including global images and local images of coal gangue (the local images are used as guide images).

[0059] 2) Multi-scale segmentation: multi-scale decomposition of the input global image and local image is performed to analyze the details of the image at different scales.

[0060] 3) The filtering formula can be expressed as:

[0061]

[0062] in, For the The pixel value of the output image of level, For the local image The pixel value of level, and For each local window The weights and biases within is the position index of the current pixel, Represented as a local area of ​​the image, specifically, It is The set of all pixels or points in a local area is used to define the application area of ​​the filter kernel.

[0063] 4) Coefficient calculation:

[0064] ,

[0065] in, For local images and global image exist covariance within the level window, For local images The variance within the level window, is the regularization parameter (used to avoid division by zero and control the smoothness of the filter), For the global image In the window The mean value within For local images In the window The mean within .

[0066] 5) Fusion process:

[0067] The filtered output of each level is upsampled and stacked, and the lower resolution results are gradually added to the high resolution results to restore the full-scale image.

[0068]

[0069] in, is the final output fused image. is the total number of levels of multi-scale decomposition, that is, the number of scale layers into which the image is decomposed, For the The weight coefficient at the level is used to adjust the impact of the output of this scale in the final fused image. For the The output image after filtering at the level scale, is the scale index, indicating the scale level currently being processed, from 0 to .

[0070] 6) Adjust parameters: including window size and regularization parameters to control the smoothing effect at different scales.

[0071] 7) Output: The smoothed global image after multi-scale filtering is output, which integrates the details of the local image to achieve image fusion with rich details.

[0072] S4: Input the fused image into a target detection model based on a color feature attention mechanism to detect and identify coal and gangue to obtain a recognition result.

[0073] In this embodiment, a YOLOv5 target detection model with color feature extraction and attention mechanism is used to identify and detect coal and gangue. This model can effectively improve the recognition accuracy of coal and gangue.

[0074] The target detection model consists of a feature extractor Backbone, a feature fusion layer Neck, and a detection head Head connected in sequence. Backbone is responsible for extracting the basic features of the image and uses the CSPDarknet structure to improve the efficiency of feature extraction. Neck is designed to perform multi-scale feature fusion and uses a combination of FPN and PAN to enhance the detection capability of small targets. Head is responsible for generating prediction boxes and classification results, and outputting the category and location coordinates of the target.

[0075] Since coal is black, has a relatively pure texture and contains fewer impurities, while gangue is usually gray or dark gray and contains a lot of impurities, the model pays special attention to color features when identifying coal and gangue. By introducing an attention mechanism based on color features, the feature extraction part of the YOLOv5 model, namely the CSPDarknet structure, is improved, enabling it to more effectively utilize color information, thereby improving recognition accuracy.

[0076] The specific steps are as follows:

[0077] 1) Before the fused image is input into the object detection model, a color feature extraction module is first used to extract color features. The color histogram is used to extract significant color information from the color channels (RGB) of the fused image, and a specific color feature map is generated as the input of the attention mechanism.

[0078] The formula is as follows:

[0079]

[0080]

[0081]

[0082] in, , and The red, green, and blue channel values ​​are The number of pixels, It is an indicator function. When the condition is true, it returns 1, otherwise it returns 0. , and Represents pixels The values ​​of the red, green, and blue channels at are usually in the range of [0, 255].

[0083] Once the color histogram of each channel is calculated, it can be used as a representation of color features. In order to further extract the color features in space, the histogram information of each channel is mapped to generate a specific color feature map:

[0084]

[0085] in, is a weighted mapping function used to associate color histogram information with pixel locations, Represented as pixel position The red channel intensity at Histogram mapping function through the red channel The result after processing, Represented as pixel position The green channel intensity at Histogram mapping function through the green channel The result after processing, Represented as pixel position The blue channel intensity at Histogram mapping function through the blue channel The result after processing.

[0086] 2) Introduce the channel attention mechanism and use the self-attention mechanism to assign weights to the feature maps of different channels. Use the color feature map as the input of the attention mechanism to guide the model to focus more on the color differences between coal and gangue. Through this process, the model can dynamically adjust the attention to color features, thereby improving the recognition effect.

[0087] When calculating self-attention, the feature map at each position is mapped to three vectors: query vector Q, link vector K, and value vector V. The representation of the feature map is adjusted by calculating the attention weight:

[0088]

[0089] in, is the dimension of the key vector, and the softmax operation is used to calculate the attention weights.

[0090] 3) In the Backbone of YOLOv5, a color attention module (i.e., an attention mechanism based on color features) is added to process the color feature map through parallel channels to emphasize the color information in the image. At the same time, the CSPDarknet structure of YOLOv5 is used to extract the spatial and semantic features of the image. The input image passes through multiple layers of convolutional layers and residual blocks to gradually extract high-level original feature maps. In each residual block, the original feature map and the color feature map are processed in parallel along independent paths, and finally weighted fused at the output of the residual block. This fusion method ensures that the model can fully utilize color information while extracting spatial features, thereby enhancing the model's expressive power.

[0091] When performing feature fusion in the feature fusion layer Neck of the target detection model, the color feature information is retained. Through the FPN and PAN structures, the extracted high-level semantic information and low-level color information are effectively fused.

[0092] The detection head of the target detection model still follows the standard structure of YOLOv5 to perform bounding box prediction and classification. However, when making predictions, the classifier additionally combines color features for category judgment, thereby improving the ability to distinguish between coal and gangue.

[0093] According to the output results of the target detection model, the opening and closing of the sorting gate is controlled. If the detection result is identified as gangue, the downward gate is opened and the gangue will fall directly below for centralized processing; if the detection result is identified as coal, the upward gate is opened and the coal is transported separately to the storage area for unified storage.

[0094] In dealing with the identification and sorting of complex types of coal gangue, the present invention proposes a new image fusion method. The current common image recognition model only detects targets by simply shooting coal gangue images, while the method of the present invention shoots global images and local images, and uses image fusion technology to generate high-resolution images containing more details, and then performs recognition analysis. This improvement can significantly improve the accuracy of recognition. Furthermore, the present invention has made targeted improvements to the YOLOv5 model. The existing technical means have not fully considered the significant differences in color between coal and gangue. For this reason, the present invention designs an attention mechanism based on color features, which enhances the YOLOv5 model's ability to pay attention to color, thereby more effectively identifying and distinguishing coal from gangue.

[0095] Embodiment 2

[0096] like Figure 2 As shown, this embodiment provides a coal gangue intelligent sorting system based on image deep learning, including:

[0097] An image acquisition module is configured to: acquire a global image and a local image of the coal gangue;

[0098] An image preprocessing module is configured to: preprocess the global image and the local image to obtain a preprocessed global image and a local image;

[0099] An image fusion module is configured to: fuse the preprocessed local image into the global image to obtain a fused image;

[0100] The target detection module is configured to: input the fused image into a target detection model based on a color feature attention mechanism, detect and identify coal and gangue, and obtain a recognition result.

[0101] The intelligent sorting system also includes a material sorting module, which arranges coal and gangue in order through a queuing mechanism, which is a conveyor belt equipped with a vibrating device, to ensure that each piece of material can be accurately detected and identified.

[0102] It also includes an intelligent sorting module. The sorting gate automatically adjusts the sorting operation according to the target detection results to separate the coal and gangue, thereby achieving efficient automatic sorting and resource utilization.

[0103] Embodiment 3

[0104] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0105] Embodiment 4

[0106] The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method of embodiment 1 are performed.

[0107] The steps involved in the apparatus of the above embodiments 3 and 4 correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0108] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0109] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0110] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. An intelligent coal gangue sorting method based on image deep learning, characterized in that: include: Obtain global and local images of coal gangue; Preprocessing the global image and the local image to obtain a preprocessed global image and a local image; The preprocessed local image is fused into the global image to obtain a fused image; Inputting the fused image into a target detection model based on a color feature attention mechanism to detect and identify coal and gangue, and obtaining a recognition result; The fusing of the preprocessed local image into the global image specifically comprises: inputting the global image and the local image, using the local image as a guide image to perform multi-scale filtering on the global image, using the multi-scale filtering in the local area, determining the filter kernel according to the local image, and outputting the filtered global image as the fused image; When performing multi-scale filtering, the input global image and local image are subjected to multi-scale decomposition and then filtered. The filtering is expressed as: in, For the The pixel value of the output image of level, For the local image The pixel value of level, and For each local window The weights and biases within is the position index of the current pixel, Represented as a local area of ​​the image, It is The set of all pixels or points in a local area.

2. The method for intelligent sorting of coal gangue based on image deep learning according to claim 1, characterized in that: The preprocessing includes denoising, image enhancement and image normalization.

3. The method for intelligent sorting of coal gangue based on image deep learning according to claim 1, characterized in that: In each local window, the weights and offsets are calculated, which can be expressed as: ， in, For local images and global image exist covariance within the level window, For local images The variance within the level window, is the regularization parameter, For the global image In the window The mean value within For local images In the window The mean within .

4. The method for intelligent sorting of coal gangue based on image deep learning according to claim 1, characterized in that: The target detection model is a YOLOv5 target detection model that adds color feature extraction and attention mechanism, including a feature extractor, a feature fusion layer and a detection head connected in sequence. The feature extractor adopts a CSPDarknet structure, and an attention mechanism based on color features is introduced to improve the CSPDarknet structure.

5. The method for intelligent sorting of coal gangue based on image deep learning as claimed in claim 4, characterized in that: Before the fused image is input into the target detection model, the color information is extracted from the color channel of the fused image using the color histogram to generate a specific color feature map as the input of the attention mechanism. The formula is as follows: in, , and The red, green, and blue channel values ​​are The number of pixels, is the indicator function, , and Represents pixels The values ​​of the red, green, and blue channels at ; Map the histogram information of each channel to generate a specific color feature map: in, is a weighted mapping function used to associate color histogram information with pixel locations, Represented as pixel position The red channel intensity at Histogram mapping function through the red channel The result after processing, Represented as pixel position The green channel intensity at Histogram mapping function through the green channel The result after processing, Represented as pixel position The blue channel intensity at Histogram mapping function through the blue channel The result after processing.

6. The intelligent gangue sorting system based on image deep learning is characterized by: include: An image acquisition module is configured to: acquire a global image and a local image of the coal gangue; An image preprocessing module is configured to: preprocess the global image and the local image to obtain a preprocessed global image and a local image; An image fusion module is configured to: fuse the preprocessed local image into the global image to obtain a fused image; The target detection module is configured to: input the fused image into a target detection model based on a color feature attention mechanism to detect and identify coal and gangue to obtain a recognition result; The fusing of the preprocessed local image into the global image specifically comprises: inputting the global image and the local image, using the local image as a guide image to perform multi-scale filtering on the global image, using the multi-scale filtering in the local area, determining the filter kernel according to the local image, and outputting the filtered global image as the fused image; When performing multi-scale filtering, the input global image and local image are subjected to multi-scale decomposition and then filtered. The filtering is expressed as: in, For the The pixel value of the output image of level, For the local image The pixel value of level, and For each local window The weights and biases within is the position index of the current pixel, Represented as a local area of ​​the image, It is The set of all pixels or points in a local area.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in the coal gangue intelligent sorting method based on image deep learning as described in any one of claims 1 to 5 are implemented.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the coal gangue intelligent sorting method based on image deep learning as described in any one of claims 1-5 are implemented.

Images