A coal gangue recognition method based on machine vision

Abstract

A coal gangue recognition method based on machine vision, a static coal gangue image acquisition platform is built, and static coal gangue image dataset acquisition operation is carried out to obtain paired static coal gangue image dataset; a fully mechanized top coal caving gangue image acquisition platform is built, and fully mechanized top coal caving gangue image dataset acquisition operation is carried out to obtain images of coal and gangue with different proportions to construct fully mechanized top coal caving gangue image dataset; based on the improved WaveNet low-illumination coal gangue image enhancement method, the low-illumination coal gangue image enhancement network is used to correct the light explosion of low-illumination coal gangue image; the improved YOLOv7-tiny coal gangue target recognition model is trained by using high-quality fully mechanized top coal caving gangue image dataset, and the fully mechanized top coal caving gangue recognition model based on YOLOv7-tiny is formed; the recognition of coal and gangue in the process of fully mechanized top coal caving. The method can solve the problem of low precision of artificial recognition of coal and gangue caused by high dust, low illumination and other harsh environments in the working face, and can improve the efficiency of top coal caving.

Description

Technical Field

[0001] This invention belongs to the field of coal and rock identification technology, specifically relating to a coal and gangue identification method based on machine vision. Background Technology

[0002] Based on my country's current stage of economic and social development and its resource endowment characteristics of being "rich in coal, poor in oil, and lacking in gas," coal will remain the primary energy source for a considerable period. Currently, thick coal seams account for approximately 45% of my country's existing coal reserves and production. Top-coal caving mining is the preferred method for safe and efficient mining of thick and extra-thick coal seams in my country, and has become the core technological approach for safe and efficient mining in thick coal seam areas. Currently, the top-coal caving process still relies on manual control based on the principle of "closing the window when gangue is seen." Due to the high levels of smoke and dust and low light conditions during the coal release process in fully mechanized longwall mining faces, it is difficult to accurately identify the coal-gangue mixture ratio in the released coal flow using only manual methods. Furthermore, the large area of ​​the fully mechanized longwall mining face, coupled with high dust and water mist, severely interferes with the accuracy of workers' judgments. In addition, the large number of hydraulic supports in the top-coal caving face ultimately leads to the current situation of low coal extraction rate, high gangue content, high labor intensity for workers, and low work efficiency in fully mechanized longwall mining.

[0003] Automated coal caving avoids the randomness of manual operation in the coal caving process, enabling true control and equal-volume coal caving. This improves the top coal release rate in fully mechanized top coal caving mining and reduces the gangue mixing rate. Therefore, achieving automated coal caving is key to the automation of fully mechanized top coal caving mining technology. Automatic control of the top coal caving process is crucial for realizing intelligent fully mechanized top coal caving mining. Identifying the gangue content in the coal-gangue mixture is an important basis for automatic control of the top coal caving process, and its accuracy and efficiency are key indicators for top coal caving mining production.

[0004] In the research of coal and gangue identification methods, the X-ray method has many limitations, the infrared detection method is easily affected by the water spray dust removal process and the temperature of the coal mining machine itself, and the sound and vibration identification method is greatly affected by the physical and mechanical properties of coal and gangue, making it difficult to accurately and efficiently identify the coal and gangue release status in fully mechanized longwall mining. This makes it difficult for these technologies to be effectively implemented underground. Image-based coal and gangue release status identification technology has a fast recognition speed and high efficiency, and requires simple equipment. However, during the coal release process, the high dust and low illumination environment leads to uneven exposure and high noise in the coal and gangue images, which is not conducive to subsequent image recognition tasks. In addition, the harsh environment of the fully mechanized longwall face, the varying sizes of coal and gangue, and the dense and unstructured characteristics of image data place higher demands on the complexity and memory consumption of target detection algorithms. Therefore, it is necessary to propose an efficient and robust coal and gangue identification method based on visible light images during top coal caving. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention provides a machine vision-based coal and gangue identification method. This method can solve the problem of low accuracy in manual identification of coal and gangue caused by harsh image acquisition environments such as high dust and low illumination at the working face. It allows workers to stay away from the hydraulic support operation area, reduces the labor intensity and risk of workers, and improves the efficiency of top coal caving mining.

[0006] To achieve the above objectives, the present invention provides a method for coal and gangue identification during fully mechanized longwall mining based on machine vision, comprising the following steps:

[0007] Step 1: Build a static coal gangue image acquisition platform and collect static coal gangue image datasets;

[0008] S11: Construct a static coal gangue image acquisition platform using a coal gangue placement platform, an adjustable lighting system A, an industrial camera, and a camera bracket;

[0009] S12: Place a mixture of coal and gangue in a certain proportion on a coal and gangue placement platform;

[0010] S13: Adjust the light intensity of the adjustable lighting system A to the normal light intensity level, and use an industrial camera to take a picture of the stationary coal gangue to obtain a static coal gangue image under normal lighting.

[0011] S14: Keep the coal and gangue placement status in S13 unchanged, adjust the light intensity of the adjustable lighting system A to the low illumination level, and use an industrial camera to take a picture of the stationary coal and gangue to obtain a static coal and gangue image under low illumination.

[0012] S15: Number and pair the images in S13 and S14, and store them in the coal gangue image dataset under normal lighting and the coal gangue image dataset under low lighting, respectively.

[0013] S16: Repeat S12 to S15 multiple times to obtain paired images of coal and gangue mixtures with different proportions, and use the coal and gangue image dataset under normal lighting and the coal and gangue image dataset under low lighting to form a paired static coal and gangue image dataset.

[0014] Step 2: Build a fully mechanized top coal caving image acquisition platform and collect the fully mechanized top coal caving image dataset;

[0015] S21: A fully mechanized top coal caving test bench, a coal and gangue collection vehicle, an adjustable lighting system B, a high-speed industrial camera, and a camera bracket are used to build a fully mechanized top coal caving image acquisition platform.

[0016] The fully mechanized top coal caving test platform mainly consists of a support frame, a support platform, a funnel, a baffle, a ball screw, and a motor. The support platform is installed on the support frame, and the funnel is in the middle of the support platform. It is used to load coal and gangue mixtures of different proportions. The baffle is located below the funnel and is connected to the ball screw through a screw slide. The output shaft of the motor is connected to the ball screw and is used to change the position of the screw slide and the baffle by driving the ball screw, thereby opening or closing the lower opening of the funnel.

[0017] S22: Before conducting the experiment, use the motor to drive the baffle to move it below the lower opening of the funnel so that the lower opening of the funnel is completely closed. Then, put a certain proportion of coal gangue mixture into the funnel, adjust the light intensity of the adjustable lighting system B, set up the high-speed industrial camera and focus it, and prepare to take pictures.

[0018] S23: When conducting the longwall top coal caving experiment, start and control the motor to run in the forward direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle away from below the lower opening end of the funnel, so that the lower opening end of the funnel opens and the coal discharge operation begins. During the coal discharge process, the high-speed industrial camera is started simultaneously to collect and save images of the falling coal and gangue in real time.

[0019] S24: After the coal discharge in S23 is completed, control the motor to rotate in the opposite direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle to the lower opening end of the funnel, so that the lower opening end of the funnel is in a completely closed state, and turn off the high-speed industrial camera.

[0020] S25: Repeat S22 to S24 multiple times to obtain images of caving coal and gangue at different proportions to construct a fully mechanized top coal and gangue caving image dataset;

[0021] Step 3: Image enhancement method for low-light coal gangue based on improved WaveNet;

[0022] S31: WaveNet is selected as the base model. A squeeze-enhanced self-attention module is introduced to improve its adaptive selection feature fusion part, thereby enhancing the global feature extraction capability of the low-light image enhancement model and obtaining a new low-light image enhancement model, denoted as SWaveNet model.

[0023] S32: Train the SWaveNet model on a paired set of static coal gangue images to obtain a low-light coal gangue image enhancement network;

[0024] S33: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction and saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset. Otherwise, the image is directly saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset.

[0025] Step 4: Construct a coal and gangue identification model based on YOLOv7-tiny for fully mechanized longwall mining;

[0026] S41: Based on the YOLOv7-tiny model, a ghost convolutional network is introduced to replace the standard convolutional network in the backbone network of the YOLOv7-tiny model.

[0027] S42: Introducing a hierarchical visual self-attention model, adding a hierarchical visual attention mechanism to the medium target detection branch and large target detection branch channels of the backbone network in S41.

[0028] S43: Add a small target detection layer for coal gangue, and increase the prediction layer to 4 scales based on S42;

[0029] S44: Introduce the small target metric normalized Vasastan distance, and replace the standard cross-union function with the normalized Vasastan distance function in the original loss function;

[0030] S45: Based on S44, a dynamic detection head framework is introduced to obtain an improved YOLOv7-tiny coal gangue target recognition model;

[0031] S45: Based on S44, a dynamic detection head framework is introduced to obtain an improved YOLOv7-tiny coal gangue target recognition model;

[0032] S46: Image annotation is performed on the high-quality fully mechanized longwall mining coal and gangue image dataset to obtain an annotated fully mechanized longwall mining coal and gangue image dataset. Then, the annotated fully mechanized longwall mining coal and gangue image dataset is used to train the improved YOLOv7-tiny coal and gangue target recognition model to obtain the YOLOv7-tiny-based fully mechanized longwall mining coal and gangue recognition model.

[0033] Step 5: Identification of coal and gangue during the fully mechanized longwall mining process;

[0034] S51: Coal release operation begins, and underground cameras are used to capture real-time images of the coal and gangue released during fully mechanized longwall mining.

[0035] S52: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction to obtain a high-quality fully mechanized top coal caving image. The high-quality fully mechanized top coal caving image is then input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition. Otherwise, the fully mechanized top coal caving image is directly input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition.

[0036] S53: Outputs coal and gangue identification results to the hydraulic support controller, which then controls the hydraulic supports to perform automated coal feeding operations.

[0037] As a preferred embodiment, in step S11 of step one, the adjustable lighting system A includes a multi-level supplementary light A and an illuminance meter A.

[0038] As a preferred embodiment, in step S21 of step two, the adjustable lighting system B includes a multi-level fill light B, an illuminance meter B, and a shooting backdrop.

[0039] As a preferred embodiment, in step S22 of step two, the light intensity of the adjustable lighting system B is adjusted to the normal light intensity.

[0040] As a preferred option, in step S23 of step two, the image data is transferred to a portable hard drive for storage using the USB 3.0 protocol.

[0041] In this invention, an industrial camera is used to photograph the coal and gangue mixture placed on a coal and gangue placement platform. At the same time, by adjusting the adjustable lighting system A to change the light intensity of the photographed images, static coal and gangue images under normal lighting and under low lighting can be easily obtained for the same proportion of coal and gangue mixture. This allows for the formation of paired static coal and gangue image sets. Thus, by collecting a large number of paired images of coal and gangue mixtures with different proportions, a paired static coal and gangue image dataset can be formed. By using a motor to drive a ball screw to rotate, which in turn causes the screw slide to move a baffle to or from the lower opening of the funnel, the opening and closing process of the lower opening of the funnel can be conveniently controlled. This allows for easy simulation of the longwall top coal caving process using the funnel. A high-speed industrial camera is then used to acquire images of the falling coal and gangue in real time, significantly reducing the cost of acquiring these images and enabling the acquisition process to be completed in a laboratory setting. Using this method, a longwall top coal caving image dataset can be easily constructed by acquiring a large number of images of falling coal and gangue at different proportions. WaveNet is selected as the base model, and a compression-enhanced self-attention module is introduced to improve its adaptive feature fusion part. This effectively strengthens the global feature extraction capability of the low-light image enhancement model, thus significantly improving the exposure correction effect. Using the low-light coal and gangue image enhancement network to perform exposure correction processing on coal and gangue images with low average grayscale effectively improves the quality of the coal and gangue images, which is beneficial for accurate and rapid identification in subsequent coal and gangue recognition processes. In constructing a YOLOv7-tiny-based coal and gangue target detection model, the introduction of a ghost convolutional network reduces the computational cost and parameters of the original network, thereby improving detection speed. Simultaneously, the introduction of a hierarchical visual self-attention model further addresses the issue of coal and gangue targets becoming less salient due to factors such as lighting, leading to inaccurate detection. Adding a small coal and gangue target detection layer enables multi-scale detection in the detection head. Introducing a normalized Vassastan distance metric for small targets effectively improves the localization accuracy of bounding boxes and the detection accuracy of small coal and gangue targets. The introduction of a dynamic detection head framework significantly enhances the representational power of the target detection head without increasing computational overhead. Training the improved YOLOv7-tiny coal and gangue target recognition model using a high-quality dataset of coal and gangue images from fully mechanized longwall mining yields a YOLOv7-tiny-based coal and gangue recognition model with high accuracy, fast efficiency, and robustness.In practical applications of coal and gangue identification during longwall top coal caving, a low-light coal and gangue image enhancement network is first used to correct the exposure of low-light coal and gangue images. Then, the resulting high-quality images of coal and gangue released during longwall top coal caving are input into a YOLOv7-tiny-based coal and gangue identification model for recognition. This significantly improves recognition accuracy and speed while reducing computational overhead. This method addresses the problem of low accuracy in manual coal and gangue identification caused by harsh image acquisition environments such as high dust and low light at the working face. It also allows workers to avoid operating hydraulic supports, reducing labor intensity and risks, and improving the efficiency of longwall top coal caving. Attached Figure Description

[0042] Figure 1 This is a flowchart of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the embodiments.

[0044] like Figure 1 As shown, this invention provides a machine vision-based method for coal gangue identification, comprising the following steps:

[0045] Step 1: Build a static coal gangue image acquisition platform and collect static coal gangue image datasets;

[0046] S11: Construct a static coal gangue image acquisition platform using a coal gangue placement platform, an adjustable lighting system A, an industrial camera, and a camera bracket;

[0047] S12: Place a mixture of coal and gangue in a certain proportion on a coal and gangue placement platform;

[0048] S13: Adjust the light intensity of the adjustable lighting system A to the normal light intensity level, and use an industrial camera to take a picture of the stationary coal gangue to obtain a static coal gangue image under normal lighting.

[0049] S14: Keep the coal and gangue placement status in S13 unchanged, adjust the light intensity of the adjustable lighting system A to the low illumination level, and use an industrial camera to take a picture of the stationary coal and gangue to obtain a static coal and gangue image under low illumination.

[0050] S15: Number and pair the images in S13 and S14, and store them in the coal gangue image dataset under normal lighting and the coal gangue image dataset under low lighting, respectively.

[0051] S16: Repeat S12 to S15 multiple times to obtain paired images of coal and gangue mixtures with different proportions, and use the coal and gangue image dataset under normal lighting and the coal and gangue image dataset under low lighting to form a paired static coal and gangue image dataset.

[0052] Step 2: Build a fully mechanized top coal caving image acquisition platform and collect the fully mechanized top coal caving image dataset;

[0053] S21: A fully mechanized top coal caving test bench, a coal and gangue collection vehicle, an adjustable lighting system B, a high-speed industrial camera, and a camera bracket are used to build a fully mechanized top coal caving image acquisition platform.

[0054] The fully mechanized top coal caving test platform mainly consists of a support frame, a support platform, a funnel, a baffle, a ball screw, and a motor. The support platform is installed on the support frame, and the funnel is in the middle of the support platform. It is used to load coal and gangue mixtures of different proportions. The baffle is located below the funnel and is connected to the ball screw through a screw slide. The output shaft of the motor is connected to the ball screw and is used to change the position of the screw slide and the baffle by driving the ball screw, thereby opening or closing the lower opening of the funnel.

[0055] S22: Before conducting the experiment, use the motor to drive the baffle to move it below the lower opening of the funnel so that the lower opening of the funnel is completely closed. Then, put a certain proportion of coal gangue mixture into the funnel, adjust the light intensity of the adjustable lighting system B, set up the high-speed industrial camera and focus it, and prepare to take pictures.

[0056] S23: When conducting the longwall top coal caving experiment, start and control the motor to run in the forward direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle away from below the lower opening end of the funnel, so that the lower opening end of the funnel opens and the coal discharge operation begins. During the coal discharge process, the high-speed industrial camera is started simultaneously to collect and save images of the falling coal and gangue in real time.

[0057] S24: After the coal discharge in S23 is completed, control the motor to rotate in the opposite direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle to the lower opening end of the funnel, so that the lower opening end of the funnel is in a completely closed state, and turn off the high-speed industrial camera.

[0058] S25: Repeat S22 to S24 multiple times to obtain images of caving coal and gangue at different proportions to construct a fully mechanized top coal and gangue caving image dataset;

[0059] Step 3: Low-light coal gangue image enhancement method based on modified WaveNet;

[0060] S31: WaveNet is selected as the base model. A squeeze-enhanced self-attention module is introduced to improve its adaptive selection feature fusion part, thereby enhancing the global feature extraction capability of the low-light image enhancement model and obtaining a new low-light image enhancement model, denoted as SWaveNet model.

[0061] S32: Train the SWaveNet model on a paired set of static coal gangue images to obtain a low-light coal gangue image enhancement network;

[0062] S33: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction and saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset. Otherwise, the image is directly saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset.

[0063] Step 4: Construct a coal and gangue identification model based on YOLOv7-tiny for fully mechanized longwall mining;

[0064] S41: Based on the YOLOv7-tiny model, a ghost convolutional network is introduced to replace the standard convolutional network in the backbone network of the YOLOv7-tiny model. By introducing the ghost convolutional network, the computational cost and parameters of the original network can be reduced, thereby improving the detection speed of the network.

[0065] S42: Introducing a hierarchical visual self-attention model, the hierarchical visual attention mechanism is added to the medium target detection branch and large target detection branch channels of the backbone network in S41. The hierarchical visual attention mechanism is used to enhance feature extraction in the medium target and large target channels, and further solves the problem that the saliency of coal gangue targets decreases due to factors such as illumination, which makes it difficult to detect the targets accurately.

[0066] S43: Add a small target detection layer for coal gangue, which is a 4-scale prediction layer based on S42, to realize multi-scale detection of the detection head part;

[0067] S44: Introduce the normalized Vasastan distance metric for small targets. Replace the standard intersection-union function with the normalized Vasastan distance function in the original loss function; optimize the loss function to improve the positioning accuracy of the bounding box and the detection accuracy of small coal gangue targets;

[0068] S45: Based on S44, a dynamic detection head framework is introduced to obtain an improved YOLOv7-tiny coal gangue target recognition model. The dynamic detection head framework coherently combines a multi-head self-attention mechanism within scale-aware feature words, spatially-aware spatial locations, and task-aware output channels, significantly improving the representation capability of the target detection head without increasing any computational overhead.

[0069] S45: Based on S44, a dynamic detection head framework is introduced to obtain an improved YOLOv7-tiny coal gangue target recognition model;

[0070] S46: Image annotation is performed on the high-quality fully mechanized longwall mining coal and gangue image dataset to obtain an annotated fully mechanized longwall mining coal and gangue image dataset. Then, the annotated fully mechanized longwall mining coal and gangue image dataset is used to train the improved YOLOv7-tiny coal and gangue target recognition model to obtain the YOLOv7-tiny-based fully mechanized longwall mining coal and gangue recognition model.

[0071] Step 5: Identification of coal and gangue during the fully mechanized longwall mining process;

[0072] S51: Coal release operation begins, and underground cameras are used to capture real-time images of the coal and gangue released during fully mechanized longwall mining.

[0073] S52: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction to obtain a high-quality fully mechanized top coal caving image. The high-quality fully mechanized top coal caving image is then input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition. Otherwise, the fully mechanized top coal caving image is directly input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition.

[0074] S53: Outputs coal and gangue identification results to the hydraulic support controller, which then controls the hydraulic supports to perform automated coal feeding operations.

[0075] As a preferred embodiment, in step S11 of step one, the adjustable lighting system A includes a multi-level supplementary light A and an illuminance meter A.

[0076] As a preferred embodiment, in step S21 of step two, the adjustable lighting system B includes a multi-level fill light B, an illuminance meter B, and a shooting backdrop.

[0077] As a preferred embodiment, in step S22 of step two, the light intensity of the adjustable lighting system B is adjusted to the normal light intensity.

[0078] As a preferred option, in step S23 of step two, the image data is transferred to a portable hard drive for storage using the USB 3.0 protocol.

[0079] In this invention, an industrial camera is used to photograph the coal and gangue mixture placed on a coal and gangue placement platform. At the same time, by adjusting the adjustable lighting system A to change the light intensity of the photographed images, static coal and gangue images under normal lighting and under low lighting can be easily obtained for the same proportion of coal and gangue mixture. This allows for the formation of paired static coal and gangue image sets. Thus, by collecting a large number of paired images of coal and gangue mixtures with different proportions, a paired static coal and gangue image dataset can be formed. By using a motor to drive a ball screw to rotate, which in turn causes the screw slide to move a baffle to or from the lower opening of the funnel, the opening and closing process of the lower opening of the funnel can be conveniently controlled. This allows for easy simulation of the longwall top coal caving process using the funnel. Real-time acquisition of images of the caving coal and gangue using a high-speed industrial camera significantly reduces the cost of acquiring these images and allows the acquisition process to be completed in a laboratory setting. Using this method, a longwall top coal caving image dataset can be easily constructed by acquiring a large number of images of caving coal and gangue at different proportions. WaveNet is selected as the base model, and an improved self-attention module is introduced to enhance its adaptive feature fusion part. This effectively strengthens the global feature extraction capability of the low-light image enhancement model, thus significantly improving the exposure correction effect. Using the low-light coal and gangue image enhancement network to perform exposure correction processing on coal and gangue images with low average grayscale effectively improves the quality of the coal and gangue images, which is beneficial for accurate and rapid identification in subsequent coal and gangue recognition processes. In constructing a YOLOv7-tiny-based coal and gangue target detection model, the introduction of a ghost convolutional network reduces the computational cost and parameters of the original network, thereby improving detection speed. Simultaneously, the introduction of a hierarchical visual self-attention model further addresses the issue of coal and gangue targets becoming less salient due to factors such as lighting, leading to inaccurate detection. Adding a small coal and gangue target detection layer enables multi-scale detection in the detection head. Introducing a normalized Vassastan distance metric for small targets effectively improves the localization accuracy of bounding boxes and the detection accuracy of small coal and gangue targets. The introduction of a dynamic detection head framework significantly enhances the representational power of the target detection head without increasing computational overhead. Training the improved YOLOv7-tiny coal and gangue target recognition model using a high-quality dataset of coal and gangue images from fully mechanized longwall mining yields a YOLOv7-tiny-based coal and gangue recognition model with high accuracy, fast efficiency, and robustness.In practical applications of coal and gangue identification during longwall top coal caving, a low-light coal and gangue image enhancement network is first used to correct the exposure of low-light coal and gangue images. Then, the resulting high-quality images of coal and gangue released during longwall top coal caving are input into a YOLOv7-tiny-based coal and gangue identification model for recognition. This significantly improves recognition accuracy and speed while reducing computational overhead. This method addresses the problem of low accuracy in manual coal and gangue identification caused by harsh image acquisition environments such as high dust and low light at the working face. It also allows workers to avoid operating hydraulic supports, reducing labor intensity and risks, and improving the efficiency of longwall top coal caving.

Claims

1. A machine vision-based method for coal gangue identification, characterized in that, Includes the following steps: Step 1: Build a static coal gangue image acquisition platform and collect static coal gangue image datasets; S11: Construct a static coal gangue image acquisition platform using a coal gangue placement platform, an adjustable lighting system A, an industrial camera, and a camera bracket; S12: Place a mixture of coal and gangue in a certain proportion on a coal and gangue placement platform; S13: Adjust the light intensity of the adjustable lighting system A to the normal light intensity level, and use an industrial camera to take a picture of the stationary coal gangue to obtain a static coal gangue image under normal lighting. S14: Keep the coal and gangue placement status in S13 unchanged, adjust the light intensity of the adjustable lighting system A to the low illumination level, and use an industrial camera to take a picture of the stationary coal and gangue to obtain a static coal and gangue image under low illumination. S15: Number and pair the images in S13 and S14, and store them in the coal gangue image dataset under normal lighting and the coal gangue image dataset under low lighting, respectively. S16: Repeat S12 to S15 multiple times to obtain paired images of coal and gangue mixtures with different proportions, and use the coal and gangue image dataset under normal lighting and the coal and gangue image dataset under low lighting to form a paired static coal and gangue image dataset. Step 2: Build a fully mechanized top coal caving image acquisition platform and collect the fully mechanized top coal caving image dataset; S21: A fully mechanized top coal caving test bench, a coal and gangue collection vehicle, an adjustable lighting system B, a high-speed industrial camera, and a camera bracket are used to build a fully mechanized top coal caving image acquisition platform. The fully mechanized top coal caving test platform mainly consists of a support frame, a support platform, a funnel, a baffle, a ball screw, and a motor. The support platform is installed on the support frame, and the funnel is in the middle of the support platform. It is used to load coal and gangue mixtures of different proportions. The baffle is located below the funnel and is connected to the ball screw through a screw slide. The output shaft of the motor is connected to the ball screw and is used to change the position of the screw slide and the baffle by driving the ball screw, thereby opening or closing the lower opening end of the funnel. S22: Before conducting the experiment, use the motor to drive the baffle to move it below the lower opening of the funnel so that the lower opening of the funnel is completely closed. Then, put a certain proportion of coal gangue mixture into the funnel, adjust the light intensity of the adjustable lighting system B, set up the high-speed industrial camera and focus it, and prepare to take pictures. S23: When conducting the longwall top coal caving experiment, start and control the motor to run in the forward direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle away from below the lower opening end of the funnel, so that the lower opening end of the funnel opens and the coal discharge operation begins. During the coal discharge process, the high-speed industrial camera is started simultaneously to collect and save images of the falling coal and gangue in real time. S24: After the coal discharge in S23 is completed, control the motor to rotate in the opposite direction, use the motor to drive the ball screw to rotate, use the rotating ball screw to drive the screw slide to move, and at the same time drag the baffle to the lower opening end of the funnel, so that the lower opening end of the funnel is in a completely closed state, and turn off the high-speed industrial camera. S25: Repeat S22 to S24 multiple times to obtain images of caving coal and gangue at different proportions to construct a fully mechanized top coal and gangue caving image dataset; Step 3: Image enhancement method for low-light coal gangue based on improved WaveNet; S31: WaveNet is selected as the base model. A squeeze-enhanced self-attention module is introduced to improve its adaptive selection feature fusion part, thereby enhancing the global feature extraction capability of the low-light image enhancement model and obtaining a new low-light image enhancement model, denoted as SWaveNet model. S32: Train the SWaveNet model on a paired set of static coal gangue images to obtain a low-light coal gangue image enhancement network; S33: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction and saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset. Otherwise, the image is directly saved to the high-quality fully mechanized top coal caving mining coal and gangue image dataset. Step 4: Construct a coal and gangue identification model based on YOLOv7-tiny for fully mechanized longwall mining; S41: Based on the YOLOv7-tiny model, a ghost convolutional network is introduced to replace the standard convolutional network in the backbone network of the YOLOv7-tiny model. S42: Introducing a hierarchical visual self-attention model, adding a hierarchical visual attention mechanism to the medium target detection branch and large target detection branch channels of the backbone network in S41. S43: Add a small target detection layer for coal gangue, and increase the prediction layer to 4 scales based on S42; S44: Introduce the small target metric normalized Vasastan distance, and replace the standard cross-union function with the normalized Vasastan distance function in the original loss function; S45: Based on S44, a dynamic detection head framework is introduced to obtain an improved YOLOv7-tiny coal gangue target recognition model; S46: Image annotation is performed on the high-quality fully mechanized longwall mining coal and gangue image dataset to obtain an annotated fully mechanized longwall mining coal and gangue image dataset. Then, the annotated fully mechanized longwall mining coal and gangue image dataset is used to train the improved YOLOv7-tiny coal and gangue target recognition model to obtain the YOLOv7-tiny-based fully mechanized longwall mining coal and gangue recognition model. Step 5: Identification of coal and gangue during the fully mechanized longwall mining process; S51: Coal release operation begins, and underground cameras are used to collect real-time images of the coal and gangue released during fully mechanized longwall mining. S52: Calculate the average gray level of each image in the fully mechanized top coal caving image dataset. If it is lower than the set threshold, it is considered a low-light coal and gangue image during the top coal caving process. It is then input into the low-light coal and gangue image enhancement network for exposure correction to obtain a high-quality fully mechanized top coal caving image. The high-quality fully mechanized top coal caving image is then input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition. Otherwise, the fully mechanized top coal caving image is directly input into the fully mechanized top coal caving coal and gangue recognition model based on YOLOv7-tiny for recognition. S53: Outputs coal and gangue identification results to the hydraulic support controller, which then controls the hydraulic supports to perform automated coal feeding operations.

2. The method for identifying coal and gangue in fully mechanized longwall mining based on machine vision according to claim 1, characterized in that, In step S11 of step one, the adjustable lighting system A includes a multi-level supplementary light A and an illuminance meter A.

3. A method for identifying coal and gangue during fully mechanized longwall mining based on machine vision, as described in claim 1 or 2, characterized in that... In step S21 of step two, the adjustable lighting system B includes a multi-level fill light B, an illuminance meter B, and a shooting backdrop.

4. The method for identifying coal and gangue in fully mechanized longwall mining based on machine vision according to claim 3, characterized in that, In step S22 of step two, the light intensity of the adjustable lighting system B is adjusted to the normal light intensity.

5. The method for identifying coal and gangue in fully mechanized longwall mining based on machine vision according to claim 4, characterized in that, In step S23 of step two, the image data is transferred to a portable hard drive for storage using the USB 3.0 protocol.

Images