Large coal block identification and distance early warning system for fully mechanized coal mining face based on edge computing

Abstract

The application provides a fully-mechanized coal mining face large coal block identification and distance early warning system based on edge computing, comprising: a hardware system comprising three groups of camera groups, edge processing nodes, a video processing server and a UI server; an edge processing node identification process: identifying large coal blocks and coal mining machine rollers through a target detection model; while identifying through the target detection model, detecting and positioning support numbers through Aruco code identification; matching the support numbers and the large coal blocks and the support numbers and the coal mining machine rollers at the same position on the video frame image to obtain the support numbers corresponding to the positions of the large coal blocks and the support numbers corresponding to the positions of the coal mining machine rollers; calculating the difference between the support numbers corresponding to the positions of the large coal blocks and the support numbers corresponding to the positions of the coal mining machine rollers as the distance between the large coal blocks and the coal mining machine rollers; and when the number of supports between the large coal blocks and the coal mining machine rollers is less than a set threshold, early warning is performed.

Description

Technical Field

[0001] This invention belongs to the field of coal mining technology, and in particular relates to a large coal block identification and distance early warning system based on edge computing in fully mechanized mining faces. Background Technology

[0002] A coal mining machine is a piece of machinery used in coal mines to cut coal from the coal face and transport the crushed ore via a scraper conveyor. However, during coal mining and transportation, the mining machine may accidentally send oversized coal chunks onto the scraper conveyor belt. If these large chunks are not removed promptly, they will accumulate on the conveyor belt with other normal coal chunks, causing wear and tear and ultimately damaging the belt, affecting the normal operation of the scraper conveyor or the mining machine, and even posing safety hazards. Therefore, manpower is required for monitoring and troubleshooting.

[0003] In order to solve the above problems, people have been seeking a technological solution. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies by providing a large coal block identification and distance early warning system based on edge computing in fully mechanized mining faces.

[0005] This invention proposes a large coal block identification and distance early warning system for fully mechanized mining faces based on edge computing, comprising: Setting up the hardware system: Three sets of cameras are deployed, each set consisting of n cameras; each camera is mounted on the bottom of the horizontal arm of the hydraulic support. Three edge processing nodes are set up, consisting of edge computing devices and switches. Each edge processing node is connected to a group of cameras to perform target detection and recognition on each frame of the video captured by the corresponding camera group, and re-encode the detection results into video and push them to the video processing server. The video processing server is used to save and cache the video encoded by the edge processing nodes, so that users can retrieve the detected video footage. The UI server communicates with the video processing server to provide users with UI reports and video system interface services. Model recognition process: S1. The edge processing node acquires video frame images from the video captured by the camera group and identifies large coal blocks and coal mining machine drums through the target detection model. S2. Affix Aruco codes to the hydraulic supports to identify their support numbers; while performing target detection model recognition, the edge processing node recognizes the Aruco codes to detect and locate the support numbers; the edge processing node matches the support numbers and large coal blocks, as well as the support numbers and coal mining machine drums, at the same position on the video frame image to obtain the support numbers corresponding to the large coal blocks and the support numbers corresponding to the coal mining machine drums. S3. The edge processing node calculates the difference between the support number at the corresponding position of the large coal block and the support number at the corresponding position of the coal mining machine drum, which is used as the distance between the large coal block and the coal mining machine drum. Alarm and Handling When the number of supports between the large coal block and the coal mining machine drum is less than the set threshold, the UI server will push the alarm information to the front-end page, and at the same time flash the screen and emit an alarm sound. If no manual intervention is performed on the alarm information within the predetermined time, the edge processing node sends a shutdown signal to the centralized control system in the fully mechanized mining face system, and the centralized control system controls the coal mining machine to stop running.

[0006] Based on the above, the object detection model is trained using the following method: Model training dataset preparation: Record and extract frames from the video captured by the camera: (1) Place large coal blocks on the conveyor belt and measure the position of the large coal blocks relative to each camera and the coal mining machine drum; (2) Use the RTSP protocol to read the camera stream and record; (3) Extract the image from the video; (4) Change the relative position of the large coal blocks and the shooting angle of the camera, and repeat the process (1)-(4). Using 3D modeling software to simulate coal blocks: (1) Establish 3D models of coal blocks of multiple sizes and shapes; (2) Perform image fusion in the recorded and extracted screen images, and embed the established 3D model images into the extracted screen images. Dataset labeling After the dataset is prepared, use the labelme software to label the images in the dataset; Data augmentation The images in the labeled dataset can be flipped horizontally or stitched together. Model training The YOLOv3+Darknet53 architecture is used as the backbone network. The dataset was imported into the model and trained for 5000 iterations until the average accuracy reached 95%.

[0007] The beneficial effects of this invention are as follows: This invention can monitor the situation on the conveyor belt 24 hours a day in real time through surveillance video. When a large coal block is detected, the system will record and analyze it immediately, issue an alarm, and link with the coal mining machine to stop the coal cutting operation in time and notify relevant personnel to go to the site to check and handle the situation.

[0008] This invention identifies large coal blocks and coal mining machine drums using a target detection model, and simultaneously detects and locates the support number by recognizing Aruco codes. By associating the positions of the large coal blocks and the coal mining machine drums with the support numbers, the distance between the large coal blocks and the coal mining machine drums can be obtained simply by calculating the difference between the support numbers, thus achieving the goal of fast and accurate calculation.

[0009] Additional aspects and advantages of the invention will become apparent in the following description or may be learned by practice of the invention. Attached Figure Description

[0010] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 The system architecture block diagram of the present invention is shown.

[0011] Figure 2 The process of identifying large coal blocks and calculating distances in the system of the present invention is shown. Detailed Implementation

[0012] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in these embodiments can be combined with each other.

[0013] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0014] like Figure 1 As shown, this embodiment provides a large coal block identification and distance early warning system for fully mechanized mining faces based on edge computing, including: Setting up the hardware system: Three sets of cameras are deployed, each set consisting of n cameras; each camera is mounted on the bottom of the horizontal arm of the hydraulic support. Three edge processing nodes are set up, consisting of edge computing devices and switches. Each edge processing node is connected to a group of cameras to perform target detection and recognition on each frame of the video captured by the corresponding camera group, and re-encode the detection results into video and push them to the video processing server. The video processing server is used to save and cache the video encoded by the edge processing nodes, so that users can retrieve the detected video footage. The UI server communicates with the video processing server to provide users with UI reports and video system interface services. Model recognition process, such as Figure 2 As shown: S1. The edge processing node acquires video frame images from the video captured by the camera group and identifies large coal blocks and coal mining machine drums through the target detection model. S2. Affix Aruco codes to the hydraulic supports to identify their support numbers; while performing target detection model recognition, the edge processing node recognizes the Aruco codes to detect and locate the support numbers; the edge processing node matches the support numbers and large coal blocks, as well as the support numbers and coal mining machine drums, at the same position on the video frame image to obtain the support numbers corresponding to the large coal blocks and the support numbers corresponding to the coal mining machine drums. S3. The edge processing node calculates the difference between the support number at the corresponding position of the large coal block and the support number at the corresponding position of the coal mining machine drum, which is used as the distance between the large coal block and the coal mining machine drum. Alarm and Handling When the number of supports between the large coal block and the coal mining machine drum is less than the set threshold, the UI server will push the alarm information to the front-end page, and at the same time flash the screen and emit an alarm sound. If no manual intervention is performed on the alarm information within the predetermined time, the edge processing node sends a shutdown signal to the centralized control system in the fully mechanized mining face system, and the centralized control system controls the coal mining machine to stop running.

[0015] Specifically, the object detection model is trained using the following methods: Model training dataset preparation: Record and extract frames from the video captured by the camera: (1) Place large coal blocks on the conveyor belt and measure the position of the large coal blocks relative to each camera and the coal mining machine drum; (2) Use the RTSP protocol to read the camera stream and record; (3) Extract the image from the video; (4) Change the relative position of the large coal blocks and the shooting angle of the camera, and repeat the process (1)-(4). Using 3D modeling software to simulate coal blocks: (1) Establish 3D models of coal blocks of multiple sizes and shapes; (2) Perform image fusion in the recorded and extracted screen images, and embed the established 3D model images into the extracted screen images. A dataset of 8000 images was created, containing the images and their quantities as shown in the table below: quantity 1200 2400 2400 3000 Dataset labeling After the dataset is prepared, use the labelme software to label the images in the dataset; Data augmentation The images in the labeled dataset can be flipped horizontally or stitched together. Model training The YOLOv3+Darknet53 architecture is used as the backbone network. The dataset was imported into the model and trained for 5000 iterations until the average accuracy reached 95%.

[0016] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A large coal block identification and distance early warning system for fully mechanized mining faces based on edge computing, characterized in that, include: Three camera groups, each consisting of n cameras; each camera is mounted on the bottom of the horizontal arm of the hydraulic support. The hydraulic supports are affixed with Aruco codes to identify their support numbers; Three edge processing nodes, each consisting of edge computing devices and switches, are connected to a group of cameras. Each edge processing node is used to perform target detection and recognition on each frame of the video captured by the corresponding camera group, and to re-encode the detection results into video and push them to the video processing server. The video processing server is used to save and cache the video encoded by the edge processing nodes, so that users can retrieve the detected video footage. The UI server communicates with the video processing server to provide users with UI reports and video system interface services. The system is used for model recognition: S1. The edge processing node acquires video frame images from the video captured by the camera group and identifies large coal blocks and coal mining machine drums through the target detection model. S2. While performing target detection and recognition, the edge processing node uses Aruco encoding to detect and locate the support number. The edge processing node matches the support number and the large coal block, and the support number and the coal mining machine drum at the same position on the video frame image to obtain the support number at the position corresponding to the large coal block and the support number at the position corresponding to the coal mining machine drum. S3. The edge processing node calculates the difference between the support number at the corresponding position of the large coal block and the support number at the corresponding position of the coal mining machine drum, which is used as the distance between the large coal block and the coal mining machine drum. The system is used for alarm and processing: When the number of supports between the large coal block and the coal mining machine drum is less than the set threshold, the UI server will push the alarm information to the front-end page, and at the same time flash the screen and emit an alarm sound. If no manual intervention is performed on the alarm information within the predetermined time, the edge processing node sends a shutdown signal to the centralized control system in the fully mechanized mining face system, and the centralized control system controls the coal mining machine to stop running.

2. The edge computing-based large coal block identification and distance early warning system for fully mechanized mining faces according to claim 1, characterized in that, The object detection model is trained using the following method: Model training dataset preparation: Record and extract frames from the video captured by the camera: (1) Place large coal blocks on the conveyor belt and measure the position of the large coal blocks relative to each camera and the coal mining machine drum; (2) Use the RTSP protocol to read the camera stream and record it; (3) Extract the images from the video; (4) Change the relative position of the large coal block and the shooting angle of the camera, and repeat the process (1)-(4); Using 3D modeling software to simulate coal blocks: (1) Establish 3D models of coal blocks of multiple sizes and shapes; (2) Perform image fusion in the recorded and extracted screen images, and embed the established 3D model images into the extracted screen images. Dataset annotation: After the dataset is prepared, use the labelme software to label the images in the dataset; Data augmentation: The images in the labeled dataset can be flipped horizontally or stitched together. Model training: The YOLOv3+Darknet53 architecture is used as the backbone network. The dataset was imported into the model and trained for 5000 iterations until the average accuracy reached 95%.

Images