Video monitoring system and method for construction area of underground drilling machine of coal mine

Abstract

The invention provides a video monitoring system and method for a construction area of an underground drilling machine of a coal mine, and the method carries out the specific improvement of a YOLOv8n algorithm through the following steps: step 401, splitting a classification head and a detection head in a Head part, and replacing Anchor-Based with Anchor-Free; and step 402, a GhostConv module is adopted to replace an original Conv module in a backbone part (Backbone), and the GhostConv module is adopted to replace the original Conv module. And step 403, adding a CA attention mechanism into the C2f module of the neck (Neck) to form a C2f-CA module, and replacing the C2f module with the C2f-CA module. And step 404, the LoSSCIOU loss function is adopted to replace the original IoU loss function. And training the improved YOLOv8n algorithm, and carrying out intrusion detection and early warning on the drill site in the construction area of the underground drilling machine of the coal mine by adopting the trained improved YOLOv8n algorithm. According to the invention, through high automation, intelligentization, real-time monitoring and response, flexible network connection, special camera deployment supports, integration of panorama and infrared imaging technologies, and accurate safety early warning and response, comprehensive upgrading of the video monitoring system for the underground drilling machine construction area of the coal mine is realized; and powerful technical support is provided for safe production and intelligent management of coal mine enterprises.

Description

Technical Field

[0001] The invention belongs to the technical field of underground coal mine drilling construction, relates to construction monitoring, and specifically relates to a video monitoring system and method for a coal mine underground drilling rig construction area. Background Art

[0002] At the underground drilling site in the coal mine, during the construction of the drilling rig, personnel operated the drilling rig in violation of regulations and rules, did not wear safety helmets, did not wear labor protection equipment properly, slept on duty underground, and entered the dangerous area where the drilling rig was under construction, and safety could not be guaranteed during the construction process.

[0003] The current existing technology uses machine vision visible light and other technologies to perform AI video recognition at the drilling site and capture illegal and irregular behaviors. However, the detection accuracy of visible light is not high when there is dust and the visibility of the drilling site is very low. Secondly, the use of infrared temperature sensing technology for AI video detection at the drilling site can clearly detect personnel intrusion into the area, but the accuracy rate is not high for the detection of safety helmets, labor protection supplies, and personnel sleeping on the job. Finally, the current detection of the drilling rig construction area is only an underground early warning detection, and the identification and analysis results cannot be uploaded in real time. Alternatively, real-time upload can be achieved, but the layout of hardware and the construction of the network at the drilling site are cumbersome, which brings many inconveniences to the subsequent relocation of the drilling site, and requires the rearrangement of network cables and the readjustment of cameras. Summary of the invention

[0004] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a video monitoring system and method for a drilling rig construction area in an underground coal mine, so as to solve the technical problem that the accuracy of video monitoring of a drilling rig construction area in the prior art needs to be further improved.

[0005] In order to solve the above technical problems, the present invention adopts the following technical solutions to achieve the above problems:

[0006] A video monitoring system for a coal mine underground drilling rig construction area comprises a drilling rig, wherein a telescopic bracket capable of vertical extension and retraction is installed on a shell body close to a drill bit in the drilling rig, a horizontally arranged support plate is installed on the top of the telescopic bracket, a mining infrared imaging camera is installed on one side of the support plate, and a mining four-eye panoramic camera is installed on the other side of the support plate; the mining infrared imaging camera and the mining four-eye panoramic camera can monitor the drilling site.

[0007] It also includes an underground electronic fence monitoring unit for a coal mine, which includes a mine-use intrinsically safe switch. The mine-use infrared imaging camera and the mine-use four-eye panoramic camera are connected to the mine-use intrinsically safe switch via an Ethernet electrical port. The mine-use intrinsically safe switch is connected to a mine-use intrinsically safe data collector via an Ethernet electrical port. The mine-use intrinsically safe data collector is electrically connected to a mine-use explosion-proof and intrinsically safe controller; the mine-use explosion-proof and intrinsically safe controller is electrically connected to a mine-use sound and light alarm.

[0008] The present invention also protects a video monitoring method for the construction area of a coal mine underground drill rig, and this method uses the video monitoring system for the construction area of a coal mine underground drill rig as described above.

[0009] This method includes the following steps:

[0010] Step 1, four-eye image stitching.

[0011] Step 2, de-shaking.

[0012] Step 3, establishment of a drill site video data set.

[0013] Step 4, improvement of the YOLOv8n algorithm:

[0014] Improve the YOLOv8n algorithm to obtain an improved YOLOv8n algorithm.

[0015] In Step 4, the specific process of improving the YOLOv8n algorithm is as follows:

[0016] Step 401, split the classification and detection heads in the Head part, and replace Anchor-Based with Anchor-Free.

[0017] Step 402, replace the original Conv module with the GhostConv module in the backbone part (Backbone).

[0018] Step 403, add the CA attention mechanism to the C2f module in the neck (Neck) to form a C2f-CA module, and replace the C2f module with the C2f-CA module.

[0019] Step 404, adopt the LoSS CIOU loss function to replace the original IoU loss function.

[0020] Step 5, intrusion detection and early warning:

[0021] Train the improved YOLOv8n algorithm obtained in Step 4, and use the trained improved YOLOv8n algorithm to conduct intrusion detection and early warning for the drill site in the construction area of the coal mine underground drill rig.

[0022] Compared with the prior art, the present invention has the following technical effects:

[0023] (Ⅰ) The present invention has a high degree of automation: A video monitoring system for the construction area of a coal mine underground drill rig equipped by the present invention realizes highly automated operation. This system uses an improved YOLOV8n network model to be able to autonomously identify, analyze, and process data information under various complex working conditions, greatly improving the accuracy and intelligent level of the drill site AI video monitoring system.

[0024] (II) By monitoring the status changes of the fence in real time, the system of the present invention can quickly make judgments and take corresponding control measures, realizing the instant response to the status changes of the fence and the remote control function, effectively avoiding the safety risks caused by human negligence or operation errors, significantly reducing the monitoring cost, and at the same time providing strong support for the intelligent upgrade of coal mine enterprises.

[0025] (III) The camera of the present invention is connected and communicates with the drill rig controller and the collector, and the collector has a WiFi function. As long as the personnel are in the WiFi coverage area, they can view the drill site video. In this way, even when the network connection in the underground drill site is unavailable, video viewing, video storage, and downloading can be carried out locally, facilitating the management of the mine side.

[0026] (IV) The present invention designs a special camera deployment bracket. This bracket has a strong adjustable function, and the design structure of the bracket is relatively compact. By adjusting the bracket, the up and down position of the camera can be adjusted. The bracket is fixed by adjusting bolts, effectively preventing the camera from shaking. The support plate is connected to the bracket through threaded holes, enabling changes in the horizontal and vertical positions of the support plate. The support plate is connected to the bracket by fastening screws, strengthening the fixation of the support plate. A mine four-eye panoramic camera and a mine infrared imaging camera are hoisted below the support plate. The camera is connected to the support plate by bolts, effectively realizing the adjustment of the horizontal azimuth and vertical height of the camera.

[0027] (V) The present invention uses a mine four-eye panoramic camera. This system uses a four-eye image stitching technology, adopts SIFT for feature extraction, uses the K-means clustering algorithm for feature point matching, uses an affine transformation for image correction of the image change model, and finally uses the best stitching line for the final image fusion to obtain a complete video image. It solves the problem of 360° video detection without dead angles in the drill rig construction area. The collected video images are composed of 360° drill site images using the image stitching technology, effectively solving the problem that the 4 video images collected by the 4 cameras (front and back, left and right) in the drill site are independent and the detection accuracy of the 360° horizontal field of view is relatively low. It not only reduces the problem of hardware connection but also effectively improves the detection accuracy of area intrusion.

[0028] (VI) The present invention uses an infrared imaging camera to solve the problems that the illumination in the drill site is relatively low, the dust is relatively large, and visible light cameras cannot clearly capture the drill site video images. It realizes the precise detection and area intrusion alarm functions in the construction area with low illuminance and high dust.

[0029] (VII) In view of the jitter phenomenon in the video images during the drilling process of the drill rig, the Kalman filter algorithm is used to remove the jitter from the video surveillance images. First, based on the motion parameters between the obtained surveillance video images, the Kalman filter theory is combined to filter the trajectories of the feature points in the video surveillance images, calculate the correction amount between the feature points of the images before and after filtering, and directly compensate the surveillance video. This ensures the stability of the images and effectively improves the accuracy of system recognition.

[0030] (VIII) The present invention uses an improved YOLOv8n deep learning algorithm for intelligent analysis. The improved YOLOv8 algorithm significantly improves the accuracy of target detection by optimizing the model structure and loss function.

[0031] (IX) While maintaining high accuracy, the improved YOLOv8n algorithm of the present invention also has the ability to process quickly. This means that the electronic fence system can quickly respond when personnel or equipment enter the dangerous area, issue an alarm in a timely manner or trigger the equipment to stop, thus effectively avoiding accidents. The real-time processing ability of YOLOv8 enables the electronic fence system to continuously monitor the underground environment and ensure the safety of personnel and equipment. This is of great significance for improving the safety level of the mine and reducing safety accidents. The underground environment of the coal mine is complex and changeable, with a large number of interference factors such as dust and noise. The improved YOLOv8n algorithm improves the robustness and stability of the system by optimizing the model structure and loss function, and can maintain stable performance in this complex environment.

[0032] (X) The present invention has strong anti-interference ability: the improved YOLOv8n algorithm has strong anti-interference ability against interference factors such as noise and occlusion in the images, and can ensure accurate target recognition in the underground environment.

[0033] (XI) For the video images collected by the camera of the present invention, after being analyzed and processed by the drill rig controller, when it is recognized that a person enters the secondary warning box, an audible and visual alarm is carried out by driving an audible and visual alarm through RS485.

[0034] (XII) For the video images collected by the camera of the present invention, after being analyzed and processed by the drill rig controller, when it is recognized that a person enters the primary warning box, a drill rig power-off operation is carried out by driving a relay through the switch quantity output.

[0035] (XIII) The underground coal mine electronic fence monitoring unit of the present invention provides a more efficient, safer and more intelligent monitoring solution for underground coal mine operations by real-time monitoring the fence status, combining the wireless communication network and intelligent analysis technology, ensuring the safety of underground operations, improving production efficiency, and having significant technological progressiveness and practical value.

[0036] (XII) In summary, through highly automated, intelligent, real-time monitoring and response, flexible network connection, dedicated camera deployment brackets, the integration of panoramic and infrared imaging technologies, and precise security warnings and responses, the present invention has achieved a comprehensive upgrade of the video monitoring system for the construction area of underground coal mine drills, providing strong technical support for the safety production and intelligent management of coal mining enterprises. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic structural diagram of the video monitoring system for the construction area of underground coal mine drills.

[0038] Figure 2 It is a connection diagram of the underground coal mine electronic fence monitoring unit.

[0039] Figure 3 It is a schematic structural diagram of the YOLOV8n network.

[0040] Figure 4 It is a schematic structural diagram of the improved YOLOv8n network.

[0041] Figure 5 It is a schematic structural diagram of the CA attention mechanism.

[0042] Figure 6 It is a schematic structural diagram of the C2f-CA module.

[0043] Figure 7 It is the process of the secondary warning mechanism for drill site video detection.

[0044] Figure 8 It is a photo of the secondary fence warning machine of the electronic fence system.

[0045] Figure 9 It is a photo of the number detection of people at the drill site.

[0046] Figure 10 It is a photo of the detection of the wearing of labor protection supplies by personnel at the drill site.

[0047] The meanings of the various labels in the figure are as follows: 1 - drill rig, 2 - drill bit, 3 - telescopic support, 4 - support plate, 5 - mine infrared imaging camera, 6 - mine four-eye panoramic camera, 7 - first camera adjustment screw, 8 - second camera adjustment screw, 9 - first fastening screw, 10 - second fastening screw, 11 - drill rig power head, 12 - drill pipe, 13 - drill pipe picking and placing robotic arm, 14 - mine intrinsically safe mobile phone, 15 - mine intrinsically safe switch, 16 - mine intrinsically safe data collector, 17 - mine flameproof and intrinsically safe controller, 18 - mine sound and light alarm, 19 - power supply.

[0048] The following further elaborates on the specific content of the present invention in conjunction with embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0049] It should be noted that all the devices, components, networks, algorithms, operators, modules and technologies in the present invention, unless otherwise specified, all adopt the devices, components, networks, algorithms, operators, modules and technologies known in the prior art.

[0050] The following are specific embodiments of the present invention. It should be noted that the present invention is not limited to the following specific embodiments, and all equivalent transformations made on the basis of the technical solutions of this application fall within the protection scope of the present invention.

[0051] Embodiment 1:

[0052] This embodiment provides a video monitoring system for the construction area of a coal mine underground drill rig. As Figure 1 shown, it includes a drill rig 1. An extendable support 3 that can be vertically extended is installed on the housing of the drill rig 1 near the drill bit 2. A horizontally arranged support plate 4 is installed at the top of the extendable support 3. A mine-used infrared imaging camera 5 is installed on one side of the support plate 4, and a mine-used four-eye panoramic camera 6 is installed on the other side of the support plate; the mine-used infrared imaging camera 5 and the mine-used four-eye panoramic camera 6 can monitor the drill site.

[0053] As a preferred solution of this embodiment, as Figure 1 shown, the mine-used infrared imaging camera 5 is connected to the support plate 4 through a first camera adjustment screw 7; the first camera adjustment screw 7 can adjust the horizontal 360° position of the mine-used infrared imaging camera 5; the mine-used four-eye panoramic camera 6 is connected to the support plate 4 through a second camera adjustment screw 8, and the second camera adjustment screw 8 can adjust the horizontal 360° position of the mine-used four-eye panoramic camera 6.

[0054] Furthermore, in this embodiment, as Figure 1 shown, adjacent two sections of the extendable support 3 are sleeved with threads and locked by a first fastening screw 9. The support plate 4 is connected to the top of the extendable support 3 through a second fastening screw 10.

[0055] Furthermore, in this embodiment, as Figure 1 shown, the drill rig power head 11 in the drill rig 1 is connected to the drill bit 2 through a drill pipe 12, and a drill pipe picking and placing robotic arm 13 is also provided in the drill rig 1.

[0056] Specifically, in this embodiment, both the mine-used infrared imaging camera 5 and the mine-used four-eye panoramic camera 6 adopt the mine-used infrared imaging cameras and mine-used four-eye panoramic cameras known in the art. The drill pipe picking and placing robotic arm 13 adopts the commonly used drill pipe picking and placing robotic arm known in the art.

[0057] As Figure 2As shown in the figure, it further includes an underground coal mine electronic fence monitoring unit. The underground coal mine electronic fence monitoring unit includes a mine intrinsically safe switch 15. A mine infrared imaging camera 5 and a mine four-eye panoramic camera 6 are connected to the mine intrinsically safe switch 15 through Ethernet electrical ports. The mine intrinsically safe switch 15 is connected to a mine intrinsically safe data collector 16 through an Ethernet electrical port. The mine intrinsically safe data collector 16 is electrically connected to a mine flameproof and intrinsically safe controller 17; the mine flameproof and intrinsically safe controller 17 is electrically connected to a mine acoustic-optic alarm 18.

[0058] In this embodiment, further, as Figure 2 shown, both the mine intrinsically safe data collector 16 and the mine flameproof and intrinsically safe controller 17 are connected to a power supply 19.

[0059] In this embodiment, further, as Figure 2 shown, the mine intrinsically safe data collector 16 is also connected to a mine intrinsically safe mobile phone 14 through WiFi. The mine intrinsically safe data collector 16 has a WIFi signal transmission function. The maximum transmission distance of the WiFi module is 30m. Devices such as mobile phones and tablets can connect to the Wifi to display the drill site video in real time. This reduces the network cable connection, and as long as the area covered by WIFi, the drill site video can be viewed in real time, which is convenient to use.

[0060] Specifically in this embodiment, the maximum connection distance between the mine infrared imaging camera 5 and the mine four-eye panoramic camera 6 and the switch 15 through Ethernet is 80m.

[0061] Specifically in this embodiment, the underground coal mine electronic fence monitoring unit uses the mine intrinsically safe switch 15 for data and video transmission. The mine intrinsically safe switch 15 has 3 Ethernet electrical ports and 2 Ethernet optical ports.

[0062] Specifically in this embodiment, the mine intrinsically safe switch 15 and the mine flameproof and intrinsically safe controller 17 are powered by a 127V underground power supply, which has many advantages such as high reliability, low interference, convenient use and maintenance. The mine infrared imaging camera 5 and the mine four-eye panoramic camera 6 are powered by Poe and are powered from the mine intrinsically safe switch 15 through an aviation plug.

[0063] Specifically in this embodiment, in the underground coal mine electronic fence monitoring unit, the acquisition layer uses the mine four-eye panoramic camera 6 and the mine infrared imaging camera 5 to acquire drill site video images; the transmission layer uses a gigabit industrial ring network; the video analysis layer uses intelligent analysis configuration for detecting whether the safety helmets and labor protection supplies of drill site personnel are worn, whether personnel are sleeping on duty, and area intrusion detection, etc.; the display layer: uses explosion-proof mobile phones and explosion-proof tablets to display the drill site detection and identification in real time.

[0064] In this embodiment, the system uses a wireless communication network, which has the advantages of flexibility, reliability, wide coverage, etc. Wireless communication can transmit the status information of the fence unit to the monitoring center in real time, improving the efficiency and real-time performance of information transmission.

[0065] In this embodiment, at the monitoring center, the operator can view the construction status of the drilling rig in the coal mine underground drilling site through the ground remote control platform in real time, and realize the remote control and operation of the fence. At the same time, the monitoring center can also store the received status information, images and videos to form a complete work record for subsequent query, analysis and processing.

[0066] Embodiment 2:

[0067] This embodiment provides a method for video monitoring of the construction area of a coal mine underground drilling rig, which uses the video monitoring system for the construction area of a coal mine underground drilling rig given in Embodiment 1.

[0068] The method includes the following steps:

[0069] Step 1, four-eye image stitching:

[0070] Use SIFT for feature extraction, K-means clustering algorithm for feature point clustering, the model of image change uses affine transformation for image correction, and finally use the best stitching line for final image fusion to obtain a complete video image.

[0071] Specifically in this embodiment, SIFT is used for four-eye image feature extraction. First, key points are extracted, which are points where the image information is very prominent and will not disappear due to factors such as light, scale, and rotation. Search for the image position in the scale space, and identify potential interest points with scale and rotation invariance through the Gaussian filtering algorithm.

[0072] Specifically in this embodiment, the process of positioning key points and determining the feature direction is: at each candidate position, a finely fitted model is used to determine the position and scale.

[0073] Specifically in this embodiment, for the feature vectors of key points, pairwise comparison is performed to find several pairs of mutually matching feature points, and the corresponding relationship between the image scenes is established.

[0074] Specifically in this embodiment, K-means feature point clustering: Initialization: Randomly select k center points as the initial clustering centers. Calculate the distance: Calculate the distance from each sample to the k center points, and divide each sample into the cluster where the nearest center point is located. Recalculate the center: The mean of all points in each cluster is used to recalculate the center of each cluster. Iteration: Continuously iterate the steps of calculating the distance and recalculating the center until there is no change in each cluster or the preset number of iterations is reached. Realize the clustering of feature data and effective extraction.

[0075] Specifically, in this embodiment, for the coordinate correspondence relationship between two images after image registration of video images, an affine transformation is used for horizontal and vertical position correspondence and matching of the images. The affine transformation is for transformations such as translation, rotation, and scaling of the images, which may cause changes in the length of straight lines between the fused images and the included angle between the images. The expression of its transformation process is:

[0076]

[0077] In the formula:

[0078] (x, y) represents a pixel point;

[0079] (x1, y1) represents the pixel point after the video image is transformed relative to the pixel point (x, y);

[0080] a i represents image rotation, mirroring, scaling, and shearing operations;

[0081] t x and t y represent the translation distances of the image in the horizontal and vertical directions.

[0082] Specifically, in this embodiment, optimal seam fusion is adopted: The key point of this fusion method is to find a seam that tries to bypass objects that may cause misalignment phenomena, find the seam in the overlapping area, take the respective images at both ends bounded by the seam, and finally fuse an image without misalignment phenomena. The calculation of the seam is realized through the Sobel operator. The expression of the Sobel operator is:

[0083] E geometry (x, y) = (S x .(I1(x, y) - I2(x, y))) 2 +(S y .(I1(x, y) - I2(x, y))) 2

[0084] In the formula:

[0085] E geometry represents the structural difference of the pixel points in the overlapping area;

[0086] (x, y) represents a pixel point;

[0087] S x and S y respectively represent the Sobel operators in the x - direction and y - direction;

[0088] I1 and I2 represent the two images to be spliced.

[0089] In this embodiment, the mine quadruple panoramic camera 6 adopts image stitching technology to achieve the video acquisition function of 360° dead - angle - free in the drill site. An operation of a two - level early warning mechanism is performed on the collected video images. When a person enters the detection frame, audible and visual alarms and the self - shutdown operation of the drill are respectively carried out. It solves the problem that 4 cameras need to be deployed in one drill site to achieve 360° detection. It saves costs and further improves the real - time performance and accuracy of recognition.

[0090] Step two, anti - jitter:

[0091] For the video image obtained in step one, an anti - jitter algorithm for video images based on Kalman filtering is adopted: perform anti - jitter processing on the video image to obtain an anti - jittered video image.

[0092] Specifically in this step, the camera installed on the drill rig, due to phenomena such as jitter during the operation of the drill rig, causes the video graphic picture to be jumpy and unstable, etc., which poses a challenge to the accurate recognition of video images. The anti - jitter algorithm for video images based on Kalman filtering: performs anti - jitter processing on the video surveillance image, improves the quality of the video image, and improves the recognition accuracy.

[0093] In step two, the specific process of anti - jitter processing is as follows:

[0094] Step 201, first extract the feature points of the first - frame video image, and combine them with the feature points of the next - frame video image. The pixel intensity of the image object with dissimilar feature points will not change between consecutive frames, and adjacent pixels in an image have similar motions.

[0095] Step 202, the pixel I(x, y, t) in the first - frame image moves to the position (x + dx, y + dy) in the second - frame image after time dt. According to step 201, the gray - scale value remains unchanged, and we get:

[0096] I(x, y, t) = I(x + dx, y + dy, t + dt)

[0097] Perform Taylor series expansion on the right - hand side of the equal sign, cancel out the same terms, and divide both sides by dt to obtain the following equation:

[0098] f x u + f y v++f t = 0

[0099] Where:

[0100]

[0101] Step 203: Based on the motion parameters obtained between the monitored video images during the process of de - jittering the video surveillance images, combined with the Kalman filtering theory, filter the trajectory of the feature points in the video surveillance images, calculate the correction amount between the feature points of the images before and after filtering, and directly compensate the monitored video, thereby eliminating the jitter of the monitored video. The specific process is as follows:

[0102] Let represent the rotation angle of the (t - 1)-th frame of the monitored video relative to the t - th frame:

[0103]

[0104] Model the motion between the monitored images using the affine transformation theory:

[0105]

[0106] In the formula:

[0107] H represents the affine matrix;

[0108] a 12 and a 22 represent the change in the displacement of the video image;

[0109] a 21 and a 22 represent the rotation and scaling changes of the video image;

[0110] (x, y) represents the pixel point;

[0111] (x1, y1) represents the pixel point after the affine transformation of the video image relative to the pixel point (x, y).

[0112] Step 204: Calculate the image motion parameters of two adjacent frames of video images:

[0113]

[0114] In the formula:

[0115] (X, Y) represents the central pixel point of the video surveillance image I;

[0116] T X and T y respectively represent the horizontal and vertical translation conversion values of the video surveillance image;

[0117] The translation conversion matrix T and the rotation transformation vector θ of the video sequence can be obtained;

[0118]

[0119] In the formula:

[0120] Ti represents the translational transformation vector between the i-th frame and the (i + 1)-th frame of the video;

[0121] θ i represents the rotation change value between the i-th frame and the (i + 1)-th frame of the video;

[0122] N represents the total number of frames of a video image.

[0123] Step 205: Based on the calculation result of Step 204, the translational transformation trajectory matrix of the video image and the rotation change vector are calculated as follows:

[0124]

[0125] Step 206: Based on the and obtained in Step 205, combined with the Kalman filtering theory, the obtained motion trajectory is filtered. For each segment of the video trajectory processed by Kalman filtering, using the spline fitting algorithm, each segment of the B-spline curve is defined as:

[0126]

[0127] Then there is:

[0128]

[0129] In the formula:

[0130] s represents the position parameter of a certain point on the curve. s = 0 corresponds to the starting point of the curve, and s = 1 corresponds to the ending point of the curve. Usually, it represents a normalized variable, and the range is between (0, 1).

[0131] i represents the index of the control point in the spline curve. It is an index variable that determines from which control point to start the calculation. Usually, it represents the starting point or reference point of the current calculation;

[0132] k represents an order or span parameter, representing the order of the basis function B j,k (s), which determines the smoothness of the curve;

[0133] j represents the sequence number of the control point;

[0134] g i,k (s) represents a spline function, which is generated by weighted summation of the control point Q j . Its specific form is determined by the basis function B j,k (s);

[0135] Q j represents the j-th control point;

[0136] B j,k (s) represents the B-spline equation.

[0137] Step 207, set p k and sp k respectively represent the i-th image feature point in the k-th frame of the trajectory of image feature points before and after video smoothing; T k represents the coefficient for repeatedly correcting the initial point of the image and the feature point, and its calculation formula: sp k = T k .p k , and T can be calculated using the affine transformation model k , to obtain sp k = T k .p k , and compensate the video image, effectively solving the problem of video jitter.

[0138] Step Three, establishment of the drill site video dataset:

[0139] The specific process of Step Three is as follows: Based on the drill site images collected by the mine quadruple panoramic camera 6 and the mine infrared imaging camera 5, use the Labelimg tool to annotate the sample set to obtain an xml label file containing the detected categories and coordinates. Divide the training sample set and the test sample set according to the ratio of 7:3.

[0140] Step Four, improvement of the YOLOv8n algorithm:

[0141] Improve the YOLOv8n algorithm to obtain the improved YOLOv8n algorithm.

[0142] Specifically in this step, as Figure 3 shown, the network structure of YOLOv8n mainly consists of three parts: Backbone, Neck, and Head. The loss function of YOLOv8n includes classification loss, localization loss, and confidence loss. The classification loss calculates whether the anchor box and the corresponding calibrated classification are correct. The localization loss represents the error between the predicted box and the calibrated box. The confidence loss is used to calculate the confidence of the network.

[0143] Specifically in this step, as Figure 3 shown, in the network model structure diagram of YOLOv8n, the backbone network consists of 3 parts: the Conv module, the SPPF module, and the C2f module.

[0144] In Step Four, as Figure 4 shown, the specific process of improving the YOLOv8n algorithm is as follows:

[0145] Step 401: Split the classification and detection heads in the Head part, and replace Anchor-Based with Anchor-Free to reduce the number of box predictions and speed up the inference speed.

[0146] Step 402: In the backbone part, replace the original Conv module with the GhostConv module to reduce the overall number of parameters in the model.

[0147] Specifically in Step 402, the GhostConv module includes:

[0148] Step 40201: Convolve a part of the image using Conv to generate a Conv convolution feature map.

[0149] Step 40202: Perform a linear operation on another part of the image to generate a Ghost feature map.

[0150] Step 40203: Fuse and splice the Conv convolution feature map obtained in Step 40201 with the Ghost feature map obtained in Step 40202 to obtain the final output feature map.

[0151] Step 403: Add the CA attention mechanism to the C2f module in the Neck to form the C2f-CA module. As Figure 6 shown, replace the C2f module with the C2f-CA module to enhance the feature extraction ability and the ability to classify and fuse image information, and improve the network recognition accuracy.

[0152] Specifically in this embodiment, as Figure 5As shown, the CA attention mechanism is a new and efficient attention mechanism. By embedding position information into channel attention, the input feature map size is (C×H×W), where C is the number of channels, H is the image height, and W is the image width. Spatial pooling: Perform global average pooling along the width W and height H for each channel to obtain feature maps of C×H×1 and C×1×W. Feature fusion and transformation: Concat+Conv2d concatenates the two pooled feature maps in the channel dimension to form a feature map of C / r×1×(W+H), and a two-dimensional convolutional layer (Conv2d) with 1×1 is used to fuse and transform the features. Batch normalization and non-linear activation: Normalize the convolutional feature map and use the activation function ReLU for model prediction. Splitting and two-dimensional convolution: Split the activated feature map into C×1×W and C×H×1, and process the split feature maps through Conv2d respectively. Apply the Sigmoid activation function to the two convolutional feature maps to generate two attention maps, which will recalibrate the input feature map in the width and height dimensions respectively. Feature re-weighting: Use the Sigmoid activation function to weight the original feature map. Finally, output the final feature map of C×H×W.

[0153] Step 404, adopt LoSS CIOU Replace the original IoU loss function with the loss function.

[0154] Specifically in this step, the convergence speed of training the neural network with the IoU loss function is very slow. Through continuous optimization, the GIoU loss function is obtained. When calculating the position loss of the target box, it considers the size, aspect ratio, and position offset of the target box. Compared with the traditional IoU loss function, it considers the non-overlapping area to more comprehensively evaluate the bounding box and can more accurately measure the matching degree of the target box, improving the overall detection performance.

[0155] In step 404, LoSS CIOU The expression of the loss function is:

[0156]

[0157] In the formula:

[0158] IoU represents the original IoU loss function;

[0159] p represents the Euclidean distance between the centers of the predicted box and the ground truth box;

[0160] b represents the predicted box, including the center point coordinates and width and height;

[0161] b gt represents the ground truth box, including the center point coordinates and width and height;

[0162] c represents the diagonal length of the minimum bounding rectangle of the predicted box and the ground truth box;

[0163] v represents the aspect ratio consistency, which is used to measure the difference in aspect ratios between the predicted box and the ground truth box;

[0164] a represents the weight function, which is used to balance the influence of IoU and aspect ratio;

[0165] w represents the width of the ground truth box;

[0166] h represents the width of the ground truth box;

[0167] w gt represents the width of the predicted box;

[0168] h gt represents the height of the predicted box.

[0169] In this step, further, data augmentation processing can also be performed in the improved YOLOv8n algorithm: Mosaic image augmentation is adopted. Four images are randomly selected and stitched together into a large image through random scaling, random cropping, and random arrangement. Four images are selected and placed in the upper left, upper right, lower left, and lower right of the large image in turn, and the coordinates of the original images on the large image and the coordinates of the intercepted original images are calculated. The size of the large image is [2×img_size, 2×img_size]; the intercepted original images are pasted on the large image; the offsets of the four original images on the new image are calculated for calculating the positions of the labels on the large image; the large image is randomly rotated, translated, scaled, sheared, and perspective-transformed; through the resize operation, the image size is adjusted to img_size.

[0170] In this embodiment, the improved YOLOv8n deep learning algorithm is used for intelligent analysis to identify the wearing of personal labor protection supplies, personal safety helmets, sleeping on duty underground, area intrusion, etc. The system can automatically judge whether the state of the fence is normal, and can identify potential abnormal situations and send out alarm notifications to the monitoring center in a timely manner. This intelligent analysis function greatly improves the monitoring efficiency and accuracy and reduces the work burden of the monitoring personnel.

[0171] Step Five, Intrusion Detection and Early Warning:

[0172] The improved YOLOv8n algorithm obtained in Step Four is trained, and the trained improved YOLOv8n algorithm is used to perform intrusion detection and early warning on the drill sites in the coal mine underground drilling construction area.

[0173] Specifically in Step Five, such as Figure 7As shown in the figure, the warning frames for intrusion detection and early warning are divided into the secondary warning frame and the primary warning frame of the drilling site; the primary warning frame of the drilling site is located within the secondary warning frame of the drilling site; when a person enters the secondary warning frame of the drilling site, the intrinsically safe data collector 16 for mines sends the video image of the drilling site to the flameproof and intrinsically safe controller 17 for mines, and the flameproof and intrinsically safe controller 17 for mines drives the mine acoustic-optic alarm 18 to perform acoustic-optic alarm operations. When a person enters the primary warning frame of the drilling site, the intrinsically safe data collector 16 for mines sends the video image of the drilling site to the flameproof and intrinsically safe controller 17 for mines, and the flameproof and intrinsically safe controller 17 for mines drives the power-off instrument of the drilling rig 1 to perform the power-off operation of the drilling rig 1.

[0174] Specifically in this embodiment, when the improved YOLOv8n algorithm is trained, the hardware configuration of the AI processor used is as follows: an i9 14900K deep learning host is adopted, which supports a dual-channel RTX4090 GPU workstation, an AI inference model training server host, 128G, D5 memory + 4090, 24G graphics cards * 2 pieces are developed on the Ubuntu 20.04 system, with the programming language Python3.9 and trained and recognized on the Pytorch 2.0.1 framework.

[0175] Specifically in this embodiment, the requirements for model training are as follows: the YOLOv8n model with a relatively small network structure is selected for training, the batch size of the training parameters is set to 16, and the number of iterations (epoch) is 300 times.

[0176] Specifically in this embodiment, the requirements for model evaluation are as follows: after the model training is completed, the model performance is evaluated through three indicators, the mean average precision (MAP), precision, and recall. The accuracy of the trained model reaches 0.99, the recall rate is 0.99, and when the GIoU threshold is equal to 0.5, the average precision is 0.98.

[0177]

[0178] Specifically in this embodiment, the improved YOLOv8n algorithm can be used for safety helmet, labor protection supplies wearing, and personnel area intrusion detection. The photo of the secondary fence warning machine of the electronic fence system is as Figure 8 shown. When a person enters the detection frame, acoustic-optic alarm and self-shutdown operation of the drilling rig are respectively performed. The problems of low light intensity and large dust in the drilling site are solved, and accurate identification of area intrusion under complex backgrounds is realized. The photo of the number of people detected in the drilling site is as Figure 9 shown. The photo of the detection of labor protection supplies wearing by the personnel in the drilling site is as Figure 10 shown.

Claims

1. A video monitoring system for a coal mine underground drilling rig construction area, comprising a drilling rig (1), characterized in that: A telescopic bracket (3) capable of vertical extension and retraction is installed on a housing near a drill bit (2) in a drilling machine (1); a horizontally arranged support plate (4) is installed on the top of the telescopic bracket (3); a mining infrared imaging camera (5) is installed on one side of the support plate (4); and a mining four-eye panoramic camera (6) is installed on the other side of the support plate; the mining infrared imaging camera (5) and the mining four-eye panoramic camera (6) can monitor the drilling site; The invention also comprises an underground electronic fence monitoring unit for a coal mine, wherein the underground electronic fence monitoring unit for a coal mine comprises an intrinsically safe switch (15) for use in a mine, wherein the infrared imaging camera (5) for use in a mine and the four-eye panoramic camera (6) for use in a mine are connected to the intrinsically safe switch (15) for use in a mine via an Ethernet electrical port, wherein the intrinsically safe switch (15) for use in a mine is connected to an intrinsically safe data collector (16) for use in a mine via an Ethernet electrical port, wherein the intrinsically safe data collector (16) for use in a mine is electrically connected to a flameproof and intrinsically safe controller (17) for use in a mine, and wherein the flameproof and intrinsically safe controller (17) for use in a mine is electrically connected to an audible and visual alarm (18) for use in a mine.

2. The video monitoring system for underground drilling rig construction area in a coal mine as claimed in claim 1, characterized in that: The mining infrared imaging camera (5) is connected to the support plate (4) via a first camera adjusting screw (7); the first camera adjusting screw (7) can adjust the horizontal 360° position of the mining infrared imaging camera (5); the mining four-eye panoramic camera (6) is connected to the support plate (4) via a second camera adjusting screw (8); the second camera adjusting screw (8) can adjust the horizontal 360° position of the mining four-eye panoramic camera (6).

3. A method for video monitoring of a drilling rig construction area in an underground coal mine, characterized in that: The method adopts the video monitoring system for the construction area of ​​the underground drilling rig in a coal mine as claimed in claim 1 or 2; The method comprises the following steps: Step 1: Four-eye image stitching; Step 2: debounce; Step 3: Establishment of drilling site video dataset; Step 4: Improvement of YOLOv8n algorithm: Improve the YOLOv8n algorithm to obtain an improved YOLOv8n algorithm; In step 4, the specific process of improving the YOLOv8n algorithm is as follows: Step 401, split the classification and detection heads in the Head part, and replace Anchor-Based with Anchor-Free; Step 402, using the GhostConv module to replace the original Conv module in the backbone; Step 403, adding the CA attention mechanism to the C2f module of the neck to form a C2f-CA module, and replacing the C2f module with the C2f-CA module; Step 404, using LoSS CIOU The loss function replaces the original IoU loss function; Step 5: Intrusion detection warning: The improved YOLOv8n algorithm obtained in step 4 is trained, and the improved YOLOv8n algorithm after training is used to perform intrusion detection and early warning on the drilling site in the drilling rig construction area of ​​the underground coal mine.

4. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: The specific process of step one is: using SIFT to extract features, K-means clustering algorithm to cluster feature points, using affine change model to correct the image, and finally using the best stitching line to perform final image fusion to obtain a complete video image.

5. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: The specific process of step 2 is: for the video image obtained in step 1, a video image de-shaking algorithm based on Kalman filtering is used to perform de-shaking processing on the video image to obtain a de-shaking video image.

6. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: The specific process of step three is as follows: based on the drilling site images collected by the mining four-eye panoramic camera (6) and the mining infrared imaging camera (5), the sample set is annotated using the Labelimg tool to obtain an xml label file containing the detected category and coordinates; the training sample set and the test sample set are divided into a ratio of 7:

3.

7. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: In step 402, the GhostConv module includes: Step 40201, using Conv to perform convolution processing on a portion of the image to generate a Conv convolution feature map; Step 40202, performing a linear operation on another portion of the image to generate a Ghost feature map; Step 40203, the Conv convolution feature map obtained in step 40201 is fused and spliced ​​with the Ghost feature map obtained in step 40202 to obtain the final output feature map.

8. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: In step 404, LoSS CIOU The expression of the loss function is: Where: IoU represents the original IoU loss function; p represents the Euclidean distance between the center point of the predicted box and the real box; b represents the prediction box; b gt represents the real frame; c represents the diagonal length of the minimum enclosing rectangle of the predicted box and the true box; v indicates the consistency of aspect ratio; a represents the weight function; w represents the width of the real box; h represents the width of the real box; w gt Indicates the width of the prediction box; h gt Indicates the height of the prediction box.

9. The method for video monitoring of a coal mine underground drilling rig construction area according to claim 3, characterized in that: In step 5, the warning frame of the intrusion detection warning is divided into a drilling field secondary warning frame and a drilling field primary warning frame; the drilling field primary warning frame is located within the drilling field secondary warning frame; When a person enters the secondary warning frame of the drilling site, the mine intrinsically safe data collector (16) sends the video image of the drilling site to the mine flameproof and intrinsically safe controller (17), and the mine flameproof and intrinsically safe controller (17) drives the mine sound and light alarm (18) to perform a sound and light alarm operation; when a person enters the primary warning frame of the drilling site, the mine intrinsically safe data collector (16) sends the video image of the drilling site to the mine flameproof and intrinsically safe controller (17), and the mine flameproof and intrinsically safe controller (17) drives the power-off device of the drilling rig (1) to perform a power-off operation of the drilling rig (1).

Images