A coal mine coal cutter multi-pan-tilt camera detection and tracking method

Abstract

The application discloses a kind of coal mine coal cutter multi-gimbal camera detection tracking methods, it includes the following steps: step S1, acquisition coal cutter image data;Step S2, the coal cutter target detection is carried out to the coal cutter image data collected;Step S3, execute multi-gimbal camera cooperative tracking algorithm;Step S4, generate gimbal camera control instruction generation and execute.The application provides a kind of coal mine coal cutter multi-gimbal camera detection tracking method, solves the problems such as frequent target loss, tracking delay, data redundancy, low positioning accuracy in the existing monitoring method, realizes the cooperative linkage and accurate control of multi-gimbal camera, carries out real-time, continuous, high-precision tracking monitoring to coal mine underground coal cutter operation target, satisfies the underground safety monitoring needs of coal mine intelligent mining.

Description

Technical Field

[0001] This invention relates to a detection and tracking method for a multi-panel camera on a coal cutter in a coal mine, belonging to the field of coal mine safety monitoring technology. Background Technology

[0002] Currently, in underground coal mining, the coal cutter is a core mining equipment, and its operating status and real-time monitoring of the surrounding environment directly affect mining efficiency and operational safety. Due to the harsh underground environment, including narrow spaces, high dust concentrations, poor lighting conditions, and severe equipment vibration, single-panel camera monitoring has limitations such as limited field of view, blind spots, and weak anti-interference capabilities, making it difficult to fully cover the coal cutter's operating area.

[0003] Existing multi-panel camera monitoring solutions mostly employ fixed-angle shooting or simple rotation methods, lacking a collaborative tracking mechanism tailored to the movement characteristics of coal cutters. This leads to frequent target loss, large tracking delays, and severe data redundancy. These solutions fail to fully integrate the movement characteristics of coal cutters into a targeted collaborative tracking mechanism, thus causing a series of problems, such as: Target tracking suffers from frequent loss of targets, significant tracking response delays, and the generation of large amounts of redundant data. This results in time lags between images captured by different pan-tilt cameras when integrating multi-source image data, reducing target positioning accuracy. Furthermore, the lack of a dynamic adjustment strategy means that when the coal cutter moves or the target position shifts, the pan-tilt camera cannot respond and adjust in time, ultimately causing the tracking task to fail.

[0004] With the development of intelligent coal mining technology, there is an urgent need for a monitoring method and system that can achieve coordinated linkage of multiple pan-tilt cameras and accurate tracking of coal cutter operation targets, so as to meet the high precision and high real-time requirements of underground safety monitoring. Summary of the Invention

[0005] The technical problem to be solved by this invention is to overcome the shortcomings of the prior art and provide a detection and tracking method for coal cutting machines using multi-panel cameras. This method solves the problems of frequent target loss, large tracking delay, data redundancy, and low positioning accuracy in existing monitoring methods. It enables the coordinated linkage and precise control of multi-panel cameras, and provides real-time, continuous, and high-precision tracking and monitoring of coal cutting machine operation targets in underground coal mines, thus meeting the underground safety monitoring needs of intelligent coal mining.

[0006] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A method for detecting and tracking coal cutters using a multi-panel camera, comprising the following steps: Step S1: Acquire image data of the coal cutter; Step S2: Perform coal cutter target detection on the acquired coal cutter image data; Step S3: Execute the multi-panel camera collaborative tracking algorithm; Step S4: Generate and execute control commands for the PTZ camera.

[0007] Furthermore, in step S1, acquiring image data of the coal cutter specifically includes the following steps: Multiple PTZ cameras are deployed along the working path of the coal cutter to collect real-time image data of the coal cutter and the equipment parameters of the PTZ cameras.

[0008] Furthermore, in step S2, target detection of the coal cutter is performed on the acquired coal cutter image data, specifically including the following steps: Step S21: Train the target detection model for the coal cutter; Step S22: Coal cutter target detection and invalid target screening.

[0009] Furthermore, in step S3, the multi-panel camera collaborative following algorithm is executed, specifically including the following steps: Step S31: Set the preset point of the pan-tilt camera; Step S32: Initialize the pan-tilt camera sequence and preset position; Step S33: Select the core tracking gimbal camera; Step S34: Using the image from the core tracking gimbal camera as a reference, calculate the positional deviation between the target center of the coal cutter and the image center; Step S35: Dynamically select neighboring PTZ cameras for collaborative tracking.

[0010] Furthermore, in step S31, setting the preset point of the pan-tilt camera specifically includes the following steps: Set P preset points, where... ; The angle of rotation of the pan-tilt camera is controlled according to the preset point.

[0011] Furthermore, in step S32, initializing the pan-tilt camera sequence and preset position specifically includes the following steps: Upon initial startup, the initial tracking sequence number of all pan-tilt cameras is recorded, forming a complete pan-tilt camera sequence S. The expression for the pan-tilt camera sequence S is as follows: ; Where n is the total number of pan-tilt cameras; At the same time, all the preset points of the pan-tilt cameras are uniformly set to the initial preset positions, and the pan-tilt cameras are driven to rotate to the initial preset positions to complete the preparation for following the camera.

[0012] Furthermore, in step S33, selecting the core tracking gimbal camera specifically includes the following steps: In the initial test, coal cutter target detection was performed on all pan-tilt camera images. Data from pan-tilt cameras containing valid targets were selected. The area of ​​the target detection frame in each pan-tilt camera was calculated and sorted in descending order according to the size of the target detection frame area. The top M pan-tilt cameras with the largest target detection frame areas were selected as the core tracking pan-tilt cameras.

[0013] Furthermore, in step S34, using the image from the core tracking gimbal camera as a reference, the positional deviation between the target center of the coal cutter and the image center is calculated, specifically including the following steps: The actual width of the image is known to be Then the x-coordinate C of the center position of the image is: ; The x-coordinate of the top left corner of the coal cutter's target frame is known to be... The target frame width of the coal cutter is Then the x-coordinate T of the target center of the coal cutter is: ; The horizontal deviation Δx between the target center of the coal cutter and the image center is: .

[0014] Furthermore, in step S35, dynamically selecting neighboring gimbal cameras for collaborative tracking specifically includes the following steps: Based on the first recorded pan-tilt camera sequence Determine the serial number of the core tracking gimbal camera. At position pos in the pan-tilt camera sequence S, dynamically select neighboring pan-tilt cameras for cooperative tracking: like The adjacent pan-tilt camera numbers for collaborative tracking are: ; like The adjacent pan-tilt camera numbers for collaborative tracking are: ; like The adjacent pan-tilt camera numbers for collaborative tracking are: .

[0015] Furthermore, in step S4, generating and executing control commands for the pan-tilt camera specifically includes the following steps: Step S41: Generate pan-tilt camera rotation control commands; Step S42: Dynamically adjust the preset points of the pan-tilt camera; Step S43: Issue and execute control commands for the pan-tilt camera.

[0016] By adopting the above technical solution, this invention is based on multi-panel camera collaborative tracking technology and combined with the characteristics of the underground coal mine working environment. Through core mechanisms such as data synchronization, intelligent target screening, dynamic tracking adjustment, and preset point management, it solves the problems of limited field of view, tracking lag, and weak anti-interference ability of traditional multi-panel camera monitoring solutions, and realizes all-round, blind-spot-free tracking and monitoring of the coal cutting machine working area. Attached Figure Description

[0017] Figure 1 This is a flowchart of the detection and tracking method for a multi-panel camera on a coal cutter in a coal mine according to the present invention; Figure 2 This is a schematic diagram of the deployment of the gimbal camera of the present invention; Figure 3 This is a schematic diagram illustrating the division of preset points according to the present invention. Detailed Implementation

[0018] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0019] like Figure 1 As shown in the figure, this embodiment provides a method for detection and tracking of a coal mine coal cutter using a multi-panel camera, which includes the following steps: Step S1: Acquire image data of the coal cutter. Specifically: like Figure 2 As shown, along the working path of the coal cutter, multiple pan-tilt cameras deployed around the coal cutter's operating area utilize standardized communication interfaces from Dahua or Hikvision to achieve unified connection, configuration, and data acquisition of the pan-tilt cameras. This allows for real-time collection of image data of the coal cutter captured by each pan-tilt camera and the camera's device parameters. The camera's device parameters include its device ID, tracking number, and image size, which are used to distinguish between different pan-tilt cameras.

[0020] When initially acquiring image data of the coal cutter, a 3-second initial wait time is set to ensure that all pan-tilt cameras can complete their initial capture. During this initial wait time, each pan-tilt camera retains the latest captured image data of the coal cutter. Subsequently, while the pan-tilt cameras are normally tracking the coal cutter, a 500-millisecond wait time is set, and the latest coal cutter image data captured by the pan-tilt cameras is sent for further evaluation every 500 milliseconds.

[0021] Step S2: Perform coal cutter target detection on the acquired coal cutter image data. Specifically: Step S21: Train the coal cutter target detection model: In model training, a large number of images containing coal cutter targets are prepared in advance. Labelimg is used to label the coal cutter targets in each image, and the images and the labeled files are divided into training set, validation set and test set in a ratio of 8:1:1. Then, the YOLOv8 target detection model is used for training to obtain a weight file. The weight file is the core model file obtained after the algorithm is trained. The weight file includes the feature extraction and recognition rules for the coal cutter targets.

[0022] Step S22, Coal cutter target detection and invalid target screening: The collected coal cutter image data is input into the YOLOv8 target detection model loaded with weight files. The model will automatically identify the coal cutter target in the image, define the target position (output detection box coordinates) according to the feature matching logic during training, and complete the screening of invalid targets (such as other equipment and debris underground).

[0023] Step S3: Execute the multi-panel camera collaborative tracking algorithm. Specifically: Step S31: Set the preset point of the pan-tilt camera: The pan-tilt camera used in this embodiment has a rotation range of 0° to 360°. During the movement of the coal cutter target, the pan-tilt camera tracks and rotates. Due to differences in control reaction time and speed, the camera's rotation and stopping require time, during which it may lose track of the coal cutter target. Then, the pan-tilt camera needs to be repeatedly rotated to adjust the shooting angle. To avoid this problem, P preset points are first set for each pan-tilt camera, where... The angle of rotation of the pan-tilt camera is controlled according to preset points. Since the pan-tilt camera only needs a rotation range of 0° to 180° when tracking and filming the coal cutter, taking P=9 as an example, the angle range of each preset point is 20°. The preset point settings for each pan-tilt camera are as follows: Figure 3 As shown, each time the pan-tilt camera is rotated, it will only rotate to the preset front and rear points.

[0024] Step S32: Initialize the pan-tilt camera sequence and preset position: Upon initial startup, the initial tracking sequence number of all pan-tilt cameras is recorded, forming a complete pan-tilt camera sequence S. The expression for pan-tilt camera sequence S is as follows: ; Where n is the total number of pan-tilt cameras; At the same time, all preset points of the pan-tilt cameras are uniformly set to the initial preset position. Taking 9 preset points as an example, the 5th preset point is selected as the initial preset position. The pan-tilt camera is driven to rotate to the initial preset position by the pan-tilt control commands of Dahua / Hikvision, completing the on-site preparation.

[0025] Step S33: Select the core tracking gimbal camera: In the initial test, coal cutter target detection was performed on all pan-tilt camera images. Data from pan-tilt cameras containing valid targets were selected. The target detection frame area in each pan-tilt camera was calculated and sorted in descending order according to the size of the target detection frame area. The top M pan-tilt cameras (M is preferably 1) with the largest target detection frame area were selected as the core tracking pan-tilt cameras to ensure that the tracking focus is always locked on the clearest and most relevant target.

[0026] Step S34: Determine the target movement direction of the coal cutter: In determining the movement direction of the coal cutter target, the image from the core tracking gimbal camera is used as a reference to calculate the positional deviation between the target center and the image center. The actual width of the image is known to be... Then the x-coordinate of the center position of the image is Target detection reveals that the top-left corner of the coal cutter's target bounding box has an x-coordinate of [value missing]. The target box width is Then the x-coordinate of the target center is The horizontal deviation Δx between the target center of the coal cutter and the image center is: .

[0027] Step S35: Dynamically select neighboring pan-tilt cameras for collaborative tracking: To gain a more comprehensive view of the coal cutter's movements, it's also necessary to select adjacent pan-tilt cameras based on the core tracking camera to observe its surroundings. Therefore, this is based on the initial recorded complete pan-tilt camera sequence. The serial number of the core tracking gimbal camera can be determined. Position in the sequence Dynamically select neighboring pan-tilt cameras for collaborative tracking: like (First in the sequence), then the sequence number of the adjacent pan-tilt camera in the collaborative tracking is ; like (At the end of the sequence), the adjacent pan-tilt camera sequence number for collaborative tracking is... ; like (In the middle of the sequence), the sequence number of the adjacent pan-tilt camera in the collaborative tracking is .

[0028] In the subsequent tracking process, it is only necessary to collect the image information of the core tracking gimbal camera and adjacent gimbal cameras based on the serial number of the adjacent gimbal camera in the collaborative tracking for detection and subsequent judgment, thereby reducing the consumption of computing resources.

[0029] Step S4: Generate and execute control commands for the pan-tilt camera. Specifically: Step S41: Generate pan-tilt camera rotation control commands: The horizontal deviation between the target center and the image center has been calculated. Through the horizontal deviation value With tolerance range Size, tolerance range The default resolution is 350 pixels, used to determine the current camera's rotation direction.

[0030] If the horizontal deviation value absolute value ≤Tolerance range If the target of the coal cutter is determined to be in the center area of ​​the image, a stop command is generated, and the pan-tilt camera maintains its current position. If the horizontal deviation value If the value is less than 0, it is determined that the coal cutter target is located to the left of the center of the image, and a left rotation command is generated; If the horizontal deviation value If the value is greater than 0, it is determined that the coal cutter target is located to the right of the center of the image, and a right rotation command is generated.

[0031] Step S42: Dynamically adjust the preset points of the pan-tilt camera: When the cameras are controlled for the first time, all cameras begin moving from their initial preset positions, adjusting their preset points according to the control commands from the PTZ camera. During subsequent control operations, the actual preset point numbers of the cameras are recorded in real time, and the preset points are dynamically adjusted according to the following rules: When the pan-tilt camera rotates to the left, the preset point number decreases by 1; When the pan-tilt camera rotates to the right, the preset point number increments by 1; When the pan-tilt camera is in a stopped state, the preset point number remains unchanged.

[0032] Step S43: Issue and execute control commands for the pan-tilt camera: Based on the PTZ camera device ID and the preset point number obtained in step S42, a PTZ camera control command is generated through the Dahua / Hikvision communication interface and sent to the corresponding PTZ camera to drive the PTZ camera to perform rotation operation, thereby realizing real-time tracking of the coal cutter target.

[0033] The specific embodiments described above further illustrate the technical problems, technical solutions, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting and tracking coal cutting machines using a multi-panel camera, characterized in that, It includes the following steps: Step S1: Acquire image data of the coal cutter; Step S2: Perform coal cutter target detection on the acquired coal cutter image data; Step S3: Execute the multi-panel camera collaborative tracking algorithm; Step S4: Generate and execute control commands for the PTZ camera.

2. The method for detecting and tracking a coal mine coal cutter using a multi-panel camera according to claim 1, characterized in that, In step S1, acquiring image data of the coal cutter specifically includes the following steps: Multiple PTZ cameras are deployed along the working path of the coal cutter to collect real-time image data of the coal cutter and the equipment parameters of the PTZ cameras.

3. The method for detecting and tracking coal cutters using a multi-panel camera according to claim 1, characterized in that, In step S2, coal cutter target detection is performed on the acquired coal cutter image data, specifically including the following steps: Step S21: Train the target detection model for the coal cutter; Step S22: Coal cutter target detection and invalid target screening.

4. The method for detecting and tracking coal cutters using a multi-panel camera according to claim 1, characterized in that, In step S3, the multi-panel camera collaborative tracking algorithm is executed, which specifically includes the following steps: Step S31: Set the preset point of the pan-tilt camera; Step S32: Initialize the pan-tilt camera sequence and preset position; Step S33: Select the core tracking gimbal camera; Step S34: Using the image from the core tracking gimbal camera as a reference, calculate the positional deviation between the target center of the coal cutter and the image center; Step S35: Dynamically select neighboring PTZ cameras for collaborative tracking.

5. The method for detecting and tracking a coal mine coal cutter using a multi-panel camera according to claim 4, characterized in that, In step S31, setting the preset point of the pan-tilt camera specifically includes the following steps: Set P preset points, where... ; The angle of rotation of the pan-tilt camera is controlled according to the preset point.

6. The method for detecting and tracking a coal cutter using a multi-panel camera according to claim 5, characterized in that, In step S32, initializing the pan-tilt camera sequence and preset position specifically includes the following steps: Upon initial startup, the initial tracking sequence number of all pan-tilt cameras is recorded, forming a complete pan-tilt camera sequence S. The expression for the pan-tilt camera sequence S is as follows: ； Where n is the total number of pan-tilt cameras; At the same time, all the preset points of the pan-tilt cameras are uniformly set to the initial preset positions, and the pan-tilt cameras are driven to rotate to the initial preset positions to complete the preparation for following the camera.

7. The method for detecting and tracking a coal mine coal cutter using a multi-panel camera according to claim 6, characterized in that, In step S33, selecting the core tracking gimbal camera specifically includes the following steps: In the initial test, coal cutter target detection was performed on all pan-tilt camera images. Data from pan-tilt cameras containing valid targets were selected. The area of ​​the target detection frame in each pan-tilt camera was calculated and sorted in descending order according to the size of the target detection frame area. The top M pan-tilt cameras with the largest target detection frame areas were selected as the core tracking pan-tilt cameras.

8. The method for detecting and tracking a coal cutter using a multi-panel camera according to claim 7, characterized in that, In step S34, the positional deviation between the target center of the coal cutter and the image center is calculated based on the image of the core tracking pan-tilt camera. This specifically includes the following steps: The actual width of the image is known to be Then the x-coordinate C of the center position of the image is: ； The x-coordinate of the top left corner of the coal cutter's target frame is known to be... The target frame width of the coal cutter is Then the x-coordinate T of the target center of the coal cutter is: ； The horizontal deviation Δx between the target center of the coal cutter and the image center is: 。 9. The method for detecting and tracking a coal cutter using a multi-panel camera according to claim 8, characterized in that, In step S35, dynamically selecting neighboring pan-tilt cameras for collaborative tracking specifically includes the following steps: Based on the first recorded pan-tilt camera sequence Determine the serial number of the core tracking gimbal camera. At position pos in the pan-tilt camera sequence S, dynamically select neighboring pan-tilt cameras for cooperative tracking: like The adjacent pan-tilt camera numbers for collaborative tracking are: ; like The adjacent pan-tilt camera numbers for collaborative tracking are: ; like The adjacent pan-tilt camera numbers for collaborative tracking are: .

10. The method for detecting and tracking a coal mine coal cutter using a multi-panel camera according to claim 1, characterized in that, In step S4, generating and executing control commands for the pan-tilt camera specifically includes the following steps: Step S41: Generate pan-tilt camera rotation control commands; Step S42: Dynamically adjust the preset points of the pan-tilt camera; Step S43: Issue and execute control commands for the pan-tilt camera.

Images