A coal mine personnel in-and-out well counting method based on combined target tracking

Abstract

A method for counting personnel entering and exiting a coal mine based on combined target tracking includes the following steps: the counting system is started and enters the working state; a bounding box of the mine entrance area in the video surveillance is selected; the corresponding video stream is captured from the obtained surveillance video; the output personnel detection location box and safety helmet detection location box are input into the personnel tracking module, and the target tracking sequence of personnel and the target tracking sequence of safety helmets are output; the position status of the safety helmet is used to assist the matching score of personnel tracking; the IOA between the personnel location detection box and the mine entrance area in each frame of the target tracking sequence of each personnel is calculated; the IOA between the safety helmet location detection box and the mine entrance area in each frame of the target tracking sequence of each safety helmet is calculated; the calculation results are used to define whether each personnel and safety helmet have entered the mine; and the number of people entering and exiting the mine is counted based on the matching results.

Description

Technical Field

[0001] This invention relates to a method for counting personnel entering and leaving coal mines based on combined target tracking. Background Technology

[0002] Coal is a major source of energy consumption in my country. However, due to the complex underground mining environment and complicated mining process, coal mine safety accidents occur frequently. Relying solely on traditional management models and experience-based methods cannot solve the safety problems in coal mine production. Only by utilizing scientific theories and advanced technologies to achieve full-process monitoring of production can accidents be minimized.

[0003] Due to its complexity, the monitoring video of the coal mine scene has the following characteristics: (1) Many underground and mine entry and exit scenes are overexposed in some areas due to artificial lighting and miners' safety signal lights; (2) The special environment of the coal mine often has dust and moisture, which affects the image quality; (3) Some areas have poor lighting and dark scenes, and the underground personnel are generally wearing dark blue work clothes. The underground personnel and the background are similar in color, which makes the outline blurry and difficult to distinguish from the background.

[0004] Therefore, relying solely on object detection results to count personnel in coal mine scenes can lead to discrepancies between the statistical results and the actual number of people due to overexposure, dim lighting, misidentification of personnel, and occlusion. Furthermore, existing vision-based personnel counting methods rely solely on object detection results to count the number of people, and cannot calculate the movement status of personnel entering and leaving the mine, let alone determine whether personnel have entered or left the mine. Summary of the Invention

[0005] This invention provides a method for counting personnel entering and exiting coal mines based on combined target tracking. The method is rationally designed and uses combined target tracking to count the number of personnel entering and exiting the mine, improving the reliability of personnel identification and the accuracy of personnel statistics. It is applicable to complex application scenarios of coal mine video surveillance, accurately calculating the movement status of personnel and determining whether they have entered or exited the mine. It avoids errors in verification and statistical results caused by overexposure, dim lighting, misidentification of personnel, and occlusion, thereby improving the continuity and accuracy of tracking personnel entering and exiting the mine. The method defines the entry and exit status of personnel and safety helmets, merges the statistical results of personnel and safety helmets, and then counts the number of personnel entering and exiting the mine, solving the problems existing in the prior art.

[0006] The technical solution adopted by the present invention to solve the above-mentioned technical problems is as follows:

[0007] A method for counting personnel entering and exiting a coal mine based on combined target tracking, the counting method comprising the following steps:

[0008] S1, the counting system starts and enters working status, and selects the mine shaft entrance area in the video surveillance;

[0009] S2, capture the corresponding video stream from the obtained surveillance video and send it to the target detection module for real-time inference. The target detection module outputs the personnel detection location box and the safety helmet detection location box in each frame of the image.

[0010] S3, input the output personnel detection location box and safety helmet detection location box into the personnel tracking module, and output the target tracking sequence of the personnel and the target tracking sequence of the safety helmet;

[0011] S4 uses the position status of the safety helmet to assist in the matching score of personnel tracking;

[0012] S5, calculate the IOA between the personnel location detection box and the wellhead region in each frame of the target tracking sequence for each personnel;

[0013] S6, calculate the IOA between the safety helmet position detection box and the wellhead region in each frame of the target tracking sequence for each safety helmet;

[0014] S7, based on the calculation results, defines whether each person and safety helmet should enter the mine;

[0015] S8 maps the target tracking sequence of personnel to the target tracking sequence of safety helmets, matches the personnel corresponding to each safety helmet according to the positional relationship between personnel and safety helmets, and counts the cases where no match can be made separately.

[0016] S9, based on the matching results, count the number of people entering the well.

[0017] The counting system includes a target detection module and a personnel tracking module.

[0018] The target detection module is a one-stage target detector used to quickly detect the personnel detection location map and the safety helmet detection location map in each frame of the video.

[0019] The personnel tracking module uses Kalman filtering to predict the positions of the personnel detection location map and the safety helmet detection location map in subsequent video frames, calculates the overlap ratio with the target detection results of subsequent frames, calculates the matching score based on the overlap ratio, and includes the target detection results with high scores in the target tracking sequence, thereby establishing the target tracking sequence for personnel and safety helmets.

[0020] The personnel tracking module defines the motion state of the tracking sequence by calculating the detection position diagram in the personnel and safety helmet target tracking sequence and the IOA of the mine entrance. The larger the IOA, the more overlapping areas there are, that is, the closer the target is to the mine entrance.

[0021] The target detection module includes a personnel detection component and a safety helmet detection component, and the personnel tracking module includes a personnel location box tracking component and a safety helmet location box tracking component.

[0022] This invention employs the aforementioned structure, jointly tracking personnel and safety helmets to obtain two sets of tracking sequences: a personnel tracking sequence and a safety helmet tracking sequence. The position of the safety helmet in the safety helmet tracking sequence can guide the tracking results of personnel in the video, thereby improving the continuity and accuracy of personnel tracking. By calculating the positional bounding box of each target in the personnel and safety helmet tracking sequences and the IOA of the wellhead area, the entry and exit status of personnel and safety helmets is defined, and the statistical results of personnel and safety helmets are fused to count the number of personnel entering and exiting the well. Through the coordinated operation of various functional component modules within the counting system, following predetermined calculation steps and sequences, accurate personnel counting is achieved, offering advantages of precision, practicality, simplicity, and efficiency. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the process of the present invention.

[0024] Figure 2 This is a schematic diagram of the counting system of the present invention.

[0025] Figure 3 This is a schematic diagram of the target detection module of the present invention.

[0026] Figure 4 This is a schematic diagram of the personnel tracking module of the present invention. Detailed Implementation

[0027] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0028] like Figure 1-4 As shown, a method for counting personnel entering and leaving a coal mine based on combined target tracking includes the following steps:

[0029] S1, the counting system starts and enters working status, and selects the mine shaft entrance area in the video surveillance;

[0030] S2, capture the corresponding video stream from the obtained surveillance video and send it to the target detection module for real-time inference. The target detection module outputs the personnel detection location box and the safety helmet detection location box in each frame of the image.

[0031] S3, input the output personnel detection location box and safety helmet detection location box into the personnel tracking module, and output the target tracking sequence of the personnel and the target tracking sequence of the safety helmet;

[0032] S4 uses the position status of the safety helmet to assist in the matching score of personnel tracking;

[0033] S5, calculate the IOA between the personnel location detection box and the wellhead region in each frame of the target tracking sequence for each personnel;

[0034] S6, calculate the IOA between the safety helmet position detection box and the wellhead region in each frame of the target tracking sequence for each safety helmet;

[0035] S7, based on the calculation results, defines whether each person and safety helmet should enter the mine;

[0036] S8 maps the target tracking sequence of personnel to the target tracking sequence of safety helmets, matches the personnel corresponding to each safety helmet according to the positional relationship between personnel and safety helmets, and counts the cases where no match can be made separately.

[0037] S9, based on the matching results, count the number of people entering the well.

[0038] The counting system includes a target detection module and a personnel tracking module.

[0039] The target detection module is a one-stage target detector used to quickly detect the personnel detection location map and the safety helmet detection location map in each frame of the video.

[0040] The personnel tracking module uses Kalman filtering to predict the positions of the personnel detection location map and the safety helmet detection location map in subsequent video frames, calculates the overlap ratio with the target detection results of subsequent frames, calculates the matching score based on the overlap ratio, and includes the target detection results with high scores in the target tracking sequence, thereby establishing the target tracking sequence for personnel and safety helmets.

[0041] The personnel tracking module defines the motion state of the tracking sequence by calculating the detection position diagram in the personnel and safety helmet target tracking sequence and the IOA of the mine entrance. The larger the IOA, the more overlapping areas there are, that is, the closer the target is to the mine entrance.

[0042] The target detection module includes a personnel detection component and a safety helmet detection component, and the personnel tracking module includes a personnel location box tracking component and a safety helmet location box tracking component.

[0043] The working principle of a coal mine personnel counting method based on combined target tracking in this invention embodiment is as follows: Combined target tracking is used to count the number of personnel entering and leaving the coal mine, improving the reliability of personnel identification and the accuracy of personnel counting. This method is applicable to complex application scenarios of coal mine video surveillance, accurately calculating the movement status of personnel, determining whether personnel have entered or left the mine, and avoiding errors in verification and statistical results due to overexposure, dim lighting, misidentification of personnel, and occlusion. This improves the continuity and accuracy of tracking personnel entering and leaving the coal mine. The method defines the entry and exit status of personnel and safety helmets, merges the statistical results of personnel and safety helmets, and then counts the number of personnel entering and leaving the mine.

[0044] The overall solution mainly includes the following steps: The counting system starts up and enters the working state, selecting the mine shaft entrance area map in the video surveillance; capturing the corresponding video stream from the obtained surveillance video and sending it to the target detection module for real-time inference, the target detection module outputs the personnel detection position box and the safety helmet detection position box in each frame of the image; inputting the output personnel detection position box and safety helmet detection position box into the personnel tracking module, outputting the personnel target tracking sequence and the safety helmet target tracking sequence; using the position status of the safety helmet to assist in the matching score of personnel tracking; calculating the IOA between the personnel position detection box and the shaft entrance area in each frame of the target tracking sequence for each person; calculating the IOA between the safety helmet position detection box and the shaft entrance area in each frame of the target tracking sequence for each safety helmet; defining whether each person and safety helmet has entered the mine based on the calculation results; mapping the personnel target tracking sequence and the safety helmet target tracking sequence, matching the personnel corresponding to each safety helmet according to the position relationship between the personnel and safety helmet, and counting separately for cases where no match can be made; counting the number of people entering the mine based on the matching results.

[0045] With the continuous advancement of artificial intelligence and computer technology, personnel counting methods based on intelligent monitoring and analysis of surveillance videos have seen significant development. These methods process surveillance videos in real time and utilize technologies such as computer vision, pattern recognition, and artificial intelligence to detect moving targets underground, thereby enabling the counting of personnel in corresponding scenarios. Although this is very difficult and challenging, it requires no additional investment in monitoring equipment and can be integrated with existing control systems to achieve real-time early warning and coordinated control, which is of great significance for safe production in coal mines and for protecting the lives of miners.

[0046] Object detection is a typical task in the field of computer vision. The detection result includes the location and category information of the target object. Current mainstream detection schemes mainly include one-stage object detection and two-stage object detection algorithms. One-stage object detection algorithms are mainly applied to scenarios with large target sizes and high real-time processing requirements; two-stage object detection algorithms are more suitable for scenarios with many small targets and high accuracy requirements. Object counting aims to estimate the number of relevant targets in an image or video using a computer. Depending on the specific application scenario, it is mainly divided into two categories: line-crossing and region-crossing. Line-crossing methods generally count the number of targets crossing a line of interest in the video to obtain the actual number passing through a certain point; region-crossing methods generally count targets within a region of interest in the scene.

[0047] The main innovation of this application is as follows: by jointly tracking the personnel and safety helmets, two sets of tracking sequences are obtained, namely the personnel tracking sequence and the safety helmet tracking sequence. The position of the safety helmet in the safety helmet tracking sequence can guide the tracking results of the personnel in the video, thereby improving the continuity and accuracy of personnel tracking. By calculating the position diagram of each target in the personnel and safety helmet tracking sequences and the IOA of the wellhead area, the entry and exit status of personnel and safety helmets is defined, and the statistical results of personnel and safety helmets are fused to count the number of personnel entering and exiting the well.

[0048] Specifically, the counting system includes a target detection module and a personnel tracking module; it can access real-time monitoring video through a video stream interface, and the backend uses our personnel counting algorithm to complete the real-time headcount of personnel entering and leaving the mine and returns the statistical results.

[0049] Preferably, the target detection module is a one-stage target detector, such as YOLOx, which is used to quickly detect the personnel detection location map and the safety helmet detection location map in each frame of the video. Combined with massive coal mine personnel data, a pre-trained large model of coal mine personnel is trained using a self-supervised method, which serves as the backbone network of the target detector. Then, the target detector is trained to improve the accuracy of target detection.

[0050] Preferably, the personnel tracking module uses Kalman filtering to predict the positions of the personnel detection location map and the safety helmet detection location map in subsequent video frames, calculates the overlap ratio with the target detection results of subsequent frames, calculates the matching score based on the overlap ratio, and includes the target detection results with high scores in the target tracking sequence, thereby establishing the target tracking sequence for personnel and safety helmets; in the specific personnel tracking process, the position status of the safety helmet is used to assist in guiding the matching score of personnel tracking.

[0051] Preferably, the personnel tracking module defines the motion state of the tracking sequence by calculating the detection position block diagram in the personnel and safety helmet target tracking sequence and the IOA at the mine entrance. The larger the IOA, the more overlapping areas there are, that is, the closer the target is to the mine entrance. Then, the entry and exit status of the target can be defined according to the beginning and end states of the IOA in the personnel and safety helmet tracking sequence, and the number of people entering and exiting the mine can be calculated by combining the calculation fusion results of personnel and safety helmets.

[0052] For the personnel location bounding box tracking component and the safety helmet location bounding box tracking component of this application, the situation where personnel move too fast in the scene can cause a certain deviation between the actual detected personnel location bounding box and the completed body bounding box in the personnel detection sequence. This deviation can mislead the tracker, resulting in incorrect tracking. To address the above problem, the personnel tracking module can use the safety helmet tracking sequence to guide the matching of the personnel sequence with the candidate target box, thereby eliminating the error caused by this deviation and finally obtaining an accurate tracking sequence for all personnel.

[0053] For the personnel tracking module of this application, the personnel tracking module defines the movement state of the target by calculating the target position block diagram at each time in the personnel and safety helmet tracking sequence and the IOA of the mine entrance and exit. That is, when the IOA of a certain personnel or safety helmet tracking sequence with the mine entrance increases from small to large, the personnel or safety helmet is entering the mine, and vice versa. Furthermore, the counting results of personnel and safety helmets are integrated to improve the accuracy and safety of personnel counting.

[0054] It should be noted that this application is not only applicable to general personnel counting scenarios and personnel entering and exiting wells, but can also be extended to personnel entering and exiting specific areas and vehicle entering and exiting specific areas.

[0055] In summary, the coal mine personnel counting method based on combined target tracking in this embodiment of the invention uses combined target tracking to count the number of personnel entering and leaving the mine, improving the reliability of personnel identification and the accuracy of personnel counting. It can be applied to complex application scenarios of coal mine video surveillance, accurately calculates the movement status of personnel, determines whether personnel have entered or left the mine, and avoids errors in verification and statistical results due to overexposure, dim environment, personnel misidentification, and occlusion. This improves the continuity and accuracy of tracking personnel entering and leaving the mine, defines the entry and exit status of personnel and safety helmets, and merges the statistical results of personnel and safety helmets to count the number of personnel entering and leaving the mine.

[0056] The above specific embodiments should not be construed as limiting the scope of protection of the present invention. For those skilled in the art, any alternative improvements or modifications made to the embodiments of the present invention shall fall within the scope of protection of the present invention.

[0057] Any aspects of this invention not described in detail are well-known to those skilled in the art.

Claims

1. A coal mine personnel in and out of shaft counting method based on combined target tracking, characterized in that, The counting method includes the following steps: S1, the counting system starts and enters working status, and selects the mine shaft entrance area in the video surveillance; S2, capture the corresponding video stream from the obtained surveillance video and send it to the target detection module for real-time inference. The target detection module outputs the personnel detection location box and the safety helmet detection location box in each frame of the image. S3, input the output personnel detection location box and safety helmet detection location box into the personnel tracking module, and output the target tracking sequence of the personnel and the target tracking sequence of the safety helmet; S4 uses the position status of the safety helmet to assist in the matching score of personnel tracking; S5, calculate the IOA between the personnel location detection box and the wellhead region in each frame of the target tracking sequence for each personnel; S6, calculate the IOA between the safety helmet position detection box and the wellhead region in each frame of the target tracking sequence for each safety helmet; S7, based on the calculation results, defines whether each person and safety helmet should enter the mine; S8 maps the target tracking sequence of personnel to the target tracking sequence of safety helmets, matches the personnel corresponding to each safety helmet according to the positional relationship between personnel and safety helmets, and counts the cases where no match can be made separately. S9, based on the matching results, count the number of people entering the well.

2. The personnel counting method based on combined target tracking for coal mine personnel in and out of shaft according to claim 1, characterized in that: The counting system includes a target detection module and a personnel tracking module.

3. The coal mine personnel counting method based on combined target tracking according to claim 2, characterized in that: The target detection module is a one-stage target detector used to quickly detect the personnel detection location map and the safety helmet detection location map in each frame of the video.

4. The personnel counting method based on combined target tracking for coal mine personnel in and out of shaft according to claim 2, characterized in that: The personnel tracking module uses Kalman filtering to predict the positions of the personnel detection location map and the safety helmet detection location map in subsequent video frames, calculates the overlap ratio with the target detection results of subsequent frames, calculates the matching score based on the overlap ratio, and includes the target detection results with high scores in the target tracking sequence, thereby establishing the target tracking sequence for personnel and safety helmets.

5. The coal mine personnel counting method based on combined target tracking according to claim 2, characterized in that: The personnel tracking module defines the motion state of the tracking sequence by calculating the detection position diagram in the personnel and safety helmet target tracking sequence and the IOA of the mine entrance. The larger the IOA, the more overlapping areas there are, that is, the closer the target is to the mine entrance.

6. A method for counting personnel entering and leaving a coal mine based on combined target tracking according to claim 2, characterized in that: The target detection module includes a personnel detection component and a safety helmet detection component, and the personnel tracking module includes a personnel location box tracking component and a safety helmet location box tracking component.

Images