People counting method based on underground tunneling face

Abstract

The invention relates to the technical field of people counting, in particular to a people counting method based on an underground tunneling face, and the method comprises the specific steps: carrying out the frame extraction of a video stream of a thermal imaging camera, and obtaining a video frame; performing target detection and semantic segmentation on each acquired video frame to obtain target description containing richer feature information; inputting the target description into an improved target tracking module, and outputting trajectory information of the operator; counting the number of people according to the track information, and giving an alarm if people are overloaded; according to the method, the YOLOv5 model and the BiSeNet model are combined, the target detection precision is improved, a CNN-based appearance feature extractor is replaced with a ResNeSt50 backbone model BoT-based appearance feature extractor in a DeepSORT model, richer appearance feature description is extracted, a missing trajectory is compensated by using a Gaussian smooth interpolation method after target tracking, and the target detection accuracy is improved. And the accuracy and robustness of subsequent target tracking are improved.

Description

Technical Field

[0001] The invention relates to the technical field of people counting, and in particular to a method for counting people based on an underground excavation face. Background Art

[0002] The headcount of underground excavation face workers is an important part of building a smart mine. This link is not only related to the safety and efficiency of mine production, but also a key step in realizing intelligent and refined management of mines. With the advancement of science and technology, especially the widespread application of technologies such as the Internet of Things, big data, and artificial intelligence, the headcount of underground excavation face workers is gradually moving towards intelligence and automation.

[0003] In traditional mine management, the headcount of underground excavation face workers mainly relies on manual roll call or paper records. This method is not only inefficient, but also prone to errors. It is difficult to monitor the dynamics of personnel in real time, which brings hidden dangers to the safe production of mines. With the advancement of smart mine construction, various advanced technical means have been introduced, making the headcount of underground excavation face workers more accurate and efficient.

[0004] For example, personnel positioning technology deploys positioning base stations underground and has workers wear positioning tags, using wireless signals and other technologies to achieve real-time positioning and tracking of workers at the underground excavation face. The intelligent video surveillance system installs high-definition cameras in key areas underground, combined with face recognition and body recognition technology, to automatically identify and count workers entering the excavation face.

[0005] Through the implementation of the above intelligent statistical scheme, the work of counting the number of workers in the underground excavation face has been significantly improved. On the one hand, it improves the accuracy and real-time nature of the statistics and reduces human errors; on the other hand, it enhances the safety management level of the mine and lays a solid foundation for the intelligent and refined management of the mine.

[0006] However, there are also some problems and challenges: personnel positioning technology has problems such as incomplete signal coverage, poor stability, uncertainty about equipment reliability, and high cost; intelligent video surveillance systems have technical problems such as poor imaging quality, limited data processing capabilities, and difficulty in ensuring privacy protection. Summary of the invention

[0007] In order to solve the technical problems existing in the prior art, the present invention provides a method for counting people based on the underground excavation face, which combines target detection and semantic segmentation to enhance the accuracy of target feature expression and is applied to target tracking. The method realizes real-time and accurate counting of the number of workers on the excavation face through efficient target detection, fine semantic segmentation and robust multi-target tracking technology.

[0008] To achieve the above purpose, the technical solution adopted by the present invention is: a method for counting people based on an underground excavation face, the specific steps are as follows:

[0009] Step 1: extract frames from the video stream of the thermal imaging camera to obtain video frames;

[0010] Step 2: Perform target detection and semantic segmentation on each acquired video frame to obtain a target description containing richer feature information;

[0011] Step 3: Input the target description into the improved target tracking module and output the trajectory information of the operator;

[0012] Step 4: Count the number of people based on the trajectory information and sound an alarm if there are more people than allowed.

[0013] The video frames described in step 1 need to be saved as image datasets for model training in steps 2 and 3. Continuous video frames are saved, and the workers are labeled with LabelImg for YOLOv5 model training, and the corresponding location information is generated. The workers are cropped out according to the location information for DeepSORT model training. In particular, overlapping or occluded workers are screened out for BiSeNet model training.

[0014] The target detection and semantic segmentation described in step 2 require the training of YOLOv5 and BiSeNet network models first, and the target detection and semantic segmentation models are trained respectively according to the data set obtained in step 1.

[0015] In the training phase, the YOLOv5 model is first trained using the acquired data set, so that it can accurately identify the workers in the video frame and give a preliminary position and bounding box. At the same time, the BiSeNet model is trained using the data set, so that it can perform semantic segmentation, depict the outline of each person in the video frame and obtain the coordinates of the outline points, thereby obtaining more accurate personnel area and edge information.

[0016] The BiSeNet model structure is mainly divided into three parts: Spatial Path, Context Path and Feature Fusion Module.

[0017] The spatial path module uses convolutional layers, normalization, and ReLU activation functions, aiming to extract low-level detail information of the image, such as edges, textures, etc., to retain sufficient spatial information.

[0018] The context path module first performs fast downsampling through the backbone network to reduce the size of the feature map and increase the receptive field. At the same time, in order to optimize the feature representation, the attention mechanism module (ARM module) is used to weight and recalibrate the features. This module aims to extract high-level semantic information of the image to obtain a larger receptive field.

[0019] The feature fusion module first concatenates the output feature maps of the two paths, and then integrates and optimizes the concatenated feature maps through a series of convolutional layers, batch normalization, and ReLU activation functions. This module efficiently fuses the output feature maps of the spatial path and the context path to achieve complementarity of detail information and semantic information.

[0020] In the inference stage, construct the YOLOv5 detection fusion BiSeNet segmentation module:

[0021] The Input module is mainly responsible for receiving the image to be detected and performing preprocessing operations, such as adjusting the image size and normalization, to prepare for subsequent feature extraction.

[0022] The Backbone module is responsible for extracting the features of the input image. Through multiple convolution and pooling operations, it gradually reduces the size of the feature map and increases the depth of the feature map, thereby extracting feature information at different levels.

[0023] The Neck module connects the backbone network and the head structure, and is responsible for further fusing feature maps from different levels of the backbone network and generating feature maps for detection.

[0024] The Head module is responsible for generating the final detection and segmentation results, including information such as the location, size, and category of the target. The present invention adds three segmentation heads parallel to the detect head in this module and adjusts the parameters so that the model can perform target detection and semantic segmentation tasks simultaneously, thereby improving model efficiency and enhancing feature representation.

[0025] The Output module outputs the detection and segmentation results.

[0026] The target tracking module described in step three adopts the DeepSORT algorithm and improves it to adapt to the special environment of the underground excavation face; the target description generated in step two is input into the improved DeepSORT model, and the model outputs the trajectory information of each operator, including the position, speed, acceleration and other motion states of each operator in different video frames.

[0027] The improved DeepSORT algorithm in step 3 is to optimize the cascade matching module. The improvement of the cascade matching module includes the following two parts:

[0028] (1) Improvement of appearance feature extractor: The deepsort module provided by the present invention adopts BoT (Bag of Tricks) based on the ResNeSt50 backbone model as a new appearance feature extractor. BoT can extract richer and more discriminative appearance features, thereby more effectively distinguishing different target individuals and being more robust in dealing with occlusion and identity recovery when the target reappears.

[0029] (2) Gaussian smoothing interpolation (GSI) after target tracking: The GSI method provided by the present invention aims to interpolate the detections missed during the tracking process, compensate for the missing trajectories caused by occlusion, detection failure, etc., and improve the integrity and accuracy of the trajectory. Its implementation process is divided into the following four parts:

[0030] 1. Define Gaussian process: The process is given by mean m and covariance k.

[0031] 2. Training model: Use known data points (i.e. detected target locations) to train the Gaussian process regression model.

[0032] 3. Predicted location: Predict the location of unknown data points (i.e., missing detections). The prediction result is a probability distribution that represents the uncertainty of the location of the unknown point given the known data points.

[0033] 4. Generate interpolation trajectory: Finally, generate interpolation trajectory based on the prediction results.

[0034] When predicting b from a, for some observed data a, b, we have:

[0035] p(b|a)=N(m(a),K(a))

[0036] Wherein, N() represents normal distribution, m(a) is mean, K(a) is covariance, and p(b|a) is a normal distribution centered at predicted mean m(a) and with predicted variance (derived from covariance function) as width. This distribution captures the prediction of unknown data point b and its uncertainty of the present invention.

[0037] In this case, it is assumed that there are conditional multidimensional normal distributions p(x|t), p(y|t), p(w|t), and p(h|t) between the frame number t and the position information (x, y, w, h), respectively. Then, the present invention estimates the value of (x, y, w, h) at time t when the detection error occurs. Note that a Gaussian process is applied to each trajectory i, specifically denoted as p t .

[0038] p t =f i(t)+∈,where f i ∈GT(0,k(.,.))

[0039] Among them, GT is the Gaussian process model of the object's true motion trajectory; k(.,.) is the kernel function number, which is used to measure the similarity between different time points. The radial basis function (RBF) is usually selected as the kernel function; ∈ represents the observation noise and is assumed to be a zero-mean Gaussian distribution.

[0040] In this process, f i (t) represents the real position of trajectory i at time t, but due to the existence of observation noise ∈, the actual position observed by the present invention will be biased. Therefore, the observed position p t is the real position f i (t) and the sum of noise ∈.

[0041] The expression of the kernel function in the above formula is:

[0042]

[0043] in, Determines the smoothness of the trajectory, set to 10.

[0044] Step 4: Count the number of people according to the trajectory information output by the tracking module. Specifically, by counting the number of workers detected at different time points and combining the trajectory information to determine whether the workers are counted repeatedly, accurate counting of people is achieved. A maximum allowed number threshold is set. When the number of people counted exceeds this threshold, the system automatically triggers an alarm mechanism to remind managers to pay attention to safety risks.

[0045] In addition, the present invention also provides a population counting system based on an underground excavation face, comprising a thermal imaging camera, an image processing server and a user terminal. The thermal imaging camera is responsible for collecting a video stream of the underground excavation face and transmitting the video stream to the image processing server in real time. The image processing server executes the above-mentioned population counting method, processes and analyzes the video stream, and obtains population counting results and alarm information. The user terminal is used to display the population counting results and receive alarm information, so that the management personnel can timely understand the personnel situation of the underground excavation face and take corresponding safety measures.

[0046] The present invention also provides a computer device, including a power supply, a processor, a storage module, an input device, and a display module. The power supply provides stable power to all hardware components to ensure that the hardware components can operate normally and efficiently; the processor implements the above-mentioned people counting method when executing the computer program; the storage module is used to store data and programs; the input device is used to receive external input, such as a keyboard, a mouse, etc., and the user sends instructions and data to the computer through these devices to perform various operations; the display module is used to present the information processed by the computer to the user in a visual form.

[0047] Finally, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method for counting people is implemented.

[0048] In summary, the present invention significantly improves the accuracy and robustness of the headcounting of underground excavation faces by combining target detection and semantic segmentation technology. At the same time, the present invention also provides a complete headcounting system, computer equipment and computer-readable storage medium, providing a new technical means for mine safety management. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A schematic flow chart of a method for counting the number of people at an underground excavation face provided in an embodiment of the present invention.

[0050] Figure 2 This is a structural diagram of the BiSeNet model provided in an embodiment of the present invention.

[0051] Figure 3 This is a structural diagram of the YOLOv5 detection fusion BiSeNet segmentation module provided in an embodiment of the present invention.

[0052] Figure 4 A flowchart for implementing Gaussian smoothing interpolation provided in an embodiment of the present invention.

[0053] Figure 5 A diagram of an improved cascade matching structure in the Deepsort model provided in an embodiment of the present invention.

[0054] Figure 6 A schematic diagram of the hardware structure of a system for counting people at an underground excavation face provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0056] like Figure 1 As shown, the present invention provides a method for counting people based on an underground excavation face, and the specific steps are:

[0057] Step 1: extract frames from the video stream of the thermal imaging camera to obtain video frames;

[0058] Step 2: Perform target detection and semantic segmentation on each acquired video frame to obtain a target description containing richer feature information;

[0059] Step 3: Input the target description into the improved target tracking module and output the trajectory information of the operator;

[0060] Step 4: Count the number of people based on the trajectory information and sound an alarm if there are more people than allowed.

[0061] The video frames described in step 1 need to be saved as image datasets for model training in steps 2 and 4. Continuous video frames are saved, and the workers are labeled with LabelImg for YOLOv5 model training, and the corresponding location information is generated. The workers are cropped out according to the location information for DeepSORT model training. In particular, overlapping or occluded workers are screened out for BiSeNet model training.

[0062] The target detection and semantic segmentation described in step 2 require the training of YOLOv5 and BiSeNet network models first, and the target detection and semantic segmentation models are trained respectively according to the data set obtained in step 1.

[0063] In the training phase, the YOLOv5 model is first trained using the acquired data set, so that it can accurately identify the workers in the video frame and give a preliminary position and bounding box. At the same time, the BiSeNet model is trained using the data set, so that it can perform semantic segmentation, depict the outline of each person in the video frame and obtain the coordinates of the outline points, thereby obtaining more accurate personnel area and edge information.

[0064] The BiSeNet model structure is as follows Figure 2 As shown in the figure, it is mainly divided into three parts: Spatial Path, Context Path and Feature Fusion Module.

[0065] The spatial path module, which uses convolutional layers, normalization, and ReLU activation functions, aims to extract low-level detail information of the image, such as edges and textures, to retain sufficient spatial information.

[0066] The context path module first performs fast downsampling through the backbone network to reduce the size of the feature map and increase the receptive field. At the same time, in order to optimize the feature representation, the attention mechanism module (ARM module) is used to weight and recalibrate the features. This module aims to extract high-level semantic information of the image to obtain a larger receptive field.

[0067] The feature fusion module first concatenates the output feature maps of the two paths, and then integrates and optimizes the concatenated feature maps through a series of convolutional layers, batch normalization, and ReLU activation functions. This module efficiently fuses the output feature maps of the spatial path and the context path to achieve the complementarity of detail information and semantic information.

[0068] In the inference stage, construct the YOLOv5 detection fusion BiSeNet segmentation module, such as Figure 3 As shown:

[0069] The Input module is mainly responsible for receiving the image to be detected and performing preprocessing operations, such as adjusting the image size and normalization, to prepare for subsequent feature extraction.

[0070] The Backbone module is responsible for extracting the features of the input image. Through multiple convolution and pooling operations, it gradually reduces the size of the feature map and increases the depth of the feature map, thereby extracting feature information at different levels.

[0071] The Neck module connects the backbone network and the head structure, and is responsible for further fusing feature maps from different levels of the backbone network and generating feature maps for detection.

[0072] The Head module is responsible for generating the final detection and segmentation results, including information such as the location, size, and category of the target. The present invention adds three segmentation heads parallel to the detect head in this module and adjusts the parameters so that the model can perform target detection and semantic segmentation tasks simultaneously, thereby improving model efficiency and enhancing feature representation.

[0073] The Output module outputs the detection and segmentation results.

[0074] The target tracking module described in step three adopts the DeepSORT algorithm and improves it to adapt to the special environment of the underground excavation face. The target description generated in step two is input into the improved DeepSORT model. The model will output the trajectory information of each operator, including the position, speed, acceleration and other motion states of each operator in different video frames.

[0075] The improvement of DeepSORT algorithm in step 1 is to optimize the cascade matching module, such as Figure 4 As shown:

[0076] In the upper part, on the one hand, the appearance model (BoT) and the motion model (Mahalanobis distance) are used to calculate the similarity to obtain the cost matrix; on the other hand, the gating matrix is ​​obtained through the confidence and the maximum distance to limit the excessive values ​​in the cost matrix.

[0077] The lower part is the data association step of cascade matching. The matching process is a cycle (the number of iterations is the maximum number of mismatches, which is 70 by default), that is, the trajectories with mismatches from 0 to 70 are matched with detections. Trajectories that have not been lost are matched first, and those that have been lost for a long time are matched later. Through this part of the processing, the occluded target can be found again, reducing the number of ID switches that occur when the target is occluded and then appears.

[0078] The improvement of the cascade matching module includes the following two parts:

[0079] (1) Improvement of appearance feature extractor: The deepsort module provided by the present invention adopts BoT based on the ResNeSt50 backbone model as a new appearance feature extractor. BoT can extract richer and more discriminative appearance features, thereby more effectively distinguishing different target individuals and being more robust in dealing with occlusion and identity recovery when the target reappears.

[0080] (2) Gaussian smoothing interpolation (GSI) after target tracking: The GSI method provided by the present invention aims to interpolate the detections missed during the tracking process, compensate for the missing trajectories caused by occlusion, detection failure, etc., and improve the integrity and accuracy of the trajectory. The implementation process is as follows: Figure 5 As shown, it is divided into the following four parts:

[0081] 1. Define Gaussian process: The process is given by mean m and covariance k.

[0082] 2. Training model: Use known data points (i.e. detected target locations) to train the Gaussian process regression model.

[0083] 3. Predicted location: Predict the location of unknown data points (i.e., missing detections). The prediction result is a probability distribution that represents the uncertainty of the location of the unknown point given the known data points.

[0084] 4. Generate interpolation trajectory: Finally, generate interpolation trajectory based on the prediction results.

[0085] When predicting b from a, for some observed data a, b, we have:

[0086] p(b|a)=N(m(a),K(a))

[0087] Where N() represents the normal distribution, m(a) is the mean, and K(a) is the covariance. p(b|a) is a normal distribution centered at the predicted mean m(a) and with the predicted variance (derived from the covariance function) as the width. This distribution captures the prediction of the unknown data point b and its uncertainty.

[0088] In this case, assuming that there are conditional multidimensional normal distributions p(x|t), p(y|t), p(w|t) and p(h|t) between the frame number t and the position information (x, y, w, h), the present invention estimates the value of (x, y, w, h) at time t when the detection error occurs, and applies a Gaussian process to each trajectory i, expressed as p t .

[0089] p t =f i (t)+∈,where f i ∈GT(0,k(.,.))

[0090] Among them, GT is the Gaussian process model of the object's true motion trajectory; k(.,.) is the kernel function number, which is used to measure the similarity between different time points. The radial basis function (RBF) is usually selected as the kernel function; ∈ represents the observation noise, which is assumed to be a zero-mean Gaussian distribution.

[0091] In this process, f i (t) represents the real position of trajectory i at time t, but due to the existence of observation noise ∈, the actual position observed by the present invention will be biased. Therefore, the observed position p t is the real position f i (t) and the sum of noise ∈.

[0092] The expression of the kernel function in the above formula is:

[0093]

[0094] in, Determines the smoothness of the trajectory, set to 10.

[0095] Step 4: Count the number of people according to the trajectory information output by the tracking module. Specifically, by counting the number of workers detected at different time points and combining the trajectory information to determine whether the workers are counted repeatedly, accurate counting of people is achieved. A maximum allowed number threshold is set. When the number of people counted exceeds this threshold, the system automatically triggers an alarm mechanism to remind managers to pay attention to safety risks.

[0096] In order to understand the effect of the improved method of the present invention, an ablation experiment was conducted, and the results are shown in the following table:

[0097]

[0098] In the table, MOTA (Multiple Object Tracking Accuracy) is an indicator that focuses on detection accuracy; IDF1 is an indicator that focuses on ID matching during tracking. The same number of IDs will lead to a long and stable track; Baseline is the original yolov5+deepsort detection and tracking model, and v1, v2, and v3 are various improvements based on Baseline (adding improved modules BiSeNet, BoT, and GSI one by one).

[0099] As can be seen from the table, the addition of the BiSeNet segmentation module significantly improves the accuracy of target detection. The improved appearance feature extractor (BoT) and Gaussian smoothing interpolation (GSI) both significantly improve IDF1 and improve the accuracy of model tracking.

[0100] In addition, the present invention also provides a population counting system based on the underground excavation face, including a thermal imaging camera, an image processing server, and a user terminal. The thermal imaging camera is responsible for collecting the video stream of the underground excavation face and transmitting the video stream to the image processing server in real time; the image processing server executes the above-mentioned population counting method, processes and analyzes the video stream, and obtains the population counting result and alarm information; the user terminal is used to display the population counting result and receive the alarm information, so that the management personnel can timely understand the personnel situation of the underground excavation face and take corresponding safety measures.

[0101] Based on the real-time streaming protocol, video is collected from the video recorder, frames are extracted into images, and the LabelImg tool is used to annotate the images. The annotated images are used for training yolov5, BiSeNet and DeepSORT models to generate trained weight parameter files, which will not be repeated here.

[0102] Put the above parameter files and the improved model files in the same directory, set the unit to automatically start the service, pull the video stream to perform the people counting task. Specifically, it is divided into the following steps:

[0103] (1) Video decoding and frame extraction: Decode the video stream and perform the following operations every 5 frames to detect and track the workers in the underground working area and count the number of people;

[0104] (2) Data preprocessing: adjust the frame image size, normalize, etc. to ensure the consistency of the input data.

[0105] (3) Object detection and overlapping person segmentation: Use Figure 3The yolov5 and BiSeNet fusion modules shown in the figure perform target detection and overlapping person segmentation to improve the detection quality and obtain the location information and confidence of the operators.

[0106] (4) Target tracking: Based on the personnel location information of the detection module, use Figure 4 The improved DeepSORT model shown generates more discriminative appearance features and compensates for the missing trajectories due to occlusion, non-detection, etc. by interpolating trajectories, thereby performing more accurate target tracking.

[0107] (5) People counting: Based on the trajectory information output by the tracking module, count the number of workers detected at different time points, and combine the trajectory information to determine whether the workers are counted repeatedly.

[0108] (6) Set up an alarm mechanism: Set a maximum allowed number of people. When the number of people counted exceeds this threshold, the alarm mechanism is triggered through the user terminal to remind the management staff to pay attention to the security risks.

[0109] This embodiment provides a system for counting people based on an underground excavation face, including:

[0110] The video stream decoding module is used to decode the real-time video stream and extract frames to obtain the image to be detected. It mainly uses a module developed based on opencv, which will not be described here;

[0111] The target detection module is used to detect the operators in the image;

[0112] Target tracking module, used to track the detection target (operator);

[0113] The people counting module is used to count the number of workers and generate an alarm when the number of workers exceeds the limit.

[0114] The present invention also provides a computer device, such as Figure 6 As shown, it includes a power supply, a processor, a storage module, an input device, and a display module. The power supply provides stable power to all hardware components to ensure that all hardware components can operate normally and efficiently. The processor implements the above-mentioned people counting method when executing the computer program. The storage module is used to store data and programs. The input device is used to receive external input, such as a keyboard, a mouse, etc., through which the user sends instructions and data to the computer in order to perform various operations. The display module is used to present the information processed by the computer to the user in a visual form.

[0115] Finally, the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the above-mentioned method for counting people is implemented.

[0116] In summary, the present invention significantly improves the accuracy and robustness of the headcounting of underground excavation faces by combining target detection and semantic segmentation technology. At the same time, the present invention also provides a complete headcounting system, computer equipment and computer-readable storage medium, providing a new technical means for mine safety management.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A method for counting people based on an underground excavation face, characterized in that: The specific steps are as follows: Step 1: extract frames from the video stream of the thermal imaging camera to obtain video frames; Step 2: Perform target detection and semantic segmentation on each acquired video frame to obtain target description; Step 3: Input the target description into the improved target tracking module and output the trajectory information of the operator; Step 4: Count the number of people according to the trajectory information, and if the result of the number count exceeds the set value, an alarm will be triggered.

2. A method for counting people based on an underground excavation face according to claim 1, characterized in that: In step one, the video frames are saved as image datasets and used for model training in steps two and three. Continuous video frames are saved, and the workers are annotated with LabelImg for YOLOv5 model training to generate corresponding location information. The workers are cropped out based on the location information for DeepSORT model training, and the overlapping or occluded workers are screened out for BiSeNet model training.

3. A method for counting people based on an underground excavation face according to claim 2, characterized in that: In step 2, the target detection and semantic segmentation models are trained respectively according to the data set obtained in step 1. In the training stage, the YOLOv5 model is first trained using the obtained data set to identify the workers in the video frame and give a preliminary position and bounding box; the BiSeNet model is trained and semantically segmented using the data set to depict the outline of each person in the video frame and obtain the coordinates of the outline points to obtain the personnel area and edge information; The BiSeNet model consists of a spatial path module, a context path module, and a feature fusion module: The spatial path module uses convolutional layers, normalization, and ReLU activation functions to extract low-level detail information of the image; The context path module downsamples through the backbone network to reduce the size of the feature map and increase the receptive field. The attention mechanism module is used to weight and recalibrate the features to extract high-level semantic information of the image. The feature fusion module splices the output feature maps of the spatial path module and the context path module, integrates and optimizes the spliced ​​feature maps through convolutional layers, batch normalization and ReLU activation functions, and fuses the output feature maps of the spatial path and the context path to achieve the complementarity of detail information and semantic information.

4. A method for counting people based on an underground excavation face according to claim 3, characterized in that: In step 2, in the inference phase, a YOLOv5 detection fusion BiSeNet segmentation module is constructed, which includes: The Input module receives the image to be detected and performs preprocessing operations; The Backbone module extracts the features of the input image and gradually reduces the size of the feature map and increases its depth through multiple convolution and pooling operations, thereby extracting feature information at different levels. The Neck module connects the backbone network and the head structure, further fuses the feature maps of different levels from the backbone network and generates feature maps for detection; The Head module generates the final detection and segmentation results. It adds three segmentation heads in parallel with the detect head and adjusts the parameters so that the model can perform both object detection and semantic segmentation tasks simultaneously. The Output module outputs the detection and segmentation results.

5. A method for counting people based on an underground excavation face according to claim 4, characterized in that: In step three, the target tracking module uses the improved DeepSORT algorithm to input the target description generated in step two into the improved DeepSORT model, and the improved DeepSORT model outputs the trajectory information of each operator.

6. A method for counting people based on an underground excavation face according to claim 5, characterized in that: In step 3, the improved DeepSORT algorithm is to optimize the cascade matching module: Improvement of appearance feature extractor: DeepSORT module adopts BoT based on ResNeSt50 backbone model as the new appearance feature extractor; Gaussian smoothing interpolation is performed after target tracking: interpolation is performed on detections missed during tracking to compensate for missing trajectories. The specific implementation process is divided into the following four parts: 1) Define Gaussian process: The process is given by mean m and covariance k; 2) Training model: Use known data points to train the Gaussian process regression model; 3) Predicted location: Predict the location of unknown data points. The prediction result is a probability distribution, which represents the uncertainty of the location of the unknown point given the known data points. 4) Generate interpolation trajectory: Generate interpolation trajectory based on prediction results.

7. A method for counting people based on an underground excavation face according to claim 6, characterized in that: When predicting b from a, for the observed data a and b, we have: p(b|a)=N(m(a),K(a)) Where N() represents the normal distribution, m(a) is the mean, K(a) is the covariance, and p(b|a) is a normal distribution centered at the predicted mean m(a) and with the predicted variance as its width. Assume that there are conditional multidimensional normal distributions p(x|t), p(y|t), p(w|t) and p(h|t) between the frame number t and the position information (x, y, w, h), respectively, and estimate the value of (x, y, w, h) at time t when the detection error occurs, where x and y represent the object coordinates, w represents the width of the object, and h represents the height of the object. Apply a Gaussian process to each trajectory i, expressed as p t : p t =f i (t)+∈,where f i ∈GT(0,k(.,.)) Among them, GT is the Gaussian process model of the object's true motion trajectory; k(.,.) is the kernel function number, which is used to measure the similarity between different time points, and the radial basis function (RBF) is selected as the kernel function; ∈ represents the observation noise and is assumed to be a zero-mean Gaussian distribution; In this process, f i (t) represents the true position of trajectory i at time t, but due to the existence of observation noise ∈, the observed position p t is the real position f i (t) and the sum of noise ∈; The expression of the kernel function in the above formula is: Among them, x and x ′ are the values ​​of the object's horizontal coordinate x at different frame numbers, exp() is the power function of e, ‖xx ′ ‖ is x and x ′ The Euclidean distance of Determines the smoothness of the trajectory, set to 10.

8. A method for counting people based on an underground excavation face according to claim 7, characterized in that: In step 4, accurate headcount is achieved by counting the number of workers detected at different time points and combining the trajectory information to determine whether the workers are counted repeatedly. A maximum allowed headcount threshold is set. When the number of people counted exceeds this threshold, the system automatically triggers an alarm mechanism.

Images