A coal mine underground person number counting method based on a bipartite graph

Abstract

The present application relates to a kind of coal mine underground number of people statistics method based on two-part graph, belong to artificial intelligence field.The image of coal mine entry or underground key place is used as input data, yolov5 is used as the main network of target detection, interval time T is obtained, and two-part graph of personnel in the coal mine entry area is formed, Hopcroft-Karp algorithm is used to match personnel at T1 and T2 time, and the number of people entering and leaving is counted;The present application is based on yolov5 lightweight personnel detection model, can realize rapid iteration and implementation deployment in field application under the condition of meeting the prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and relates to a method for counting the number of people underground in coal mines based on bipartite graphs. Background Technology

[0002] To ensure safe production in coal mines, limiting the number of people entering the mine is one of the important indicators. Explosion-proof cameras are usually installed at the auxiliary shaft entrance, main shaft entrance, ventilation shaft entrance, or other key underground locations. Relevant algorithms are used to identify personnel entering the mine at the edge of the equipment and count the number of people entering the mine.

[0003] Coal mines have complex underground environments, such as underground roadways and mining faces, where insufficient lighting, dense crowds, and dust or smoke obstruction are common. Therefore, using bipartite graph matching to accurately match targets in time-series images is an important method for conducting underground population statistics.

[0004] Currently, the literature describes methods for counting people under good lighting conditions, but none of them propose or explicitly state that they are based on the complex background environment of underground coal mines. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a method for counting the number of people in underground coal mines based on bipartite graphs, which can accurately track the personnel entering the coal mine and realize the counting of the number of people in underground coal mines under conditions of insufficient light and certain dust and fog obstruction.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] A method for counting the number of people underground in coal mines based on bipartite graphs, the method is as follows:

[0008] Using images of key locations at the coal mine entrance or underground as input data, YOLOv5 is used as the backbone network for object detection. Images are acquired at intervals T to form a bipartite map of personnel in the coal mine entrance area. The Hopcroft-Kapp algorithm is used to match personnel at times T1 and T2 to count the number of people entering the mine.

[0009] The input image is a scene containing people at the observation location. The image is sliced, and YOLOv5 is used as the target detection network. The sliced ​​image is input into the backbone network, and after convolution, residual processing, and upsampling, the location information P of the people is generated. i (x1,x2,y1,y2); x1 is the x-coordinate of the top left corner of the marker box, x2 is the x-coordinate of the bottom right corner of the marker box, y1 is the y-coordinate of the top left corner of the marker box, and y2 is the y-coordinate of the bottom right corner of the marker box;

[0010] To determine the number of people entering and exiting the mine, we identify whether the personnel have passed through the straight line L(x) = kx + b in the image. Line L is used as the area line for entry and exit detection, where k and b are constants and x is the independent variable. We acquire target detection images at times T1 and T2 and determine whether the personnel in the images have moved and crossed the area line L between the two times.

[0011] Figure G1(v1,v2,…,v) i ) and G2(u1,u2,…,u i These are schematic diagrams of personnel identification at times T1 and T2, respectively. Point v in G1... i Let u represent the center point of the position of the i-th person before moving, and the midpoint of G2. i This represents the center point of the position after the i-th person moves; it is the matching set (v1, v2, ..., v i ) and set (u1, u2, ..., u i For the points in the array, the Hopcroft-Kapp algorithm is used to achieve maximum matching;

[0012] In each iteration, the Hopcroft-Kapp algorithm searches for augmenting paths based on the current solution, which is the current match. If an augmenting path exists, it means that the current solution is not yet the optimal solution. After a certain iteration, if no augmenting path can be found, then the current optimal solution, i.e., the maximum match, is found, matching the people in the images at times T1 and T2.

[0013] The matching pattern is based on the assumption that person P is the correct person. i For uniform motion or rest, with v i and u i The distance between position coordinates is used as the criterion for judgment, with a default minimum distance s. i For personnel P i From v i and u i The distance traveled; Figure G1(v1,v2,…,v i ) and G2(u1,u2,…,u i The system iterates and matches each store to find each pair of v. i and u i The best match between E i (v i ,u i );

[0014] After finding the optimal matching combination at times T1 and T2, the calculator P i The direction and distance of movement; if P i If region line L is not crossed, then P i This indicates the product has not entered or exited the well; when P is detected... i For the cross-regional line L, the number of people counted is adjusted according to the direction of crossing.

[0015] Optionally, the image slicing process is as follows: the 608×608×3 image is processed into a 304×304×12 feature map.

[0016] The beneficial effects of this invention are as follows:

[0017] (1) By introducing the principle of bipartite graphs to construct a model of personnel location information at different times, it is possible to effectively analyze personnel location information at different times and explore personnel location relationships.

[0018] (2) The Hopcroft-Kapp algorithm is used to perform path finding and maximum matching in the bipartite graph of personnel, which can improve the association speed and ensure the real-time requirement of personnel statistics.

[0019] (3) Based on the YOLOv5 lightweight personnel detection model, rapid iteration and deployment can be achieved in field applications while meeting the requirements of prediction accuracy.

[0020] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein:

[0022] Figure 1 This is a flowchart of a method for counting the number of people underground in coal mines based on a bipartite graph.

[0023] Figure 2 Match two-part diagrams for people at different times. Detailed Implementation

[0024] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0025] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0026] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0027] like Figure 1 As shown, images of key locations at the coal mine entrance or underground are used as input data. YOLOv5 is used as the backbone network for object detection. Images are acquired at intervals T to form a bipartite map of personnel in the coal mine entrance area. The Hopcroft-Kapp algorithm is used to match personnel at times T1 and T2 to count the number of people entering the mine.

[0028] The input image is a scene of people at the observation location. The image is sliced, for example, a 608×608×3 image is processed into a 304×304×12 feature map. Using YOLOv5 as the object detection network, the sliced ​​image is input into the backbone network and processed through a series of modules including convolution, residual, and upsampling to generate the location information P of the people. i (x1,x2,y1,y2).

[0029] like Figure 2 As shown, to determine the number of people entering and exiting the mine, it is necessary to identify whether personnel pass through the key region L(x,y) in the image, using line L as the region line for entry and exit detection. Target detection images are acquired at times T1 and T2, and it is determined whether personnel in the images have moved and crossed the region line L between the two time points. The underground environment is relatively harsher than the surface environment, making personnel identification more ambiguous. Under poor lighting and occlusion conditions, matching the identified target personnel involves introducing the bipartite graph maximum matching theory to achieve accurate matching.

[0030] Figure G1(v1,v2,…,v) i ) and G2(u1,u2,…,u iThese are schematic diagrams of personnel identification at times T1 and T2, respectively. Point G1 represents information before the personnel move, and point G2 represents information after the personnel move. To accurately match the set (v1, v2, ..., v...),... i ) and set (u1, u2, ..., u i The Hopcroft-Kapp algorithm is used to achieve maximum matching for the points in the given data.

[0031] In each iteration, the Hopcroft-Kapp algorithm searches for augmenting paths based on the current solution (i.e., the current match). If an augmenting path exists, it means the current solution is not yet optimal. After an iteration, if no augmenting path is found, then the current optimal solution (i.e., the maximum match) has been found, thus matching the people in the images at times T1 and T2.

[0032] The matching pattern here is based on the assumption that person P... i For uniform motion or rest, with v i and u i The distance between position coordinates is used as the criterion for judgment, with a default minimum distance s. i For personnel P i From v i and u i The distance traveled. Figure G1(v1,v2,…,v i ) and G2(u1,u2,…,u i The system iterates and matches each store to find each pair of v. i and u i The best match between E i (v i ,u i ).

[0033] After finding the optimal matching combination at times T1 and T2, the calculator P i The direction and distance of movement. If P i If region line L is not crossed, then P i This indicates the product has not entered or exited the well; when P is detected... i For the cross-regional line L, the number of people counted is adjusted according to the direction of crossing.

[0034] By combining the target personnel detection method based on YOLOv5 and introducing the principle of bipartite graphs to achieve personnel matching, we can realize real-time judgment of personnel entering and leaving coal mines and count the number of people in key underground areas in real time.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for counting the number of people underground in a coal mine based on a bipartite graph, characterized in that: The method is as follows: Using images of key locations at the coal mine entrance or underground as input data, YOLOv5 is used as the backbone network for object detection. Images are acquired at intervals T to form a bipartite map of personnel in the coal mine entrance area. The Hopcroft-Kapp algorithm is used to match personnel at times T1 and T2 to count the number of people entering the mine. The input image is a scene containing people at the observation location. The image is sliced, and YOLOv5 is used as the target detection network. The sliced ​​image is input into the backbone network, and after convolution, residual processing, and upsampling, the location information P of the people is generated. i (x1,x2,y1,y2); x1 is the x-coordinate of the top left corner of the marker box, x2 is the x-coordinate of the bottom right corner of the marker box, y1 is the y-coordinate of the top left corner of the marker box, and y2 is the y-coordinate of the bottom right corner of the marker box; To determine the number of people entering and exiting the mine, we identify whether the personnel have passed through the straight line L(x) = kx + b in the image. Line L is used as the area line for entry and exit detection, where k and b are constants and x is the independent variable. We acquire target detection images at times T1 and T2 and determine whether the personnel in the images have moved and crossed the area line L between the two times. Figure G1(v1,v2,…,v) i ) and G2(u1,u2,…,u i These are schematic diagrams of personnel identification at times T1 and T2, respectively. Point v in G1... i Let u represent the center point of the position of the i-th person before moving, and the midpoint of G2. i This represents the center point of the position after the i-th person moves; it is the matching set (v1, v2, ..., v i ) and set (u1, u2, ..., u i For the points in the array, the Hopcroft-Kapp algorithm is used to achieve maximum matching; In each iteration, the Hopcroft-Kapp algorithm searches for augmenting paths based on the current solution, which is the current match. If an augmenting path exists, it means that the current solution is not yet the optimal solution. After a certain iteration, if no augmenting path can be found, then the current optimal solution, i.e., the maximum match, is found, matching the people in the images at times T1 and T2. The matching pattern is based on the assumption that person P is the correct person. i For uniform motion or rest, with v i and u i The distance between position coordinates is used as the criterion for judgment, with a default minimum distance s. i For personnel P i From v i and u i The distance traveled; Figure G1(v1,v2,…,v i ) and G2(u1,u2,…,u i Iterative matching is performed on each point in the array to find each pair of v. i and u i The best match between E i (v i ,u i ); After finding the optimal matching combination at times T1 and T2, the calculator P i The direction and distance of movement; if P i If region line L is not crossed, then P i This indicates the product has not entered or exited the well; when P is detected... i For the cross-regional line L, the number of people counted is adjusted according to the direction of crossing.

2. The method for counting the number of people underground in a coal mine based on a bipartite graph according to claim 1, characterized in that: The image slicing process involves processing a 608×608×3 image into a 304×304×12 feature map.

Images