Automatic detection method for crowd gathering on construction site

Abstract

The invention discloses an automatic detection method for crowd gathering on a construction site, and the method comprises the following steps: extracting a video frame from a construction site monitoring video stream, recognizing construction site personnel in the video frame through a construction site personnel target detection model, and obtaining the position coordinates of the construction site personnel; converting the video frame into a bird's-eye view by using inverse perspective transformation, and mapping the position coordinates of the construction site personnel to the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view; obtaining the spatial resolution of the aerial view; the actual physical distance between every two construction site personnel is calculated, and a first batch of candidate clusters are screened out; fusing the first batch of candidate clusters to obtain a second batch of candidate clusters; screening out an aggregation cluster from the second batch of candidate clusters; and continuously monitoring the aggregation cluster, and if the continuous existence time of the aggregation cluster is not less than the time threshold value, determining that the crowd aggregation exists. The method can well cope with a complex and changeable environment of a construction site, and realizes rapid and accurate automatic detection of crowd gathering.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and particularly to an automatic detection method for crowd gathering at a construction site. Background Art

[0002] With the rapid development of the construction industry, safety management at a construction site has become one of the key factors for project success. The management of people, as the first element among the five elements of "people, machines, materials, methods, and environment" in construction safety management, directly affects the smooth progress of the project and the safety of personnel.

[0003] During the construction process, especially at large or busy construction sites, situations of abnormal crowd gathering may occur, such as during workers' rest breaks, emergency evacuation drills, or when emergencies occur. The crowd gathering in these situations may not only lead to chaos at the scene, increasing the risk of stampedes and other safety accidents, but also may block rescue channels and affect the efficiency of emergency response. In addition, at the initial stage of the occurrence of abnormal situations, construction site personnel may spontaneously gather. If the phenomenon of crowd gathering can be automatically detected in a timely manner at this stage, effective measures can be taken before the abnormal situation develops into an accident, and potential accidents can be nipped in the bud, thereby avoiding greater casualties and economic losses.

[0004] Traditional monitoring and management at a construction site mainly rely on manual monitoring of video images. This method is extremely vulnerable to factors such as the fatigue, distraction, shift gaps, and differences in personal experience of the monitoring personnel, which may lead to omissions or false alarms during the monitoring process. In addition, the method of manually monitoring video images is difficult to ensure 24-hour uninterrupted effective supervision, and relying on the methods of continuous manual monitoring and subsequent retrieval and judgment makes the supervision efficiency low and the response not timely, and it is impossible to detect and handle abnormal situations in the first place.

[0005] In recent years, the rapid development of visual recognition technology has brought new possibilities for the automatic detection of crowd gathering at a construction site. However, most of the current visual recognition technologies focus on the detection of crowd gathering in public places. When applied to a construction site, the effects are not satisfactory. The environment at a construction site is more complex and changeable than that in ordinary public places, with more occlusions and the situation that workers wear special equipment, etc. These factors significantly increase the detection difficulty and put higher requirements on the existing technologies, resulting in a lower application accuracy of the existing crowd gathering detection methods concentrated on public places at a construction site and being unable to meet the actual needs. Summary of the Invention

[0006] To solve the problem that the application accuracy of automatic crowd gathering detection in the prior art at the construction site is relatively low and cannot meet the actual needs, the present invention provides a method for automatic crowd gathering detection at the construction site, which can well cope with the complex and changeable environment at the construction site, realize fast, accurate and highly adaptable automatic crowd gathering detection, and improve the safety management level at the construction site.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A method for automatic crowd gathering detection at the construction site includes the following steps:

[0009] S101. Extract video frames from the construction site monitoring video stream, use a pre-trained construction site personnel target detection model to identify the construction site personnel in the video frames, and obtain the position coordinates of the construction site personnel;

[0010] S102. Use inverse perspective transformation to convert the video frames into a bird's-eye view, and map the position coordinates of the construction site personnel onto the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view;

[0011] S103. Obtain the spatial resolution of the bird's-eye view;

[0012] S104. Calculate the actual physical distance between every two construction site personnel, and screen out the first batch of candidate clusters;

[0013] S105. Based on similarity, fuse the first batch of candidate clusters to obtain the second batch of candidate clusters;

[0014] S106. Screen out the gathering clusters from the second batch of candidate clusters based on the number threshold;

[0015] S107. Continuously monitor the gathering clusters. If the continuous existence time of the gathering clusters is not less than the time threshold, determine that the gathering clusters are crowd gatherings.

[0016] Further, the method for extracting video frames from the construction site monitoring video stream, using a pre-trained construction site personnel target detection model to identify the construction site personnel in the video frames, and obtaining the position coordinates of the construction site personnel is as follows:

[0017] Extract video frames from the construction site monitoring video stream;

[0018] Collect a large number of photos of construction site personnel to form a data set, mark the specific positions of the construction site personnel in the data set, and use the data set to train the target detection model;

[0019] Use the object detection model trained with the dataset to identify the construction site personnel in the video frame, and obtain the pixel coordinates (u 1 , v 1 ) of the upper left corner of the detection box for the construction site personnel and the pixel coordinates (u 2 , v 2 ) of the lower right corner;

[0020] Define the position coordinates (u, v) of the construction site personnel as the pixel coordinates of the center point of their two feet in the video frame. Then, the pixel coordinates of the midpoint at the bottom of the detection box for the construction site personnel can be used as the position coordinates of the construction site personnel. It can be obtained that u = (u 1 + u 2 ) / 2, v = v 2 .

[0021] Furthermore, use the inverse perspective transformation to convert the video frame into a bird's-eye view, and map the position coordinates of the construction site personnel onto the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view. The specific method is as follows:

[0022] Obtain the perspective transformation matrix M of the construction site monitoring camera, M ∈ R 3×3 , where R is the set of real numbers;

[0023] Based on the perspective transformation matrix M, perform an inverse perspective transformation on the video frame to generate a bird's-eye view B V . At the same time, map the position coordinates (u, v) of the construction site personnel onto the bird's-eye view B V through the inverse perspective transformation, so as to obtain the position coordinates (X V , Y b ) of the construction site personnel on the bird's-eye view B b .

[0024] Furthermore, the specific method for obtaining the perspective transformation matrix M of the construction site monitoring camera is as follows: First, after the construction site monitoring camera is installed and fixed, use the Zhang Zhengyou camera calibration method to obtain the calibration parameters of the construction site monitoring camera, including the internal parameter matrix K 3×3 and the external parameter matrix T 3×4 ; then, on the condition that the depth Z = Z 0 of the main standing plane is assumed, the third column of the external parameter matrix T 3×4 can be directly ignored to obtain the external parameter matrix T 3×3 ; from this, the perspective transformation matrix M 3×3 of the construction site monitoring camera can be calculated as M 3×3 = K 3×3 · T 0Represents the ground height, usually taking the value of 0; the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, usually parallel to the ground, and it is assumed that all reference points for calibration and calculation are located on this plane.

[0025] Furthermore, the specific method for obtaining the perspective transformation matrix M of the construction site monitoring camera is as follows: Based on four-point mapping, obtain the perspective transformation matrix M of the construction site monitoring camera, M ∈ R 3×3 , where R is the set of real numbers.

[0026] Furthermore, based on four-point mapping, the specific method for obtaining the perspective transformation matrix M of the construction site monitoring camera is as follows:

[0027] Manually select and mark four points on the main standing plane in the video frame. The four points are required to have no three points collinear. Record the actual physical coordinates (X W , Y W , Z W ) of the four points in the world coordinate system, and the pixel coordinates (U, V) in the video frame; the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, usually parallel to the ground, and it is assumed that all reference points for calibration and calculation are located on this plane;

[0028] Using the corresponding relationship between the actual physical coordinates and pixel coordinates of the four points pairwise Calculate the perspective transformation matrix M of the construction site monitoring camera, M ∈ R 3×3 .

[0029] Furthermore, based on the perspective transformation matrix M, perform an inverse perspective transformation on the video frame to generate a bird's-eye view B V , and at the same time, map the position coordinates (u, v) of the construction site personnel to the bird's-eye view B through the inverse perspective transformation V to obtain the position coordinates (X V , Y b ) of the construction site personnel on the bird's-eye view B b , and the specific method is as follows:

[0030] Based on the perspective transformation matrix M of the construction site monitoring camera, use inverse perspective transformation to convert the video frame into a bird's-eye view B V , that is, B V = M -1 ·P V , where P V represents the video frame under the perspective V of the construction site monitoring camera;

[0031] Based on the perspective transformation matrix M of the construction site monitoring camera, the position coordinates (u, v) of the construction site personnel are mapped to the bird's-eye view B by using inverse perspective transformation V to obtain the position coordinates (X b , Y b ) of the construction site personnel on the bird's-eye view, that is

[0032] Furthermore, the spatial resolution of the bird's-eye view is obtained, and the specific method is as follows:

[0033] Select two reference points with a known actual physical distance of s on the main standing plane, and record the pixel coordinates of the two reference points in the bird's-eye view as (x 1 , y 1 ) and (x 2 , y 2 ); the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, usually parallel to the ground, and it is assumed that all reference points used for calibration and calculation are located on this plane;

[0034] According to the pixel distance d between the two reference points, the spatial resolution of the bird's-eye view can be calculated

[0035] Furthermore, calculate the actual physical distance between every two construction site personnel and screen out the first batch of candidate clusters, and the specific method is as follows:

[0036] Create an initial cluster with each construction site personnel as the central personnel P i ;

[0037] For each central personnel P i , according to its position coordinates on the bird's-eye view with other construction site personnel P j , calculate the pixel distance d j between it and other construction site personnel P ij ;

[0038] Multiply the pixel distance d i between each central personnel P j and other construction site personnel P ij by the spatial resolution of the bird's-eye view to obtain the actual physical distance D i between each central personnel P j and other construction site personnel P ij ;

[0039] If the actual physical distance D between a certain central personnel P i and other construction site personnel P j pairwise...ij not greater than the distance threshold D th , then add this construction site personnel P j to the initial cluster created by the central personnel P i ; otherwise, do not add; thus, the first batch of candidate clusters are screened out.

[0040] Furthermore, based on the similarity, fuse the first batch of candidate clusters to obtain the second batch of candidate clusters. The specific method is as follows:

[0041] For each pair of clusters ClusterA and ClusterB in the first batch of candidate clusters, calculate the similarity Similarity between them. The similarity Similarity is the ratio of the number of people shared by the two clusters to the number of people in the smaller cluster. The calculation formula for the similarity Similarity is as follows:

[0042]

[0043] where 0 ≤ Similarity ≤ 1;

[0044] If the similarity Similarity between two clusters is not less than the similarity threshold S th , 0 < S th < 1, then fuse these two clusters to form a new cluster;

[0045] Repeat the above steps until there are no more clusters that can be fused, and the second batch of candidate clusters are obtained.

[0046] Furthermore, based on the number threshold, screen out the aggregated clusters from the second batch of candidate clusters. The specific method is as follows:

[0047] Traverse the second batch of candidate clusters. The second batch of candidate clusters contains one or more clusters. For each cluster in the second batch of candidate clusters, if the number of people is not less than the number threshold N th , then consider this cluster as an aggregated cluster; otherwise, consider this cluster not as an aggregated cluster.

[0048] Furthermore, continuously monitor the aggregated clusters. If the continuous existence time of the aggregated clusters is not less than the time threshold, determine that the aggregated clusters are crowd gatherings. The specific method is as follows:

[0049] For each cluster determined to be an aggregated cluster, starting from the video frame when this aggregated cluster is first detected, continuously monitor its status for a certain period of time;

[0050] If within the continuous monitoring time, the continuous existence time of the aggregated cluster is not less than the time threshold T, then determine that the aggregated cluster is a crowd gathering.

[0051] Compared with the prior art, the automatic detection method for crowd gathering at the construction site provided by the present invention has the following beneficial effects:

[0052] (1) The present invention can not only achieve high-precision detection of construction site personnel and position mapping in a complex and changeable construction environment, but also accurately calculate the actual physical distance between construction site personnel.

[0053] (2) The present invention proposes a composite aggregation cluster screening strategy that combines the actual physical distance and the number threshold determination.

[0054] (3) The present invention effectively avoids the false alarm problem caused by short-term aggregation by continuously monitoring and combining the time threshold to verify the continuous existence of the aggregation cluster, and significantly improves the accuracy and reliability of the detection result. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the embodiments or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0056] Figure 1 It is a flowchart of an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary embodiment of the present invention;

[0057] Figure 2 It is a schematic diagram of the position identification of construction site personnel on a video frame according to an exemplary embodiment of the present invention;

[0058] Figure 3 It is a schematic diagram of the main standing plane at a construction site according to an exemplary embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0059] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0060] Please refer to Figure 1 , which is an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary embodiment of the present invention. The method includes the following steps:

[0061] S101. Extract video frames from the construction site surveillance video stream, use a pre-trained construction site personnel object detection model to identify the construction site personnel in the video frames, and obtain the position coordinates of the construction site personnel.

[0062] Specifically, in this embodiment, a pre-trained construction site personnel object detection model can be used to identify the construction site personnel from the video frames extracted from the construction site surveillance video stream, and obtain the position coordinates of each personnel. The identification objects cover all personnel entering the construction site, such as construction workers, management personnel, engineers, visitors, and temporary workers.

[0063] As an embodiment, in step S101, to extract video frames from the construction site surveillance video stream, use a pre-trained construction site personnel object detection model to identify the construction site personnel in the video frames, and obtain the position coordinates of the construction site personnel, the specific method is as follows:

[0064] First, collect a large number of photos containing construction site personnel. These photos should cover various construction scenarios, lighting conditions, and human postures at different angles to ensure that the dataset includes a sufficient variety of samples to cover the complex situations in actual applications. Subsequently, accurately annotate each photo in the dataset, marking the specific positions of all construction site personnel. The annotation work can be completed manually or with semi-automatic tools to ensure that each annotated object has an accurate bounding box. Using the above-mentioned annotated dataset, select a suitable object detection algorithm (such as YOLOv8) to train a construction site personnel object detection model specifically for the complex construction site environment. After training, this model has high precision and efficiency and is suitable for the complex and changeable construction site environment.

[0065] Then, obtain the video stream of the construction site surveillance camera in real time and extract video frames frame by frame as the model input. Each video frame image is input into the pre-trained object detection model, and the model will scan the entire image to find objects that conform to the characteristics of construction site personnel. For each area suspected to be a construction site personnel, the model will output a bounding box and give the confidence score corresponding to this bounding box, indicating the confidence level of the model for this prediction. To remove redundant detection results, improve the positioning accuracy and reduce the false alarm rate, the non-maximum suppression (NMS) technique is used to process multiple overlapping or adjacent bounding boxes. NMS will select the bounding box with the highest confidence score as the final result and ignore other candidate boxes that highly overlap with it but have lower scores.

[0066] Finally, for each valid bounding box confirmed to be a construction site personnel, that is, the finally output detection box, record its upper left pixel coordinates (u 1 , v 1 ) and the lower right pixel coordinates (u2 , v 2 ). To more accurately reflect the actual position of the personnel and adapt to the coplanar assumption of subsequent inverse perspective transformation, the position coordinates (u, v) of the detected construction site personnel are defined as the pixel coordinates of the midpoint of their two feet in the video frame. Specifically, the pixel coordinates of the midpoint at the bottom of the detection box of the construction site personnel are used as the position coordinates, that is, u = (u 1 + u 2 ) / 2, v = v 2 .

[0067] The above technical solution realizes the accurate identification of construction site personnel and the acquisition of position coordinates, providing a reliable basis for subsequent analysis and processing.

[0068] As Figure 2 shown, it is a schematic diagram of the position identification of construction site personnel on a video frame according to an exemplary embodiment of the present invention. For the identified personnel (the schematic diagram only shows one identified personnel, denoted as Personnel A), record the pixel coordinates (u 1 , v 1 ) of the upper left corner and the pixel coordinates (u 2 , v 2 ) of the lower right corner of its detection box, and use the position coordinates (u, v) of the midpoint at the bottom of the detection box as the position coordinates of Personnel A on the video frame.

[0069] Please continue to refer to Figure 1 , which is an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary embodiment of the present invention. This method further includes the following steps:

[0070] S102. Use inverse perspective transformation to convert the video frame into a bird's-eye view, and map the position coordinates of the construction site personnel onto the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view.

[0071] Specifically, in this embodiment, this step includes using inverse perspective transformation technology to convert the video frame into a bird's-eye view and mapping the position coordinates of the construction site personnel onto the bird's-eye view, so as to obtain the accurate position coordinates of these personnel on the bird's-eye view.

[0072] This process first involves the perspective transformation matrix M of the construction site monitoring camera, which is a key element for subsequent transformation calculations. To obtain the perspective transformation matrix M, two different methods are provided in the embodiments of the present application. The first method is obtained based on Zhang Zhengyou camera calibration method, and the second method is obtained based on four-point mapping.

[0073] Among them, the first method is to obtain the perspective transformation matrix M based on Zhang Zhengyou's camera calibration method. Specifically, after the construction site monitoring camera is installed and fixed, Zhang Zhengyou's camera calibration method is used to obtain the calibration parameters of the construction site monitoring camera, including the internal parameter matrix K 3×3 and the external parameter matrix T 3×4 ; Under the condition that the depth Z of the main standing plane is assumed to be Z 0 , the third column of the external parameter matrix T 3×4 can be directly ignored, and the external parameter matrix T 3×3 is obtained; From this, the perspective transformation matrix M of the construction site monitoring camera can be calculated 3×3 = K 3×3 ·T 3×3 ; Among them, Z 0 represents the ground height, usually taking the value of 0.

[0074] Specifically, for the fixed-installed construction site monitoring camera, only one calibration is required during the initial installation. By placing a calibration board with known geometric dimensions (such as a checkerboard) and adjusting the calibration board itself from different positions to ensure that the calibration board covers different parts of the camera's field of view. For a larger range of monitoring cameras, the calibration effect can be improved by arranging multiple points or using a larger calibration board. Using the feature points in these images and their corresponding world coordinates, the internal parameter matrix K 3×3 and the external parameter matrix T 3×4 are solved through an optimization algorithm. In the external parameter matrix T 3×4 , assuming that the depth Z of the main standing plane is Z 0 (usually Z 0 = 0 represents the ground height), at this time, the third column can be ignored and simplified to T 3×3 , and then the perspective transformation matrix M 3×3 = K 3×3 *T 3×3 is calculated. For the adjustable construction site monitoring camera, a comprehensive calibration can be performed during the initial installation, and the internal parameter matrix K 3×3 is saved, and the external parameter matrix is recalibrated and simplified to T 3×3 after each camera adjustment.

[0075] The main standing plane mentioned here and later refers to the main horizontal plane where most construction site personnel stand or move. It is parallel to the ground, and all reference points used for calibration and calculation are located on this plane. As Figure 3 shown, it is a schematic diagram of the main standing plane of a construction site in an exemplary embodiment of the present invention. The area marked by the red polygon frame in the figure represents a main standing plane in this monitoring view.

[0076] The second method is to obtain the perspective transformation matrix M based on four-point mapping. Specifically, four points that are not collinear with each other are manually selected in the main standing plane in the video frame, and the actual physical coordinates (X W , Y W , Z W ) and the pixel coordinates (U, V) in the video frame. Since these points are located on the same horizontal plane, their world coordinates are Z W The value is zero. Then use the correspondence between the actual physical coordinates of these four points and the pixel coordinates Solve the perspective transformation matrix M of the construction site monitoring camera, M∈R 3×3 .

[0077] Furthermore, based on the perspective transformation matrix M obtained by the above method, when the video frame is subjected to inverse perspective transformation to generate a bird's-eye view, the position coordinates of the construction site personnel in the video frame will also be mapped to the bird's-eye view, so that their position coordinates on the bird's-eye view can be obtained (X b , Y b ). Among them, if the video frame under the current viewing angle V of the construction site monitoring camera is represented as P V , then the video frame P V Formula B V =M -1 ·P V Convert to Bird's Eye View B V The specific mapping process of the position coordinates of the construction site personnel in the video frame uses the same perspective transformation matrix M, that is, through the formula The position coordinates (u, v) of the construction site personnel in the original video frame are converted into the position coordinates (X b , Y b ).

[0078] The above technical solution, by converting video frames into a more intuitive bird's-eye view, effectively reduces the detection error caused by occlusion by people or objects, thereby achieving more accurate detection and positioning of each person in the construction site, which is crucial for judging the state of crowd gathering. In addition, the intuitive perspective provided by the bird's-eye view lays the foundation for the subsequent direct calculation of the actual physical distance between people.

[0079] Please continue to see Figure 1 , which is an exemplary embodiment of a method for automatically detecting crowd gathering at a construction site of the present invention, the method further includes the following steps:

[0080] Step S103: Obtain the spatial resolution of the bird's-eye view.

[0081] Specifically, in this embodiment, in order to accurately calculate the spatial resolution of the bird's-eye view, two reference points with a known actual physical distance of s need to be selected on the main standing plane first. The actual physical distance between these two reference points should be measured and recorded in advance. Next, record the pixel coordinates of these two reference points in the bird's-eye view, denoted as (x 1 , y 1 ), and (x 2 , y 2 ). Then, according to the pixel coordinates of these two points, use the formula to calculate the pixel distance d between them. Finally, through the known actual physical distance s and the calculated pixel distance d, the spatial resolution r of the bird's-eye view can be calculated by the formula . r is also the actual physical distance represented by each pixel in the bird's-eye view.

[0082] In the above technical solution, by introducing the calculation of the spatial resolution from the bird's-eye view perspective, it is ensured that subsequent analysis is based on the actual physical distance rather than the perspective-affected pixel distance. Using the actual physical distance to judge the crowd gathering at the construction site is more accurate because people's perception and decision-making are based on the real physical distance. Different from the pixel distance in the image, the actual physical distance is not affected by the perspective effect and can provide a more reliable and intuitive safety assessment.

[0083] Please continue to refer to Figure 1 , which is an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary aspect of the present invention. This method further includes the following steps:

[0084] Step S104: Calculate the actual physical distance between each pair of construction site personnel and screen out the first batch of candidate clusters.

[0085] Specifically, in this embodiment, in order to accurately evaluate the gathering situation among construction site personnel, it is first necessary to calculate the actual physical distance between each pair of construction site personnel based on the position coordinates (X b , Y b ) of the construction site personnel on the bird's-eye view and the spatial resolution r of the bird's-eye view, and then screen out the first batch of candidate clusters based on this actual physical distance. Specifically, this process starts from creating initial clusters. First, create an initial cluster with each construction site personnel as the central person P i . Among them, each initial cluster contains only one central person P i . Then, for each central person P i , according to its position coordinates (X j , Y bi ) and (X bi , Y bj ) on the bird's-eye view with other construction site personnel P bj) Use the formula to calculate its pixel distance d on the bird's-eye view with each other construction site personnel P j pairwise. ij Then, for each central personnel P on the bird's-eye view obtained from the above calculation i and each other construction personnel P j calculate the pairwise pixel distance d ij Multiply it by the spatial resolution r of the bird's-eye view to obtain the actual physical distance D i between each central personnel P j and each other construction site personnel P ij pairwise, that is, D ij = d ij × r; Next, based on the obtained actual physical distance D i between each central personnel P j and each other construction site personnel P ij pairwise and the set distance threshold D th , screen out the first batch of candidate clusters. Specifically, for each central personnel Pi, check the actual physical distance D j between it and each other construction site personnel P ij to see if it is not greater than the set distance threshold D th . If D ij ≤ D th , then add this construction personnel P j to the initial cluster created by the central personnel P i ; if D ij > D th , then do not add. Through the above process, the first batch of candidate clusters can be screened out.

[0086] Please continue to refer to Figure 1 , which is an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary embodiment of the present invention. This method further includes the following steps:

[0087] Step S105: Fuse the first batch of candidate clusters based on similarity to obtain the second batch of candidate clusters.

[0088] Specifically, in this embodiment, for each pair of clusters ClusterA and ClusterB in the first batch of candidate clusters, first calculate their similarity Similarity. The similarity Similarity is defined as the ratio of the number of people shared by the two clusters to the number of people in the smaller cluster, that is:

[0089]

[0090] where 0 ≤ Similarity ≤ 1.

[0091] Then, it is determined whether to perform fusion based on the similarity between them. If the similarity between two clusters is not less than the similarity threshold S th , where 0 < S th < 1, then these two clusters are fused to form a new cluster; otherwise, no fusion is performed.

[0092] Repeat the above process to calculate the similarity and perform the fusion operation for each pair of clusters in all the first - batch candidate clusters until there are no more clusters that can be fused. Finally, the second - batch candidate clusters can be obtained in this way.

[0093] Please continue to refer to Figure 1 , which is an embodiment of an automatic detection method for crowd gathering at a construction site according to an exemplary aspect of the present invention. The method further includes the following steps:

[0094] Step S106: Screen out the gathering clusters from the second - batch candidate clusters based on the number - of - people threshold.

[0095] Step S107: Continuously monitor the gathering clusters screened out in step S106. If the continuous existence time of the gathering cluster is not less than the time threshold, it is determined that the gathering cluster is a crowd gathering.

[0096] Specifically, in step S106 of this embodiment, first traverse the second - batch candidate clusters. For each cluster in the second - batch candidate clusters, check whether the number of people in it is not less than the set number - of - people threshold N th , if the number of people is not less than the number - of - people threshold N th , then consider this cluster as a gathering cluster; otherwise, consider this cluster not to be a gathering cluster.

[0097] Specifically, in step S107 of this embodiment, for each gathering cluster screened out in step S106, starting from the video frame when the gathering cluster is first detected, continuously monitor its status. During the continuous monitoring period, record the existence time of the gathering cluster. If within a set continuous monitoring time, the continuous existence time of the gathering cluster is not less than the time threshold T, then it is determined that the gathering cluster is a crowd gathering.

[0098] Finally, it should be noted that the above - described are only embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structural or equivalent process transformation made using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for automatically detecting crowd gathering at a construction site, characterized in that: The steps include: S101, extracting video frames from a construction site monitoring video stream, using a pre-trained construction site personnel target detection model to identify construction site personnel in the video frames, and obtaining position coordinates of the construction site personnel; S102, converting the video frame into a bird's-eye view by using an inverse perspective transformation, and mapping the position coordinates of the construction site personnel onto the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view; S103, obtaining the spatial resolution of the bird's-eye view; S104, calculating the actual physical distance between the construction site personnel, and screening out the first batch of candidate clusters; S105, merging the first batch of candidate clusters based on similarity to obtain a second batch of candidate clusters; S106, selecting a cluster from the second batch of candidate clusters based on a threshold number of people; S107: Continuously monitor the cluster, and if the cluster persists for a period not less than a time threshold, determine that the cluster is a crowd gathering.

2. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: Extract video frames from the construction site monitoring video stream, use the pre-trained construction site personnel target detection model to identify the construction site personnel in the video frame, and obtain the location coordinates of the construction site personnel. The specific method is as follows: Extract video frames from construction site monitoring video stream; Collect a large number of photos of construction site personnel to form a data set, mark the specific locations of the construction site personnel in the data set, and use the data set to train the target detection model; Use the target detection model trained by the data set to identify the construction site personnel in the video frame, and obtain the upper left corner pixel coordinates (u1, v1) and the lower right corner pixel coordinates (u2, v2) of the construction site personnel detection frame; The position coordinates (u, v) of the construction site personnel are defined as the pixel coordinates of the center points of their feet in the video frame. The pixel coordinates of the bottom midpoint of the construction site personnel detection frame can be used as the position coordinates of the construction site personnel, and u=(u1+u2) / 2, v=v2.

3. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: The video frame is converted into a bird's-eye view by using an inverse perspective transformation, and the position coordinates of the construction site personnel are mapped onto the bird's-eye view to obtain the position coordinates of the construction site personnel on the bird's-eye view. The specific method is as follows: Get the perspective transformation matrix M of the construction site monitoring camera, M∈R 3×3 , R is the set of real numbers; Based on the perspective transformation matrix M, the video frame is subjected to inverse perspective transformation to generate a bird's-eye view B. v At the same time, the position coordinates (u, v) of the construction site personnel are mapped to the bird's-eye view B through the inverse perspective transformation v Thus, the construction site personnel can obtain the bird's-eye view B v The position coordinates (X b , Y b ).

4. The method for automatically detecting crowd gathering at a construction site according to claim 3, characterized in that: The specific method for obtaining the perspective transformation matrix M of the construction site monitoring camera is as follows: after the construction site monitoring camera is installed and fixed, the calibration parameters of the construction site monitoring camera are obtained using Zhang Zhengyou's camera calibration method, including the internal reference matrix K 3×3 and the external parameter matrix T 3×4 ; Then, assuming that the depth Z of the main standing plane is Z0, the external parameter matrix T 3×4 The third column can be ignored directly, and the external parameter matrix T is obtained. 3×3 ; From this, the perspective transformation matrix M of the construction site monitoring camera can be calculated 3×3 =K 3×3 ·T 3×3 ; Wherein, Z0 represents the height from the ground, which is usually taken as 0; the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, which is usually parallel to the ground, and it is assumed that all reference points used for calibration and calculation are located on this plane.

5. The method for automatically detecting crowd gathering at a construction site according to claim 3, characterized in that: The specific method for obtaining the perspective transformation matrix M of the construction site monitoring camera is as follows: based on the four-point mapping, the perspective transformation matrix M of the construction site monitoring camera is obtained, M∈R 3×3 , R is the set of real numbers.

6. The method for automatically detecting crowd gathering at a construction site according to claim 5, characterized in that: The perspective transformation matrix M of the construction site monitoring camera is obtained based on four-point mapping. The specific method is as follows: Manually select and mark four points on the main standing plane in the video frame. The four points require that any three points are not collinear. Record the actual physical coordinates (X, X) of the four points in the world coordinate system. W , Y W , Z W ), and the pixel coordinates (U, V) of the four points in the video frame; the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, which is usually parallel to the ground, and it is assumed that all reference points used for calibration and calculation are located on this plane; Using the correspondence between the actual physical coordinates of the four points and the pixel coordinates Calculate the perspective transformation matrix M of the construction site monitoring camera, M∈R 3×3 .

7. The method for automatically detecting crowd gathering at a construction site according to claim 3, characterized in that: Based on the perspective transformation matrix M, the video frame is subjected to inverse perspective transformation to generate a bird's-eye view B. V At the same time, the position coordinates (u, v) of the construction site personnel are mapped to the bird's-eye view B through the inverse perspective transformation V Thus, the construction site personnel can obtain the bird's-eye view B V The position coordinates (X b , Y b ), the specific method is as follows: Based on the perspective transformation matrix M of the construction site monitoring camera, the video frame is converted into a bird's-eye view B using an inverse perspective transformation. V , that is, B V =M -1 ·P V , where P V represents the video frame from the perspective V of the construction site monitoring camera; Based on the perspective transformation matrix M of the construction site monitoring camera, the position coordinates (u, v) of the construction site personnel are mapped to the bird's-eye view B using inverse perspective transformation V The construction site personnel obtain the aerial view B V The position coordinates (X b , Y b ),Right now 8. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: The spatial resolution of the bird's-eye view is obtained as follows: Select two reference points with a known actual physical distance s on the main standing plane, and mark the pixel coordinates of the two reference points in the bird's-eye view as (x1, y1) and (x2, y2); the main standing plane is the main horizontal plane where most construction site personnel stand or move in the video frame, usually parallel to the ground, and it is assumed that all reference points used for calibration and calculation are located on this plane; According to the pixel distance d between the two reference points, the spatial resolution of the bird's-eye view can be calculated:

9. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: Calculate the actual physical distance between two people on the construction site and select the first batch of candidate clusters. The specific method is as follows: Each construction site personnel is regarded as the core personnel P i Create an initial cluster; For each central person P i , according to its and other construction site personnel P j The position coordinates on the bird's-eye view are calculated to compare the position with other construction site personnel P j The pixel distance between two pixels is d ij ; Each central staff P i With other construction site personnel j The distance between two pixels is d ij Multiplying by the spatial resolution of the bird's-eye view, we get P for each central person i With other construction site personnel j The actual physical distance between two ij ; If a central person P i With other construction site personnel j The actual physical distance between two ij Not greater than the distance threshold D th , then other construction site personnel P j Join the center staff P i In the initial cluster created; Otherwise, it will not be added; thus, the first batch of candidate clusters are screened out.

10. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: The first batch of candidate clusters are merged based on similarity to obtain the second batch of candidate clusters. The specific method is as follows: For each pair of clusters ClusterA and ClusterB in the first batch of candidate clusters, the similarity between them is calculated. The similarity is the ratio of the number of people shared by the two clusters to the number of people in the smaller cluster. The calculation formula of the similarity is as follows: Among them, 0≤Similarity≤1; If the similarity between the two clusters is not less than the similarity threshold S th , 0 th <1, the two clusters are merged to form a new cluster;​ Repeat the above steps until there are no more clusters that can be merged, and obtain the second batch of candidate clusters.

11. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: The clusters are selected from the second batch of candidate clusters based on the number threshold. The specific method is as follows: Traverse the second batch of candidate clusters, which include one or more clusters. For each cluster in the second batch of candidate clusters, if the number of people is not less than the number threshold N th , then the cluster is considered to be an aggregate cluster; otherwise, the cluster is considered not to be an aggregate cluster.

12. The method for automatically detecting crowd gathering at a construction site according to claim 1, characterized in that: Continuously monitor the cluster, and if the cluster persists for a period of time not less than a time threshold, determine that the cluster is a crowd gathering, and the specific method is as follows: For each cluster determined to be a clustered cluster, its status is continuously monitored for a certain period of time starting from the video frame where the cluster is first detected; If the duration of the existence of the cluster is not less than the time threshold T during the continuous monitoring time, the cluster is determined to be a crowd gathering.

Images