Construction site violation behavior recognition method and system based on target detection

Abstract

The present application relates to the field of image processing, more particularly, the present application relates to a construction site violation behavior recognition method and system based on target detection, the method comprises: shooting continuous frame images of construction personnel from multiple angles and pre-acquiring template images of correct safety helmet wearing under each angle, rectangular frame labeling of head and safety helmet is carried out; one angle corresponds to a group of to-be-recognized images; for a single angle, the to-be-recognized images are compared with the template images, the behavior violation score of the construction personnel in the to-be-recognized images is calculated, and multi-angle fusion is carried out to obtain a comprehensive violation score; when the comprehensive violation score exceeds a preset violation threshold, the violation behavior is determined and a violation warning is sent to the corresponding construction personnel. The present application effectively overcomes the defects that the traditional method only relies on the boundary box intersection over union which is easily affected by background interference and posture change, and significantly improves the accuracy, robustness and environmental adaptability of the violation behavior recognition.

Description

Technical Field

[0001] This invention relates to the field of image processing. More specifically, this invention relates to a method and system for identifying violations at construction sites based on target detection. Background Technology

[0002] On construction sites, the correct wearing of safety helmets is one of the important measures to ensure the personal safety of construction workers. Traditional construction site safety management mainly relies on manual inspection and video monitoring for manual review. This method has problems such as poor real-time performance, limited coverage, large subjective judgment errors, and high labor intensity, which makes it difficult to meet the needs of modern construction sites for efficient and accurate safety management.

[0003] With the rapid development of computer vision technology, image-based object detection methods are increasingly being applied to safety behavior recognition at construction sites. Existing technologies typically use object detection models (such as YOLO and SSD) to detect "personnel" and "safety helmets" from images or video frames. Then, they calculate the IoU (Intersection over Union) of the bounding boxes of the personnel and safety helmets to determine whether the helmet is being worn correctly. If the IoU between the personnel's bounding box and the safety helmet's bounding box is greater than a preset threshold, it is considered correctly worn; otherwise, it is considered a violation.

[0004] However, actual construction sites are complex environments with workers exhibiting varied postures, including tilting their heads forward or backward. Safety helmets may also be worn incorrectly, such as being worn askew or coming loose. Current technologies rely solely on the IoU (Intersection of Units) of the bounding box as the sole criterion, which has limitations: the bounding box contains not only the target itself but also a significant amount of background. When the background area constitutes a large portion of the target, the IoU may be excessively large or small, leading to misidentification or missed identification. Summary of the Invention

[0005] To address the technical problem that existing technologies identify violations by calculating the IoU (Intersection over Union) using the bounding boxes of the head and helmet, ignoring the influence of the background area within the extracted bounding boxes, resulting in an excessively large or small IoU, leading to misidentification or failure to identify violations, this invention provides solutions in the following aspects.

[0006] In the first aspect, the method for identifying violations at construction sites based on target detection includes: The construction workers were photographed in a series of frames from multiple angles, and template images of the workers wearing safety helmets correctly were pre-collected from each angle. Rectangular boxes were marked on the heads and safety helmets. Each angle corresponds to a set of images to be identified. For a single angle, the image to be identified is compared with the template image to calculate the violation score of the construction workers in the image to be identified, and multi-angle fusion is performed to obtain a comprehensive violation score; When the total violation score exceeds the preset violation threshold, it is judged as a violation and a violation warning is issued to the corresponding construction personnel; The calculation of the behavioral violation score includes: Head key points are extracted from all images to be identified and template images. The spatiotemporal saliency, geometric stability score and recognition information score of each head key point in the image to be identified are calculated. The effectiveness of each head key point in the image to be identified is then calculated by combining the ratio of the geometric stability score to the recognition information score of other head key points in the image to be identified. The head key points of the image to be identified are matched with the head key points of the template image. Edge pixels are obtained from the helmet rectangles of the image to be identified and the template image respectively. The edge pixels are segmented by slope to obtain the first edge sequence and the second edge sequence. The distance between the head key points in the image to be identified and the first edge sequence and the second edge sequence is calculated. Combining the effectiveness, the distance and the IoU between the image to be identified and the template image, the behavior violation score of each image to be identified is calculated.

[0007] Optionally, the calculation of the spatiotemporal saliency includes: Take any head key point in the image to be identified as the center and take the pixel matrix of the preset neighborhood. Calculate the normalized mutual information value between the pixel matrix and the pixel matrix of each head key point in other images to be identified at the same angle, and construct the mutual information sequence. Calculate the standard deviation and mean of the mutual information sequence. Take the negative value of the standard deviation of the mutual information sequence and use it as the exponent of the natural constant for calculation. Multiply the result by the mean of the mutual information sequence to obtain the spatiotemporal significance.

[0008] Optionally, the calculation of the geometric stability fraction includes: The connected component of the safety helmet rectangle in the image to be recognized is extracted to obtain the connected component of the safety helmet. The Euclidean distance between any head key point in the image to be recognized and all pixels in the connected component of the safety helmet is calculated to obtain the distance set. The standard deviation of the distance set is then calculated. The standard deviation of the distance set is negative and used as the exponent of the natural constant for calculation. The result of the calculation is used as the geometric stability score.

[0009] Optionally, the calculation of the identification information score includes: Take any head key point in the image to be identified as the center and take a pixel matrix of a preset neighborhood. Extract the gray values ​​of all pixels in the pixel matrix of the head key point and calculate the standard deviation of the gray values ​​of all pixels as the recognition information score.

[0010] Optionally, the calculation of the degree of effectiveness includes: Calculate the ratio of the geometric stability score of any head keypoint in the image to be identified to the geometric stability score of other head keypoints in the image to be identified, and calculate the ratio of the recognition information score of the head keypoint to the recognition information score of other head keypoints. For each other head keypoint, the ratio of the geometric stability score to the ratio of the recognition information score is added together, and then the sum of the results of all other head keypoints is multiplied by the spatiotemporal saliency to obtain the effectiveness.

[0011] Optionally, matching the head key points of the image to be identified with the head key points of the template image includes: Take any head key point in the image to be identified as the center and take the pixel matrix of the preset neighborhood. Calculate the average difference matrix of the pixel matrix of each head key point in the image to be identified. The average difference matrix is ​​the average value of each pixel in the pixel matrix of the head key point minus the average value of the pixel matrices of all other head key points. The average difference matrix of each head key point in the template image at the same angle is obtained synchronously according to the method of obtaining the average difference matrix of the head key points in the image to be identified. Calculate the mutual information value and multi-scale structural similarity value of the average difference matrix between the head key points in the image to be identified and the head key points in the template image. Select the head key point in the template image with the largest product of mutual information value and multi-scale structural similarity value as the matching key point.

[0012] Optionally, after calculating the distances between the head key points in the image to be identified and the first edge sequence and the second edge sequence, the first geometric sequence and the second geometric sequence are obtained respectively; for matching key points, after calculating the distances between the matching key points and the first edge sequence and the second edge sequence, the third geometric sequence and the fourth geometric sequence are obtained respectively. The DTW algorithm is used to calculate the similarity distance between the first geometric sequence and the third geometric sequence as the first similarity distance, and the similarity distance between the second geometric sequence and the fourth geometric sequence as the second similarity distance.

[0013] Optionally, the calculation of the behavioral violation score includes: The IoU of the head rectangle and helmet rectangle in the image to be identified is divided by the IoU of the head rectangle and helmet rectangle in the template image at the same angle to obtain a ratio. This ratio is then multiplied by the sum of the normalized values ​​of the effectiveness of all head key points and the sum of the corresponding first similarity distance and second similarity distance to obtain the behavior violation score.

[0014] Optionally, the calculation of the overall violation score includes: Normalize the behavior violation scores of all images to be identified under a single angle to obtain a normalized violation score; normalize the IoU of all images to be identified under that angle to obtain a normalized IoU; multiply the normalized violation score of each image to be identified by the normalized IoU, and then sum them up for all images to be identified under that angle to obtain the violation score for that angle. The sum of the IoU of all images to be identified at that angle is used as the recognition weight for that angle; the violation score is multiplied by the recognition weight, and the sum is accumulated over all angles to obtain the comprehensive violation score.

[0015] Secondly, a construction site violation identification system based on target detection includes: a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the construction site violation identification method based on target detection is implemented as described in any one of the claims.

[0016] The present invention has the following beneficial effects: 1. This invention achieves accurate detection of safety helmet wearing behavior at construction sites by constructing a complete recognition framework consisting of "multi-angle image acquisition—spatiotemporal stability assessment of key points—geometric comparison of key points and safety helmet edges—multi-view information fusion." This framework integrates key point stability information from consecutive frames, geometric position information from single frames, and complementary information from multiple perspectives. It effectively overcomes the shortcomings of traditional methods that rely solely on the intersection-over-union (IoU) ratio of bounding boxes and are susceptible to background interference and pose changes, significantly improving the accuracy, robustness, and environmental adaptability of violation behavior recognition.

[0017] 2. This invention automatically selects stable and reliable key points for subsequent analysis by comprehensively evaluating the spatiotemporal saliency, geometric stability, and recognition information score of key points in the head. This avoids misjudgments caused by key point occlusion, blurring, or dynamic changes, thereby improving the anti-interference ability and reliability of the recognition system.

[0018] 3. This invention adopts a multi-angle fusion strategy, which integrates the violation scores and identification weights from various perspectives, eliminates the blind spots that may exist in a single perspective, and ensures that comprehensive and stable judgment results can be obtained even in complex construction environments, thereby improving the practicality and generalization ability of the system. Attached Figure Description

[0019] Figure 1 This is a flowchart of steps S1-S3 in the construction site violation identification method based on target detection according to an embodiment of the present invention.

[0020] Figure 2 This is a flowchart illustrating the calculation of behavioral violation scores in the construction site violation identification method based on target detection, as described in this embodiment of the invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments.

[0022] Reference Figure 1 The method for identifying violations at construction sites based on target detection includes steps S1-S3, as detailed below: S1: Take continuous frame images of construction workers from multiple angles and pre-collect template images of workers correctly wearing safety helmets from each angle, and mark the head and safety helmet with rectangular boxes; one angle corresponds to a set of images to be identified.

[0023] On construction sites, workers' postures are constantly changing, and a single camera's shooting angle can easily lead to the loss of safety helmet features due to occlusion or angular deviation, thus affecting the accuracy of violation identification. Existing technologies typically rely solely on the intersection-over-union (IoU) ratio of the head bounding box and the safety helmet bounding box from a single viewpoint for judgment, which is insufficient for complex scenarios. Therefore, this invention first acquires image sequences of the same target from different perspectives through multi-angle image acquisition, laying the foundation for subsequent multi-view fusion analysis.

[0024] Specifically, multiple fixed cameras are deployed at the construction site to simultaneously capture consecutive frames of images of the same worker from different angles (such as front, side, and obliquely above). Each angle yields a set of images to be identified. Simultaneously, for each shooting angle, a pre-captured image of the worker correctly wearing a safety helmet is used as a template image. The template images are manually annotated: a rectangle is used to outline the head and safety helmet, ensuring the rectangle is the smallest bounding rectangle completely containing either the head or the helmet. The template image and its annotation frame for each angle are saved to the system database.

[0025] By acquiring data from multiple angles, the helmet-wearing status of construction workers in different postures can be captured, avoiding blind spots from a single perspective. The template image provides a "positive sample" benchmark for subsequent comparison, while the labeled rectangles are used to calculate basic geometric features such as IoU.

[0026] At this point, this step has yielded the image sequence to be identified, consisting of each shooting angle taken by the construction workers, the corresponding template image, and the rectangular bounding boxes for the head and safety helmet in the template image. This data forms the basis for subsequent analysis.

[0027] S2: For a single angle, compare the image to be identified with the template image, calculate the violation score of the construction workers in the image to be identified, and perform multi-angle fusion to obtain a comprehensive violation score.

[0028] Construction workers' movements at construction sites are complex and varied, and different actions at different times can lead to changes in head posture. From certain shooting angles, the features of the safety helmet and head may be obscured. Existing technologies rely solely on the IoU (Intersection of Union) of the head and safety helmet bounding boxes to determine violations, which is prone to misidentification or underidentification due to background interference and posture changes. Furthermore, directly using head keypoints for judgment can also lead to misjudgments if the keypoints themselves are unstable (e.g., unreliable due to occlusion, blurring, or dynamic changes). Therefore, this invention first assesses the effectiveness of each head keypoint, then performs a detailed analysis combining the edge geometry of the safety helmet, and finally integrates multi-angle information to calculate a behavioral violation score that accurately reflects the wearing condition.

[0029] Reference Figure 2 The calculation process for the aforementioned violation score includes the following steps S20-S22: S20: Extract head key points from all images to be identified and template images, and calculate the spatiotemporal saliency, geometric stability score and recognition information score of each head key point in the image to be identified. Then, combine the ratio of the geometric stability score to the recognition information score of other head key points in the image to be identified to calculate the effectiveness of each head key point in the image to be identified.

[0030] In consecutive frame images, due to changes in worker movements, some head keypoints (such as the tip of the nose and eyes) may become unreliable due to occlusion, blurring, or drastic changes in posture. Directly using all keypoints for violation analysis can lead to misjudgments. Therefore, it is necessary to evaluate the stability of each keypoint in the spatiotemporal dimensions and its geometric and pixel-level uniqueness, assigning an appropriate validity level to each head keypoint, so that stable and feature-rich head keypoints can play a more important role in subsequent analysis.

[0031] For all images to be identified and template images of any construction worker from each shooting angle, the MediaPipe Pose model is used to extract head key points in each image. Taking any image to be identified from any shooting angle of the construction worker as the target image, the YoLO model is used to detect the head bounding box and safety helmet bounding box in the target image. Any head key point is selected from the head key points of the target image as the target key point, and one head key point is randomly selected from each image to be identified from the same shooting angle as the comparison key point.

[0032] For example, the structure of the existing head keypoint extraction model - MediaPipe Pose model - is as follows: Encoder: Input layer (256×256×3), convolutional layer (128×128×3), BlazeBlock module (64×64×32), BlazeBlock module (8×8×128); Decoder: Upsampled convolutional layer (16×16×96), BlazeBlock module (16×16×96), upsampled convolutional layer (32×32×64), convolutional layer (64×64×10); 3D boosting network: Fully connected layer (512), fully connected layer (256), output layer (10).

[0033] For example, the structure of the selectable model-YoLO model is as follows: Input layer (640×640×3), slice module (320×320×12), convolutional layer (320×320×12), C3 module (320×320×64), convolutional layer (160×160×128), C3 module (80×80×256), SPPE module (40×40×512), convolutional layer (40×40×256), upsampling (80×80×256), C3 module (80×80×256), convolutional layer (160×160×128), downsampling (80×80×128), C3 module (40×40×512), detection layer (80×80×[3×(5+2)]), detection layer (40×40×[3×(5+2)]).

[0034] For each key objective, perform the following operations: First, taking the target key point as the center, a pixel matrix of a preset neighborhood such as 7×7 pixels is taken. The same operation is performed to take the pixel matrix of the comparison key point. The normalized mutual information value of the pixel matrix between the target key point and the comparison key point is calculated. Then, in all the images to be identified from the same shooting angle, the normalized mutual information value between the target key point and all head key points in each image to be identified is calculated. The images to be identified are selected from those with the largest normalized mutual information value to form the mutual information sequence of the target key point.

[0035] Then, the mean of the standard deviations of the above mutual information sequences is calculated. The spatiotemporal saliency of the target keypoints is then calculated based on the mean of the standard deviations of the mutual information sequences, defined as: In the formula, For the spatiotemporal saliency of the target key points, The standard deviation of the mutual information sequence of the target key points. The mean of the mutual information sequence of the target key points.

[0036] By calculating spatiotemporal saliency, the stability and matching degree of target keypoints in consecutive frames are measured. Reflecting the volatility of mutual information values, The smaller The larger the value, the more stable the matching performance of the target keypoint across different frames. It reflects the overall matching level. This allows stable and highly matched key points to acquire greater spatiotemporal significance, thus gaining higher weight in subsequent analyses.

[0037] Next, connected component extraction is performed on the helmet rectangle of the target image to obtain the helmet's connected component. The Euclidean distance between the target keypoints and all pixels within this helmet's connected component is calculated, resulting in a distance set. The standard deviation of the distance set is then calculated, and the geometric stability score of the target keypoints is calculated based on the standard deviation of the distance set, defined as: In the formula, The geometrically stable fraction of the target key point. The standard deviation of the set of distances to the target key points.

[0038] The geometric stability of the target key points relative to the helmet area is measured by calculating their geometric stability scores. The smaller the value, the more concentrated the distances between the target key points and the pixels in the helmet area, meaning the less likely the position of the target key points is to jitter due to pose changes. By using a negative exponential transform, smaller values ​​are... Mapping to larger fractions assigns higher value to key points that are geometrically stable.

[0039] Secondly, extract the gray values ​​of all pixels in the pixel matrix of the target key point, calculate the standard deviation of all gray values, and use the standard deviation of all gray values ​​in the pixel matrix of the target key point as the recognition information score of the target key point.

[0040] By calculating the recognition information score, the degree of pixel variation within the neighborhood (i.e., the pixel matrix) of the target keypoint is evaluated. The larger the standard deviation of the gray values ​​within the neighborhood, the richer the texture in that neighborhood, and the more likely the target keypoint is to be a significant head feature point (such as the corner of the eye or the tip of the nose) rather than a noise point in a flat area. Therefore, by directly using the standard deviation as the score, keypoints with significant features will obtain a higher recognition information score.

[0041] After obtaining the spatiotemporal saliency, geometric stability score, and recognition information score of the target key points through the above operations, it is necessary to further evaluate the merits of the target key points relative to other head key points in the target image.

[0042] Furthermore, the ratio of the geometric stability score of the target keypoint to the geometric stability scores of other head keypoints in the target image is calculated, as is the ratio of the recognition information score of the target keypoint to the recognition information scores of other head keypoints in the target image. The ratio of the geometric stability score reflects the advantage of the target keypoint in geometric positional stability compared to other head keypoints. If the geometric stability of the target keypoint is superior to that of other head keypoints, the ratio is greater than 1, and vice versa. The ratio of the recognition information score reflects the advantage of the target keypoint in feature uniqueness compared to other head keypoints. If the feature uniqueness of the target keypoint is superior to that of other head keypoints, the ratio is greater than 1, and vice versa. If the recognition information score of other head keypoints is 0 (i.e., no change in neighboring pixels), the default ratio of the recognition information score of the target keypoint to that of other head keypoints in the target image is 1.

[0043] Furthermore, for each other head keypoint, the ratio of the geometric stability score of the target keypoint to that of other head keypoints and the ratio of the recognition information score are added together. Then, the sum of the results of all other head keypoints is multiplied by the spatiotemporal saliency to obtain the effectiveness of the head keypoint.

[0044] The above calculations combine the absolute stability of target key points in the temporal dimension with their relative advantages in a single frame image, thus obtaining a quantitative index that comprehensively reflects the reliability of target key points.

[0045] Based on the above operations, each head key point is assigned a validity level, and the higher the value of the head key point, the more suitable it is for subsequent violation analysis.

[0046] S21: Match the head key points of the image to be identified with the head key points of the template image. Obtain edge pixels from the helmet rectangles of the image to be identified and the template image respectively. Perform slope segmentation on the edge pixels to obtain the first edge sequence and the second edge sequence. Calculate the DTW distance between the head key points in the image to be identified and the first edge sequence and the second edge sequence. Combine the calculated validity, DTW distance and IoU between the image to be identified and the template image to calculate the behavior violation score of each image to be identified.

[0047] Traditional methods rely solely on the In-U (IoU) between the head and helmet bounding box to determine violations. However, the bounding box contains a large number of background pixels, and the IoU may not change significantly when the worker's head is tilted or the helmet is worn askew, leading to missed or false detections. Therefore, it is necessary to introduce more refined geometric features, namely the relative positional relationship between key points and the helmet edge, and combine this with the IoU for a comprehensive judgment.

[0048] For the target image above, perform the following operations: First, calculate the average difference matrix of the pixel matrix of the target key point. This average difference matrix is ​​the average value of each pixel in the pixel matrix of the target key point minus the average value of the pixel matrix of all head key points in the target image. Similarly, calculate the average difference matrix of the pixel matrix of each head key point in the template image under the same shooting angle.

[0049] Furthermore, the mutual information value and MS-SSIM (multi-scale structural similarity) value of the average difference matrix between the target keypoint and the head keypoint of the template image are calculated, and the head keypoint of the template image with the largest product of the two is selected as the matching keypoint.

[0050] The purpose of the above matching is to establish the correspondence between the target key points and the head key points in the correct wearing template.

[0051] Next, the Sobel operator is applied to the connected components of the helmet in the target image to extract edge pixels. Any two edge pixels are selected as the start and end points, dividing the edge into two consecutive edge sequences. For each segment, the slope sequence (the slope of the line connecting adjacent pixels) of the two edge sequences is calculated, and then the average slope of each slope sequence is calculated. The segment with the largest absolute value of the difference between the two average slopes is the optimal segment. At this point, the edge sequence with the larger average slope is called the first edge sequence, and the other is called the second edge sequence.

[0052] The above segmentation adaptively separates the upper and lower edges of the helmet, because the slopes of the upper and lower edges are usually quite different (e.g., the tilt directions of the upper and lower edges of the brim are different).

[0053] Next, for each head keypoint in the target image, its Euclidean distance to each pixel in the first edge sequence is calculated to obtain the first geometric sequence. Similarly, its Euclidean distance to each pixel in the second edge sequence is calculated to obtain the second geometric sequence. For the matching keypoints of each head keypoint in the target image, the Euclidean distances to the pixels in the first and second edge sequences are calculated to obtain the third and fourth geometric sequences, respectively.

[0054] Furthermore, the DTW (Dynamic Time Warping) algorithm is used to calculate the similarity distance between the first geometric sequence and the third geometric sequence as the first similarity distance, and the similarity distance between the second geometric sequence and the fourth geometric sequence as the second similarity distance.

[0055] Furthermore, the IoU between the head bounding box and the helmet bounding box in the target image is calculated, as well as the IoU between the head bounding box and the helmet bounding box in the template image at the same shooting angle. Then, the behavior violation score of the target image is calculated based on the validity level, the first similarity distance, and the second similarity distance, satisfying the following relationship: In the formula, The score for behavioral violations in the target image. The IoU between the head bounding box and the helmet bounding box in the target image. To determine the IoU (Intersection over Union) of the head bounding box and the helmet bounding box in the template image taken from the same shooting angle as the target image. This represents the total number of head key points in the target image. For the target image, the first The normalized value of the effectiveness of the first head key point (i.e., the first head key point) The effectiveness of each head keypoint is divided by the sum of the effectiveness of all head keypoints in the target image. The first similarity distance mentioned above, This is the second similarity distance mentioned above.

[0056] in, This reflects the degree of difference in overall overlap between the target image and the template image. When construction workers wear safety helmets correctly, the IoU of the target image should be close to that of the template image, with a ratio approaching 1. When the helmet is not worn correctly (such as being worn crookedly, loosely, or falling off), the IoU of the target image will usually deviate from that of the template image, with a ratio that may be less than or greater than 1. This ratio serves as a global scaling factor, used to adjust the contribution of subsequent local geometric deviations to the behavioral violation score.

[0057] It is a weighted sum of local geometric deviations, reflecting the degree of geometric deviation of all head key points relative to the edge of the helmet.

[0058] When construction workers wear safety helmets in violation of regulations, the geometric distribution of the key points on the head and the edge of the safety helmet will deviate from the template, resulting in a significant increase in the DTW distance. Even if the IoU ratio may be less than 1, the product result will still be large, thus effectively identifying violations.

[0059] Finally, based on the above operations, the behavior violation score of each image to be identified under the same shooting angle can be obtained. This behavior violation score combines the geometric deviation and overall overlap of all head key points in the image to be identified. The larger the value, the higher the probability of violation.

[0060] Furthermore, to improve the robustness of identification, it is necessary to integrate information from multiple shooting angles to comprehensively determine whether the target person has violated regulations.

[0061] The behavior violation scores of all images to be identified under a single shooting angle are normalized (e.g., divided by the maximum behavior violation score within that shooting angle) to obtain a normalized violation score. At the same time, the IoU of all images to be identified under that shooting angle is normalized to obtain a normalized IoU. Then, the normalized violation score and the normalized IoU of each image to be identified are multiplied, and then the results of all images to be identified under that shooting angle are summed to obtain the violation score for that shooting angle. In this way, the contribution of each image to be identified is jointly determined by the degree of violation and the degree of feature exposure under that shooting angle. The more fully the features are exposed, the greater the contribution of the image to the angle violation score.

[0062] Next, the sum of the IoU of all images to be identified under a single shooting angle is used as the recognition weight for that shooting angle.

[0063] Finally, the violation score for a single shooting angle is multiplied by the recognition weight, and the scores for all shooting angles are summed to obtain the comprehensive violation score.

[0064] S3: When the total violation score exceeds the preset violation threshold, it is judged as a violation and a violation warning is issued to the corresponding construction personnel.

[0065] After obtaining the comprehensive violation score, it needs to be converted into specific early warning actions to remind workers to correct violations in a timely manner and prevent accidents from occurring.

[0066] Set a violation threshold, such as 0.6 (the value can be adjusted according to the actual scenario). When the total violation score exceeds the threshold, the system will issue a voice or visual warning to the corresponding construction worker, reminding them to wear their safety helmet correctly. Simultaneously, the system will record the time, location, and images of the violation for future reference.

[0067] The present invention also provides a construction site violation identification system based on target detection. The system includes a processor and a memory, the memory storing computer program instructions. When the processor executes the computer program instructions, it implements the construction site violation identification method based on target detection according to the first aspect of the present invention.

[0068] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0069] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A method for identifying violations at construction sites based on target detection, characterized in that, include: Take continuous frame images of construction workers from multiple angles and pre-collect template images of correctly wearing safety helmets from each angle, and mark the head and safety helmet with rectangular boxes; One angle corresponds to a set of images to be recognized; For a single angle, the image to be identified is compared with the template image to calculate the violation score of the construction workers in the image to be identified, and multi-angle fusion is performed to obtain a comprehensive violation score; When the total violation score exceeds the preset violation threshold, it is judged as a violation and a violation warning is issued to the corresponding construction personnel; The calculation of the behavioral violation score includes: Head key points are extracted from all images to be identified and template images. The spatiotemporal saliency, geometric stability score and recognition information score of each head key point in the image to be identified are calculated. The effectiveness of each head key point in the image to be identified is then calculated by combining the ratio of the geometric stability score to the recognition information score of other head key points in the image to be identified. The calculation of the spatiotemporal saliency includes: taking a pixel matrix of a preset neighborhood centered on any head key point in the image to be identified; calculating the normalized mutual information value between this pixel matrix and the pixel matrices of each head key point in other images to be identified at the same angle; constructing a mutual information sequence; calculating the standard deviation and mean of the mutual information sequence; taking the negative value of the standard deviation of the mutual information sequence as the exponent of the natural constant; and multiplying the calculated result by the mean of the mutual information sequence to obtain the spatiotemporal saliency. The calculation of the geometric stability score includes: extracting the connected components of the safety helmet rectangle in the image to be recognized to obtain the safety helmet connected component; calculating the Euclidean distance between any head key point in the image to be recognized and all pixels in the safety helmet connected component to obtain the distance set; calculating the standard deviation of the distance set; taking the negative value of the standard deviation of the distance set as the exponent of the natural constant for calculation; and using the calculation result as the geometric stability score. The calculation of the recognition information score includes: taking a pixel matrix of a preset neighborhood centered on any head key point in the image to be recognized, extracting the gray values ​​of all pixels in the pixel matrix of the head key point, and calculating the standard deviation of the gray values ​​of all pixels as the recognition information score. The head key points of the image to be identified are matched with the head key points of the template image. Edge pixels are obtained from the helmet rectangles of the image to be identified and the template image respectively. The edge pixels are segmented by slope to obtain the first edge sequence and the second edge sequence. The distance between the head key points in the image to be identified and the first edge sequence and the second edge sequence is calculated. Combining the effectiveness, the distance and the IoU between the image to be identified and the template image, the behavior violation score of each image to be identified is calculated.

2. The method for identifying construction site violations based on target detection according to claim 1, characterized in that, The calculation of the degree of effectiveness includes: Calculate the ratio of the geometric stability score of any head keypoint in the image to be identified to the geometric stability score of other head keypoints in the image to be identified, and calculate the ratio of the recognition information score of the head keypoint to the recognition information score of other head keypoints. For each other head keypoint, the ratio of the geometric stability score to the ratio of the recognition information score is added together, and then the sum of the results of all other head keypoints is multiplied by the spatiotemporal saliency to obtain the effectiveness.

3. The method for identifying construction site violations based on target detection according to claim 1, characterized in that, Matching the head key points of the image to be identified with the head key points of the template image includes: Take any head key point in the image to be identified as the center and take the pixel matrix of the preset neighborhood. Calculate the average difference matrix of the pixel matrix of each head key point in the image to be identified. The average difference matrix is ​​the average value of each pixel in the pixel matrix of the head key point minus the average value of the pixel matrices of all other head key points. The average difference matrix of each head key point in the template image at the same angle is obtained synchronously according to the method of obtaining the average difference matrix of the head key points in the image to be identified. Calculate the mutual information value and multi-scale structural similarity value of the average difference matrix between the head key points in the image to be identified and the head key points in the template image. Select the head key point in the template image with the largest product of mutual information value and multi-scale structural similarity value as the matching key point.

4. The method for identifying construction site violations based on target detection according to claim 3, characterized in that, After calculating the distances between the head key points in the image to be identified and the first edge sequence and the second edge sequence, the first geometric sequence and the second geometric sequence are obtained respectively. for After matching key points and calculating the distances between the matched key points and the first and second edge sequences, the third and fourth geometric sequences are obtained respectively. The DTW algorithm is used to calculate the similarity distance between the first geometric sequence and the third geometric sequence as the first similarity distance, and the similarity distance between the second geometric sequence and the fourth geometric sequence as the second similarity distance.

5. The method for identifying construction site violations based on target detection according to claim 4, characterized in that, The calculation of the behavioral violation score includes: The IoU of the head rectangle and helmet rectangle in the image to be identified is divided by the IoU of the head rectangle and helmet rectangle in the template image at the same angle to obtain a ratio. This ratio is then multiplied by the sum of the normalized values ​​of the effectiveness of all head key points and the sum of the corresponding first similarity distance and second similarity distance to obtain the behavior violation score.

6. The method for identifying construction site violations based on target detection according to claim 5, characterized in that, The calculation of the overall violation score includes: Normalize the behavior violation scores of all images to be identified under a single angle to obtain a normalized violation score; normalize the IoU of all images to be identified under that angle to obtain a normalized IoU; multiply the normalized violation score of each image to be identified by the normalized IoU, and then sum them up for all images to be identified under that angle to obtain the violation score for that angle. The sum of the IoU of all images to be identified at that angle is used as the recognition weight for that angle; the violation score is multiplied by the recognition weight, and the sum is accumulated over all angles to obtain the comprehensive violation score.

7. A construction site violation identification system based on target detection, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement the method for identifying construction site violations based on target detection according to any one of claims 1-6.

Images