A safety helmet wearing detection method and system in a construction scene

Abstract

The application discloses a safety helmet wearing detection method and system, which comprises a data acquisition module, a target detection module, a target tracking module and a camera equipment automatic control module, adds on-site monitoring images on the basis of the disclosed safety helmet wearing detection data set, obtains a new data set after data enhancement, constructs a safety helmet wearing detection network, simultaneously optimizes the initial anchor frame and the hyperparameter by using a genetic algorithm, trains the network to obtain a safety helmet wearing detection model, then combines a target tracking algorithm with a monitoring device to realize automatic tracking of the camera equipment on the target working personnel. The tracking algorithm automatically analyzes and extracts the motion trajectory features of the target, considers the interframe correlation, thereby making up for the shortcomings of the visual target detection method, improving the detection accuracy and the identification degree of small targets. The application can effectively solve the problems of small target missed detection in the construction environment and false detection in the complex background, and realizes automatic tracking detection of the camera equipment on the illegal target.

Description

Technical Field

[0001] This invention relates to the field of construction safety technology, specifically to a method and system for detecting the wearing of safety helmets in construction scenarios. Background Technology

[0002] Safety accidents are frequent in the construction industry. Currently, falls from heights account for more than half of all workplace safety accidents. Wearing safety helmets can effectively reduce injuries to workers from falling objects; however, due to the complex environment and high personnel turnover at construction sites, monitoring and managing helmet use is challenging. Therefore, accurately and effectively monitoring helmet use among workers is a crucial measure to reduce head injuries and ensure construction safety.

[0003] With the rise of the concept of smart construction sites, the engineering and construction industry is gradually moving towards informatization and intelligence. In recent years, intelligent helmet-wearing detection methods based on deep learning computer vision have improved in terms of detection accuracy, detection speed, and on-site adaptability. However, due to the complex environment of construction sites, large traffic flow, and frequent occlusion, the detection accuracy of existing technologies remains low, small target detection is difficult, and occlusion problems cannot be handled. At the same time, most monitoring equipment installed on construction sites is at a fixed angle, making it difficult to capture and issue warnings for violations; it is difficult to distinguish whether small targets at a distance are violating regulations; and it is impossible to accurately judge the helmet-wearing status of targets with low confidence levels or located in partially obscured areas. Summary of the Invention

[0004] This invention provides a method and system for detecting the wearing of safety helmets in construction scenarios, which solves the problems of low detection accuracy and difficulty in detecting small targets in current construction scenarios; and addresses the technical problems of frequent false alarms and missed alarms in the detection of safety helmets for construction workers in partially obscured states.

[0005] To solve the above-mentioned technical problems, the present invention provides a method for detecting helmet wearing, comprising the following steps:

[0006] Step S1: Construct a helmet wearing detection dataset;

[0007] Step S2: Improve the YOLOv5 target detection algorithm and construct a helmet-wearing detection network;

[0008] Step S3: Optimize the initial anchor frame size of the helmet wearing detection network using K-means clustering algorithm and genetic algorithm, and optimize the hyperparameters of the helmet wearing detection network using genetic algorithm;

[0009] Step S4: Train the helmet-wearing detection network obtained in step S2 using the helmet-wearing detection dataset to obtain a weight file and construct a helmet detection model;

[0010] Step S5: Based on the safety helmet detection model and DeepSort target tracking algorithm, detect construction site video frame images in real time, annotate the target people in the current frame image to generate target detection boxes; and assign different IDs to the target people, perform feature extraction, continuous tracking and obtain target trajectory;

[0011] Step S6: During continuous tracking, key targets that are not wearing helmets are identified based on the helmet detection model, and the key targets are divided into non-violation targets A and potential non-violation targets B that are difficult to identify. When the key target meets the determination condition of not wearing a helmet for more than 5 consecutive frames, the key target is tracked and magnified for continued detection.

[0012] Step S7: When the key target is found to be in violation of not wearing a safety helmet, the key target is captured and an alert is issued.

[0013] Preferably, step S1 includes the following steps:

[0014] Step S11: Based on the publicly available GDUT-HWD safety helmet detection dataset, add information images after the construction site monitoring video has been collected, filtered, and organized;

[0015] Step S12: Preprocess the information image: randomly flip, rotate, crop, and scale the image horizontally / vertically;

[0016] Step S13: Label the processed images using labeling. Label the helmets as blue, yellow, white, and red. Label the helmets not being worn as none. Construct a helmet wearing detection dataset.

[0017] Preferably, step S2 specifically includes: based on the large, medium and small scale predictions of the YOLOv5 algorithm, the original feature map is further upsampled in the Neck layer and spliced ​​and fused with the shallow feature map containing more detailed information, thereby changing the three-scale prediction structure to a four-scale prediction, and using the deep learning framework PyTorch to build a helmet wearing detection network.

[0018] Preferably, step S3 includes the following steps:

[0019] Step S31: Use the K-means clustering algorithm to cluster the height and width of the target boxes in the helmet wearing detection dataset to obtain twelve initial anchor boxes;

[0020] Step S32: Use a genetic algorithm to optimize the twelve sets of initial anchor boxes, select the best possible recall (BPR) as the fitness function in the optimization process. When the fitness function of the initial anchor box is less than the set value, use a genetic algorithm to optimize the initial anchor box, and select the initial anchor box with the highest fitness function in 1000 rounds of optimization.

[0021] Step S33: Use a genetic algorithm to select the individual with the highest fitness function as the optimal hyperparameter.

[0022] Preferably, step S4 specifically involves: inputting the initial anchor box obtained in step S3 and the hyperparameters obtained in step S4 into the helmet wearing detection network; and inputting the helmet wearing detection dataset into the helmet wearing detection network for training to obtain a weight file and construct a helmet detection model.

[0023] Preferably, the specific steps for training the helmet-wearing detection network are as follows:

[0024] Step S41: Adopt a multi-scale training method, set multiple different image input scales, randomly select one scale from multiple scales during training, scale the input image to that scale and input it into the helmet wearing detection network;

[0025] Step S42: During the training process, when the fitness function of the helmet wearing detection network on the helmet wearing detection dataset no longer increases or decreases, the training is terminated in advance to ensure the training results.

[0026] Preferably, step S5 includes the following steps:

[0027] Step S51: Using the target detection box as input, label the IDs of all targets appearing in the first frame;

[0028] Step S52: Use Kalman filtering to predict the next state of the target and match the ID with the predicted state;

[0029] Step S53: Use a re-identification network to extract deep target features within the target detection box, calculate its feature vector for association with target appearance information in the following text, and match the extracted target features with the ID;

[0030] Step S54: Calculate the Mahalanobis distance and cosine distance between the detection box and the ground truth box, fuse the motion information and appearance information of the target, and obtain the degree of correlation between the detection box and the trajectory;

[0031] Step S55: Use the Hungarian matching algorithm to match the target based on the correlation results. If the match is successful, assign the same ID. If the match is unsuccessful, proceed to the IoU matching stage and continue using the Hungarian algorithm. If the match is successful, assign the same ID and assign a new ID to the detection results of the unmatched trajectory.

[0032] Step S56: For successful matches, update the target state using Kalman filtering and proceed to the prediction of the next frame.

[0033] Preferably, the specific steps for defining and judging key targets in step S6 are as follows:

[0034] Step S611: Set the resolution of the video captured by the camera device to P. CX ×P CY ;

[0035] Step S612: Based on the safety helmet detection model and Deepsort tracking algorithm, obtain all target information in the image: Target = (cls, x, y, w, h, conf), where cls represents the target category, with a value of 0 for non-compliant targets; conf represents the confidence level, which is a decimal between 0 and 1; (x, y) and (w, h) represent the ratio of the target center point coordinates to the width and height of the target box relative to the width and height of the entire image.

[0036] Step S613: Calculate the pixel area S of each target region. T =(x*P cx ):×(y*P cy );

[0037] Step S614: Analyze the target information Target. When target cls = 0, it is a violation target A; when S T ≤22×22 pixels are considered extremely small targets and are classified as case B1; when conf < 0.5, they are considered targets with low confidence and are classified as case B2; B1 and B2 constitute potential illegal targets B.

[0038] Preferably, the specific steps of the method for scaling the key target in step S6 are as follows:

[0039] Step S621: Control the camera device to rotate to the target position and acquire the region of interest S R For the violation target A, the S R S represents the actual size of the detection box; for a potential non-compliant target B, S R Set the pixel range to 200×200 centered on the target;

[0040] Step S622: Set the ideal region S iThe area is one-sixtieth of the captured image;

[0041] Step S623: Set the zoom threshold A:

[0042]

[0043] Step S624: When S i -S R When A > 0, amplification is performed; when 0 ≤ S i -S R When <A, no doubling is required; when Si-S R When <0, reduce;

[0044] Step S625: Repeat the above steps until the size is doubled to a reasonable size.

[0045] The present invention also provides a safety helmet wearing detection system for construction scenarios, including a data acquisition module, a target detection module, a target tracking module, and an automatic control module for camera equipment;

[0046] The data acquisition module, based on the publicly available GDUT-HWD safety helmet detection dataset, adds information images obtained from construction site monitoring video collection, filtering, and processing. It preprocesses the information images and uses labeling to annotate the processed images to construct a safety helmet wearing detection dataset, which is then input into the target detection module.

[0047] The target detection module is used to improve the YOLOv5 target detection algorithm, construct a helmet wearing detection network, improve the training strategy and optimize the hyperparameters of the helmet wearing detection network, train the helmet wearing detection network using the helmet wearing detection dataset, construct a helmet detection model, and preliminarily detect the helmet wearing status of the target person using the helmet detection model, generating target detection boxes and category confidence scores.

[0048] The target tracking module is used to continuously track the target person, combining the DeepSort target tracking algorithm with the safety helmet detection model to detect construction site video frame images in real time, annotate the target person in the current frame image to generate target detection boxes, and assign different IDs to the target person for feature extraction.

[0049] Simultaneously, based on the detection results obtained from the safety helmet detection model, key targets are identified, and the key targets are tracked using the target ID and target trajectory obtained by the tracking algorithm. When the key target meets the determination condition of not wearing a safety helmet for more than 5 consecutive frames, the key target is tracked and magnified, and the key target information is transmitted to the automatic control module of the camera equipment for continued detection.

[0050] The automatic control module of the camera equipment is used to control the camera equipment to rotate to the position of the key target, adjust the zoom of the camera equipment, and detect the key target through the target detection module. When the key target is found to be violating the rule of not wearing a safety helmet, the module will capture the key target and issue an early warning.

[0051] Compared with the closest existing technology, the technical solution provided by the present invention has the following beneficial effects: by improving and optimizing the original YOLOv5 algorithm, a safety helmet detection model is constructed to detect the target person, and the DeepSort target tracking algorithm is used to continuously track the target person, effectively solving the problem of missed detection of small targets in the construction environment and the problem of false detection in complex backgrounds.

[0052] Meanwhile, the target tracking algorithm automatically analyzes and extracts the motion trajectory features of the target person, taking into account the inter-frame correlation, thereby making up for the shortcomings of visual target detection methods and improving detection accuracy and the recognition of small targets. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the improved YOLOv5 network structure of the present invention;

[0054] Figure 2 The flowchart shows the improved YOLOv5 hyperparameters optimized using a genetic algorithm.

[0055] Figure 3 This is a schematic diagram of YOLOv5 multi-scale training.

[0056] Figure 4 Main flowchart for the DeepSort tracking algorithm and helmet wearing detection model;

[0057] Figure 5 Flowchart for identifying key targets;

[0058] Figure 6 This is a schematic diagram of the horizontal field of view of the camera equipment;

[0059] Figure 7 This is a schematic diagram of the vertical field of view of the camera equipment;

[0060] Figure 8 This is a schematic diagram showing the camera's perspective.

[0061] Figure 9 This is a flowchart of the camera and pan-tilt-zoom (PTZ) equipment control process. Specific implementation methods

[0062] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.

[0063] Current methods for detecting whether workers are wearing safety helmets on construction sites still suffer from low accuracy, difficulty in detecting small targets, and inability to handle partial obstruction. Furthermore, most monitoring equipment installed at construction sites is angled at fixed points, making it difficult to capture and issue warnings for violations; it also struggles to distinguish whether small, distant targets are in violation and cannot accurately determine confidence levels, leading to frequent false alarms and missed alarms.

[0064] To address the above problems, this invention provides a method for detecting helmet wearing, comprising the following steps:

[0065] Step S1: Construct a helmet wearing detection dataset;

[0066] This embodiment is based on the publicly available GDUT-HWD safety helmet detection dataset. It then adds filtered and organized construction site video surveillance images to the dataset and preprocesses the images to obtain a dataset suitable for construction sites.

[0067] The dataset collection in this embodiment takes into account the differences in construction site scenes under different times, lighting conditions, and weather conditions. The images were collected from construction sites with dense crowds and complex environments, and were filtered and designed to increase the pixel information of small targets.

[0068] The image preprocessing methods described above include, but are not limited to, the following:

[0069] Flip Enhancement: Flips the image horizontally or vertically;

[0070] Rotation enhancement: Rotates the image around a point by a certain angle;

[0071] Resize enhancement: Scaling the image to ensure the model has scale invariance;

[0072] Noise enhancement: Adds noise to the image;

[0073] This enriches the dataset to some extent, increases the robustness of the detection system, and improves the accuracy of monitoring small targets.

[0074] Finally, the dataset obtained above is labeled with the following labels: blue for safety helmets, yellow for safety helmets, white for safety helmets, red for safety helmets, and none for safety helmets, thus constructing a safety helmet wearing detection dataset.

[0075] Step S2: Improve the YOLOv5 target detection algorithm and construct a helmet-wearing detection network;

[0076] like Figure 1 The diagram illustrates the process of improving the YOLOv5 object detection algorithm using the deep learning framework PyTorch and constructing a helmet-wearing detection network.

[0077] Specifically, based on the large, medium, and small scale predictions of the original YOLOv5 algorithm, a tiny target detection layer is added. That is, the original feature map is upsampled in the Neck part of the YOLOv5 algorithm and then spliced ​​and fused with the shallow feature map containing more detailed information. This optimizes the original three-scale prediction structure of YOLOv5 into a four-scale prediction, thereby improving the detection accuracy of tiny targets.

[0078] Step S3: Optimize the initial anchor frame size of the helmet wearing detection network using K-means clustering algorithm and genetic algorithm, and optimize the hyperparameters of the helmet wearing detection network using genetic algorithm;

[0079] The helmet-wearing detection network constructed in step S2 has an added minimal target detection layer, making the original nine initial anchor frames of the YOLOv5 algorithm no longer suitable for the helmet detection task of this invention.

[0080] This embodiment first uses the K-means clustering algorithm to cluster the width and height of the target bounding boxes in the helmet wearing detection dataset obtained in step S1, obtaining twelve initial anchor boxes. The specific steps are as follows: based on the width and height of the initial anchor boxes, twelve cluster centers are randomly generated, and the IOU distance between each target ground truth bounding box in the dataset and its neighboring cluster centers is calculated using the following formula.

[0081] D(box,cluster)=1-IOU(box,cluster)

[0082] In the formula, IOU(box,cluster) represents the IOU value between the target ground truth box and the cluster center, and D(box,cluster) represents the IOU distance between the target ground truth box and the cluster center.

[0083] Assign the ground truth bounding box to the nearest cluster center, based on the size (W) of the ground truth bounding box corresponding to each cluster center. ground_truth Hground_truth The cluster center values ​​are recalculated using the following formula to obtain twelve new initial anchor frames (W). i H i );

[0084]

[0085] In the formula, n represents the number of all true target boxes under the current calculated cluster center.

[0086] After obtaining twelve initial anchor boxes through K-means clustering, a genetic algorithm is used to optimize them. The best possible retrieval rate (BRP) is selected as the fitness function during the optimization process. BRP represents the ratio of the maximum number of targets that the model can recall to the total number of true frames. When the fitness function of the initial anchor boxes is less than a set value of 0.98, the genetic algorithm is used to further optimize the initial anchor boxes, selecting the initial anchor boxes with the highest fitness after 1000 evolution rounds. This results in initial anchor boxes that are more practical and relevant to actual construction sites, leading to higher detection accuracy.

[0087] The original YOLOv5 algorithm has a large number of hyperparameters. To accelerate the model's convergence speed and improve its generalization ability and accuracy, this embodiment uses a genetic algorithm to find the most suitable combination of hyperparameters in a high-dimensional space.

[0088] like Figure 2 As shown, the main steps of hyperparameter optimization based on genetic algorithms include:

[0089] 1) Determine the parameters to be optimized and their value range. Twenty-nine hyperparameters were selected as the hyperparameters to be optimized, including learning rate, initial momentum, loss function weight coefficients, IoU threshold, data augmentation coefficient, and probability of Mosaic augmentation.

[0090] 2) Construct the fitness function. The fitness function of an individual is:

[0091]

[0092] Among them, T x，i Evaluation metrics for object detection include precision (P), recall (R), mAP50, and mAP50:95. x，i These represent the weights of each indicator. A larger fitness function indicates better model performance.

[0093] 3) To prevent the best individuals from disappearing, which would prevent the genetic algorithm from converging to the global optimum, an elite retention strategy is adopted to avoid destroying high-quality individuals during the evolution process. Specifically, the n individuals (n≤5) with the highest fitness function are retained during the evolution process, while individuals with low fitness functions are eliminated.

[0094] 4) Crossover and mutation. Mutated individuals are randomly selected from the retained individuals; individual x... k The probability of being selected, P(x) k Its fitness is directly proportional to its value, that is:

[0095]

[0096] Employing Gaussian mutation helps to escape local optima, improving search accuracy and speed. This involves adding a random term following a Gaussian normal distribution to the selected individuals, i.e.:

[0097] X k =R×[X k ⊙(1+G⊙N)]

[0098] In the formula, N represents a Gaussian-compliant random disturbance vector, and the individual x k Let G represent a 29-dimensional hyperparameter vector, G represent a vector representing the mutation scale of each hyperparameter, ⊙ represent the element-wise distribution of the vector, and R represent a random number between 0 and 1.

[0099] 5) The hyperparameter vector generated by the mutation is used in the helmet-wearing detection network. After training for 200 epochs, the model's precision (P), recall (R), mAP50, and mAP50:95 metrics on the helmet-wearing detection dataset are calculated, and the individual's x is calculated accordingly. k The fitness function;

[0100] 6) Repeat the above steps. When the iteration conditions are met, the evolution ends, and the individual with the highest fitness function is selected as the optimal hyperparameter.

[0101] Step S4: Input the above helmet wearing detection dataset into the above helmet wearing detection network for training, obtain the weight file, and construct the helmet detection model;

[0102] Specifically, the initial anchor boxes calculated in step S3 and the hyperparameters optimized by the genetic algorithm in step S4 are first input into the helmet wearing detection network. The helmet wearing detection network is then trained using the training set obtained in step S1 to obtain a weight file and construct a helmet detection model. The helmet detection model is then used to perform real-time recognition and detection of the input video to obtain the size, category, and confidence of the target box in the video frame.

[0103] like Figure 3 As shown, during the training process, several different image input scales are set. During training, one scale is randomly selected from multiple scales, the input image is scaled to that scale, and then input into the helmet-wearing detection network. By setting different scales, the method is simple and effectively improves the accuracy of multi-scale object detection.

[0104] Although each iteration uses a single scale, each iteration is different, increasing the network's robustness without excessively increasing computational cost. During testing, to obtain more accurate detection results, the scale of the test image can be enlarged, for example, by a factor of 4, which avoids showing too many small objects.

[0105] Multi-scale training is a highly effective trick that amplifies the scale of small objects while increasing the diversity of objects at multiple scales, and can be directly embedded in multiple detection algorithms. In each training round, images from each training batch are read, and interpolation is used to upsample and downsample the images to enlarge or reduce the input image.

[0106] Meanwhile, this embodiment employs an early stopping mechanism. Early stopping is simple in form, highly effective, and widely used in deep learning model training. When the fitness function of the helmet-wearing detection network on the helmet-wearing detection dataset no longer increases or even decreases, the training process is terminated early to prevent the network from learning useless features and avoid overfitting caused by repeated training. To avoid overfitting, training is terminated when the fitness function no longer improves after 30 consecutive rounds.

[0107] Step S5: Based on the safety helmet detection model and the DeepSort target tracking algorithm, detect construction site video frame images in real time, annotate the target people in the current frame image to generate target detection boxes, assign different IDs to the target people, perform feature extraction, continuous tracking and obtain target trajectory;

[0108] DeepSort is a tracking algorithm for multi-target tracking. The algorithm first assigns an ID to each target in the frame. Then, it constructs a motion state model of the target using Kalman filtering and employs a cascaded matching strategy for target association. Finally, it uses the Hungarian algorithm to perform optimal allocation between the detection results and the tracking trajectory frame by frame. Furthermore, DeepSort introduces a target re-identification model to improve recognition accuracy. This embodiment uses the target detection bounding boxes obtained by the aforementioned helmet detection model as input and utilizes the DeepSort algorithm for target tracking, such as... Figure 4 As shown, the specific steps are as follows:

[0109] 1) ID encoding: The target detection box obtained by the safety helmet detection model is used as input, and all target figures appearing in the first frame are labeled with IDs in sequence;

[0110] 2) State prediction part: Use Kalman filtering to predict the next state of the target person and match the ID with the predicted state;

[0111] 3) Feature extraction: A re-identification network is used to extract deep target features within the target detection box, and its feature vector is calculated for association with target appearance information. The extracted target features are then matched with the ID.

[0112] 4) Correlation Calculation: Calculate the Mahalanobis distance and cosine distance between the detected bounding box and the ground truth bounding box, fuse the target's motion information and appearance information, and obtain the correlation degree c between the detected bounding box and the trajectory. i，j ;

[0113] 5) Feature matching: matching the degree of association c i，j The result uses the Hungarian matching algorithm to match the target. If the match is successful, the same ID is assigned. If the match is unsuccessful, the process enters the IOU matching stage, and the Hungarian algorithm is used again. If the match is successful, the same ID is assigned. New IDs are assigned to the detection results of the unmatched trajectories, and the trajectories that are not matched are further judged.

[0114] 6) State update: For successful matches, the target state is updated using Kalman filtering and the prediction proceeds to the next frame.

[0115] It should be noted that the purpose of tracking in this embodiment is to acquire the trajectory of the target, and then call the camera device to move the key target to the center of the image and zoom in. Specifically, the camera device captures and tracks the key target in the distance, and completes the tracking and trajectory acquisition of the key target in the distance within a limited tracking time. Then, through the movement and zooming of the camera device, it is transformed into close-up tracking, and the camera device acquires more pixel information, thus completing the detection and tracking more effectively and accurately.

[0116] Step S6: During continuous tracking, identify non-wearing targets and potential non-wearing targets based on the helmet detection model. When a target meets the criteria for not wearing a helmet for more than 5 consecutive frames, zoom in and capture the target.

[0117] In this embodiment, targets are divided into two categories: key targets and non-key targets. Key targets are further divided into two categories: one is violation target A, and the other is potential violation target B, which is difficult to identify. Target B includes targets with minimal risk (B1) and targets with low confidence (B2). Figure 5 The definition and determination method of the key targets shown are as follows:

[0118] Set the resolution of the video footage captured by the camera device to P. cx ×P cy The pixel-based system monitors the current camera's captured image in real time using a helmet detection model and the Deepsort tracking algorithm, outputting information on all targets in the current image. The output format for the target information (Target) is as follows:

[0119] Target = (cls, x, y, w, h, conf)

[0120] In the formula, cls represents the target category, with a value of 0 for violating targets; conf is the confidence level, a decimal between 0 and 1; (x,y) and (w,h) represent the ratios of the target center point coordinates to the width and height of the target bounding box relative to the width and height of the entire image. Figure 7 As shown, x, y, w, and h are calculated using the following formulas:

[0121]

[0122] In the formula, x1, x2, y1, and y2 represent the pixel coordinates of the target border, and w and h represent the width and height of the target pixel, respectively.

[0123] The pixel area of ​​each target region is ST = (x * P) cx )×(y*P cy Since the targets in the open construction site are relatively small, in order to ensure the uniformity of target distribution, a category of extremely small targets has been added to the dataset partitioning criteria. The target partitioning criteria adopted in this embodiment are as follows:

[0124] 1) Minimal target: S T ≤22×22 pixels;

[0125] 2) Small target: 22×22 pixels < S T ≤32×32 pixels;

[0126] 3) Medium target: 32×32 pixels < S T ≤96×96 pixels;

[0127] 4) Large target: 96×96 pixels < S T .

[0128] In the collected target information Target, if cls = 0, it is determined to be case A; if S T If the pixel size is ≤22×22 pixels, it is determined as B1; if conf < 0.5, it is determined as B2.

[0129] like Figure 5 As shown, the specific workflow of the key target identification system is as follows:

[0130] 1) Based on the safety helmet detection model and DeepSort tracking algorithm, obtain the target ID and corresponding location information, category information, confidence level, and other information;

[0131] 2) Perform key target analysis on the information corresponding to the target ID, identify key targets and non-key targets, and classify key targets into categories A, B1, and B2. At this time, the status of the target person is pending.

[0132] 3) Track key targets. If a target person corresponding to a certain ID meets the criteria of wearing a safety helmet for more than 5 consecutive frames, the pending state will be changed to the confirmed state.

[0133] 4) Control the camera equipment and pan-tilt equipment to further inspect key targets and take photos as evidence.

[0134] During the tracking of the aforementioned key targets, since the targets are often not located in the center of the frame, it is necessary to control the camera and pan / tilt equipment to rotate to a designated position to center the target in the frame. Then, the image can be zoomed in and out using the zoom function to obtain a clearer image. When the camera lens is fixed, the maximum range that the camera can capture is usually expressed as the field of view (FOV). The FOV is the angle formed by the line connecting the edge of the image to the camera's origin, representing the area that the lens can cover. The larger the FOV, the larger the area the image can cover.

[0135] Horizontal field of view such as Figure 6 As shown, ∠DOE represents the maximum horizontal field of view, with an angle of ±θ; the vertical field of view is as follows: Figure 7 As shown, ∠AOB represents the maximum vertical field of view, with an angle of...

[0136] To avoid frequent rotation of the camera equipment, this embodiment sets a rotation threshold. The camera equipment only starts rotating when the pixel Euler difference between the target and the center of the image exceeds a certain threshold. This threshold is set as P. T The formula is as follows:

[0137]

[0138] In the formula, x represents the horizontal coordinate of the target center point, y represents the vertical coordinate of the target center point, and P cx and P xy These represent the width and height resolution of the image, respectively. When the center of the target exceeds this threshold, the camera is controlled to rotate. By calculating the change in the angle of the target relative to the center of the field of view in the image, the camera is controlled to rotate by the corresponding angle in real time to complete the capture of the target.

[0139] like Figure 8 The diagram shows the rotation of the gimbal device. ∠FOC is set to +θ, ∠EOC to -θ, and ∠AOC to... ∠BOC is Field of view θ, The specific values ​​can be obtained from the information transmitted by the device.

[0140] According to the Pythagorean theorem:

[0141]

[0142] Based on the coordinate difference between the target and the center of the image, establish the following equation:

[0143]

[0144] Calculate the horizontal angle α that the gimbal needs to be adjusted:

[0145]

[0146] Vertical angle β:

[0147]

[0148] When the calculated α and β are positive, they represent turning to the right and turning upward, respectively; when α and β are negative, they represent turning to the left and moving downward, respectively.

[0149] Once the camera is rotated to the designated position, the region of interest S can be acquired. R For case A, S R This indicates the actual size of the detection box; for cases B1 and B2, S R Set the target center to a range of 200×200 pixels.

[0150] Then, the region of interest S was analyzed. R Enlarge S R The area is scaled up to a reasonable size so that the target can be displayed at a suitable size in the image. The ideal area after scaling is S. i This embodiment, derived from extensive experiments and field data collection, defines the ideal region S after magnification. i The optimal size is set to one-sixtieth of the area of ​​the image being captured.

[0151]

[0152] Therefore, the scaling factor can be determined:

[0153]

[0154] To prevent errors caused by changes in the target itself and device latency during zooming, a zoom threshold A is set.

[0155]

[0156] When the pixel area of ​​the region of interest is S R With the ideal region pixel area S iWhen the difference reaches the zoom threshold A, the gimbal is controlled to zoom, and the zoom coefficient γ is adjusted.

[0157] Obtain the region of interest S of the key target R Then, the zoom level is first adjusted, and after zooming is completed, the region of interest S is redefined. R To determine whether a multiplication needs to be applied, the following rules apply:

[0158] 1) When S i -S R When the value is greater than A, it indicates that the new region of interest is too small and differs from the ideal region. It needs to be magnified by zooming in. In this case, zooming in increases the size of the target on the screen, magnifying it to a reasonable size.

[0159] 2) When 0 ≤ S i -S R When <A, it indicates the target's size and appropriate size within the image. Zooming is not necessary in this case.

[0160] 3) When S i -S R When the value is less than 0, it indicates that the region of interest is too large and needs to be reduced by zooming in. In this case, zooming out will make the target appear smaller on the screen, reducing it to a reasonable size.

[0161] Since the rotation of the equipment takes time and the target's position changes during operation, in order to prevent the camera equipment from failing to capture the target effectively and accurately after rotation and zoom, this embodiment introduces a feedback control method. By comparing the deviation between the feedback value and the ideal value, the current parameters and state of the camera equipment are adjusted to achieve the purpose of precise control.

[0162] like Figure 9 As shown, the specific process steps are as follows:

[0163] 1) Equipment parameter acquisition. Obtain the current horizontal deflection angle α, vertical deflection angle β, and magnification factor γ of the equipment;

[0164] 2) Calculation of target information. From the target information, the center point coordinates (x, y) of the key target are read, and the area S of the region of interest is set. R ;

[0165] 3) Rotation threshold determination. Calculate the pixel Euler distance between the center of the target and the center of the image to determine if it exceeds the threshold P. T ;

[0166] 4) Magnification threshold discrimination. Calculate the area S of the region of interest for the key target. R And determine the area S of the region of interest. R With the ideal area S iDoes the difference exceed the scaling threshold A?

[0167] 5) Calculation of control quantity μ. If the current horizontal deflection angle and vertical deflection angle are α and β respectively, and the scaling factor is γ, the required further horizontal and vertical rotation angle increments are α1 and β1 respectively, and the scaling increment is γ1;

[0168] 6) Target information update. Reacquire the target's trajectory and information, calculate the target's center point coordinates, and set the final horizontal and vertical deflection angles as α' = α + α1 and β' = β + β1, with the final scaling factor as γ' = γ × γ1.

[0169] By adjusting the gimbal's deflection angle and the camera's zoom level, the key target is moved to the center of the image for magnification and further monitoring, thereby enabling facial capture of illegal targets and monitoring of blurred targets.

[0170] The present invention also provides a safety helmet wearing detection system for construction scenarios, including a data acquisition module, a target detection module, a target tracking module, and an automatic control module for camera equipment;

[0171] The data acquisition module, based on the publicly available GDUT-HWD safety helmet detection dataset, adds information images from construction site monitoring videos that have been collected, filtered, and organized. The information images are preprocessed and labeled using labeling to construct a safety helmet wearing detection dataset, which is then input into the target detection module.

[0172] The object detection module is used to improve the YOLOv5 object detection algorithm, construct a helmet wearing detection network, improve the training strategy and optimize the hyperparameters of the helmet wearing detection network, train the helmet wearing detection network using a helmet wearing detection dataset, and construct a helmet detection model; and use the helmet detection model to initially detect the helmet wearing status of the target person, and generate object detection boxes and category confidence scores.

[0173] The target tracking module is used to continuously track target individuals. It combines the DeepSort target tracking algorithm with the safety helmet detection model to detect construction site video frames in real time, annotate target individuals in the current frame to generate target detection boxes, and assign different IDs to target individuals for feature extraction.

[0174] Meanwhile, based on the detection results obtained from the safety helmet detection model, key targets are identified, and the key targets are tracked using the target ID and target trajectory obtained by the tracking algorithm. When a key target meets the criteria of not wearing a safety helmet for more than 5 consecutive frames, the key target is tracked and magnified, and the key target information is transmitted to the automatic control module of the camera equipment for continued detection.

[0175] The automatic control module for the camera equipment is used to control the camera equipment to rotate to the position of the key target, adjust the zoom of the camera equipment, and detect the key target through the target detection module. When the key target is found to be violating the rule of not wearing a safety helmet, the module will capture the key target and issue an early warning.

[0176] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting the wearing of safety helmets in a construction scenario, characterized in that, Includes the following steps: Step S1: Construct a helmet wearing detection dataset; Step S2: Improve the YOLOv5 target detection algorithm and construct a helmet-wearing detection network; Step S3: Optimize the initial anchor frame size of the helmet wearing detection network using K-means clustering algorithm and genetic algorithm, and optimize the hyperparameters of the helmet wearing detection network using genetic algorithm; Step S4: Train the helmet-wearing detection network obtained in step S2 using the helmet-wearing detection dataset to obtain a weight file and construct a helmet detection model; Step S5: Based on the safety helmet detection model and DeepSort target tracking algorithm, detect construction site video frame images in real time, annotate the target people in the current frame image to generate target detection boxes; and assign different IDs to the target people, perform feature extraction, continuous tracking and obtain target trajectory; Step S6: During continuous tracking, key targets that are not wearing helmets are identified based on the helmet detection model, and the key targets are divided into non-violation targets A and potential non-violation targets B that are difficult to identify. When the key target meets the determination condition of not wearing a helmet for more than 5 consecutive frames, the key target is tracked and magnified for continued detection. Step S7: When the key target is found to be in violation of not wearing a safety helmet, capture an image of the key target and issue an alert; Step S2 specifically includes: based on the large, medium and small scale predictions of the YOLOv5 algorithm, the original feature map is further upsampled in the Neck layer and spliced ​​and fused with the shallow feature map containing more detailed information, thereby changing the three-scale prediction structure to a four-scale prediction, and using the deep learning framework PyTorch to build a helmet wearing detection network.

2. The method for detecting the wearing of safety helmets in a construction scenario according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: Based on the publicly available GDUT-HWD safety helmet detection dataset, add information images after the construction site monitoring video has been collected, filtered, and organized; Step S12: Preprocess the information image: randomly flip, rotate, crop, and scale the image horizontally / vertically; Step S13: Label the processed images using labeling. Label the helmets as blue, yellow, white, and red. Label the helmets not being worn as none. Construct a helmet wearing detection dataset.

3. The method for detecting the wearing of safety helmets in a construction scenario according to claim 1, characterized in that, Step S3 includes the following steps: Step S31: Use the K-means clustering algorithm to cluster the height and width of the target boxes in the helmet wearing detection dataset to obtain twelve initial anchor boxes; Step S32: Use a genetic algorithm to optimize the twelve sets of initial anchor boxes, select the best possible recall (BPR) as the fitness function in the optimization process. When the fitness function of the initial anchor box is less than the set value, use a genetic algorithm to optimize the initial anchor box, and select the initial anchor box with the highest fitness function in 1000 rounds of optimization. Step S33: Use a genetic algorithm to select the individual with the highest fitness function as the optimal hyperparameter.

4. The method for detecting the wearing of safety helmets in a construction scenario according to claim 1, characterized in that, Step S4 specifically involves: inputting the initial anchor boxes obtained in step S3 and the hyperparameters obtained in step S4 into the helmet wearing detection network; and inputting the helmet wearing detection dataset into the helmet wearing detection network for training to obtain a weight file and construct a helmet detection model.

5. The method for detecting the wearing of safety helmets in a construction scenario according to claim 4, characterized in that, The specific steps of the training are as follows: Step S41: Adopt a multi-scale training method, set multiple different image input scales, randomly select one scale from multiple scales during training, scale the input image to that scale and input it into the helmet wearing detection network; Step S42: During the training process, when the fitness function of the helmet wearing detection network on the helmet wearing detection dataset no longer increases or decreases, the training is terminated in advance to ensure the training results.

6. The method for detecting the wearing of safety helmets in a construction scenario according to claim 1, characterized in that, Step S5 includes the following steps: Step S51: Using the target detection box as input, label the IDs of all targets appearing in the first frame; Step S52: Use Kalman filtering to predict the next state of the target, and match the ID with the predicted state; Step S53: Use a re-identification network to extract deep target features within the target detection box, calculate its feature vector for association with target appearance information in the following text, and match the extracted target features with the ID; Step S54: Calculate the Mahalanobis distance and cosine distance between the detection box and the ground truth box, fuse the motion information and appearance information of the target, and obtain the degree of correlation between the detection box and the trajectory; Step S55: Use the Hungarian matching algorithm to match the target based on the correlation results. If the match is successful, assign the same ID. If the match is unsuccessful, proceed to the IoU matching stage and continue using the Hungarian algorithm. If the match is successful, assign the same ID and assign a new ID to the detection results of the unmatched trajectory. Step S56: For successful matching results, use Kalman filtering to update the target state and proceed to the prediction of the next frame to obtain the target trajectory.

7. The method for detecting the wearing of safety helmets in a construction scenario according to claim 1, characterized in that, The definition and judgment method of key targets in step S6 are as follows: Step S611: Set the resolution of the video captured by the camera device to P. CX ×P CY ; Step S612: Based on the safety helmet detection model and Deepsort tracking algorithm, obtain all target information in the image: Target=(cls,x,y,w,h,conf), where cls represents the target category, and the value of a violation target is 0; conf represents the confidence level, which is a decimal between 0 and 1. (x,y) and (w,h) represent the coordinates of the target center point and the ratio of the width and height of the target bounding box to the width and height of the entire image; Step S613: Calculate the pixel area of ​​each target region. ; Step S614: Analyze the target information Target. When target cls=0, it is a violation target A; when S T ≤22×22 pixels, is considered a very small target, and is classified as case B1; When conf < 0.5, it is a target with low confidence and is judged as case B2; B1 and B2 constitute potential violation target B.

8. The method for detecting the wearing of safety helmets in a construction scenario according to claim 7, characterized in that, The method for tracking and zooming in on the key target in step S6 is as follows: Step S621: Control the camera device to rotate to the target position and acquire the region of interest S R For the violation target A, the S R Indicates the actual size of the detection frame; For potential violation target B, S R Set the pixel range to 200×200 centered on the target; Step S622: Set the ideal region S i The area is one-sixtieth of the captured image; Step S623: Set the zoom threshold A: ； Step S624: When S i -S R When A > 0, amplification is performed; when 0 ≤ S i -S R When <A, no doubling is required; when Si-S R When <0, reduce; Step S625: Repeat the above steps until the value is doubled to 0 ≤ S i -S R <A range.

9. A safety helmet wearing detection system for construction scenarios, applicable to the method described in any one of claims 1 to 8, characterized in that: include Data acquisition module, target detection module, target tracking module, and automatic control module for camera equipment; The data acquisition module, based on the publicly available GDUT-HWD safety helmet detection dataset, adds information images obtained from construction site monitoring video collection, filtering, and processing. It preprocesses the information images and uses labeling to annotate the processed images to construct a safety helmet wearing detection dataset, which is then input into the target detection module. The target detection module is used to improve the YOLOv5 target detection algorithm, construct a helmet wearing detection network, improve the training strategy and optimize the hyperparameters of the helmet wearing detection network, train the helmet wearing detection network using the helmet wearing detection dataset, construct a helmet detection model, and preliminarily detect the helmet wearing status of the target person using the helmet detection model, generating target detection boxes and category confidence scores. The target tracking module is used to continuously track the target person, combining the DeepSort target tracking algorithm with the safety helmet detection model to detect construction site video frame images in real time, annotate the target person in the current frame image to generate target detection boxes, and assign different IDs to the target person for feature extraction. Simultaneously, based on the detection results obtained from the safety helmet detection model, key targets are identified, and the key targets are tracked using the target ID and target trajectory obtained by the tracking algorithm. When the key target meets the determination condition of not wearing a safety helmet for more than 5 consecutive frames, the key target is tracked and magnified, and the key target information is transmitted to the automatic control module of the camera equipment for continued detection. The automatic control module of the camera equipment is used to control the camera equipment to rotate to the position of the key target, adjust the zoom of the camera equipment, and detect the key target through the target detection module. When the key target is found to be violating the rule of not wearing a safety helmet, the module will capture the key target and issue an early warning.

Images