A construction site electric welding operation mask wearing state automatic detection method
By combining YOLOv8 object detection and multi-label attribute classification models, the real-time and accuracy issues of mask wearing status detection in electric welding operations are solved, achieving efficient and accurate detection of mask wearing status of electric welding workers, and adapting to complex construction environments and multi-person operation scenarios.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 武汉数字建造产业技术研究院有限公司
- Filing Date
- 2026-02-24
- Publication Date
- 2026-07-14
AI Technical Summary
Current safety monitoring for electric welding operations relies on manual inspections, which suffers from high labor costs, poor real-time performance, frequent false alarms, insufficient detection accuracy, and poor environmental adaptability. In particular, it is difficult to accurately detect the mask-wearing status of electric welding workers in complex construction environments.
By combining the YOLOv8 target detection model and the multi-label attribute classification model, video streams are acquired through cameras, and after adaptive preprocessing, the working status and mask wearing status are collaboratively determined. Combined with associated target clipping and dynamic window differential voting, the mask wearing status of welding workers can be accurately detected.
It achieves high-precision, low-false-rate detection of the mask-wearing status of welding workers in complex environments, adapts to multi-person operation scenarios, improves the real-time performance and consistency of detection, and reduces labor costs.
Smart Images
Figure CN122391937A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the intersection of computer vision, deep learning, and safety production monitoring technologies, and in particular to an automatic detection method for the wearing status of welding masks at construction sites, applicable to safety monitoring in various welding operation scenarios such as petrochemical sites, power construction, and machinery manufacturing. Background Technology
[0002] Electric welding is a common metal processing technique on construction sites. The process generates dangerous factors such as arc light and spatter, which can easily cause injury to the face and eyes of workers. Safety regulations require welders to wear protective masks correctly; however, in actual construction, due to factors such as working space and ease of operation, it is common for welders not to wear masks.
[0003] Current safety monitoring of electric welding operations mainly relies on manual inspections, which has three major drawbacks: First, the labor cost is high, requiring dedicated personnel to conduct continuous inspections, making it difficult to cover large-area and multi-regional simultaneous operation scenarios; second, the real-time performance is poor, as there are time intervals in manual inspections, making it impossible to detect instantaneous violations in a timely manner; and third, it is highly subjective, easily leading to missed violations due to blind spots and human negligence.
[0004] With the development of computer vision technology, some automated detection solutions have emerged, but they still have obvious shortcomings:
[0005] Firstly, existing detection models mostly use single target detection or single attribute classification methods, without considering the correlation between working status and wearing status, resulting in frequent false alarms in non-working status.
[0006] Secondly, the target cropping and region division rules are scattered, and different steps use different proportions of frame expansion strategies, which causes computational redundancy and affects detection consistency.
[0007] Third, the attribute determination rules are too simplistic and do not combine the target spatial relationship with the classification model confidence for fusion and correction, resulting in insufficient detection accuracy;
[0008] Fourth, the pre-processing process has poor adaptability to complex construction environments, and the target detection rate is low in scenarios such as overexposure, low light, and noise interference.
[0009] Therefore, there is an urgent need for an automated detection solution that can collaboratively determine the operational status and mask wearing status, adapt to complex environments, have a low false alarm rate, and high real-time performance, in order to overcome the shortcomings of existing technologies. Summary of the Invention
[0010] The purpose of this invention is to provide an automatic detection method for the wearing status of welding masks at construction sites, which solves the problems of high manpower consumption, poor real-time performance, and insufficient accuracy of existing monitoring methods, and realizes automatic and accurate detection of the wearing status of welding protective masks.
[0011] To achieve the above objectives, the technical solution adopted by the present invention is as follows:
[0012] An automatic detection method for the wearing status of welding masks at construction sites includes the following steps:
[0013] S1. Real-time video stream of welding operations is collected through cameras at the construction site. Frames are extracted at a preset analysis frame rate of 5-10 FPS to generate images to be inspected. Multiple continuously extracted frames are combined into batch processing data.
[0014] S2. Perform an adaptive preprocessing procedure for welding scenarios on the image to be inspected to obtain standardized input data;
[0015] S3. Input the standardized input data into the pre-trained YOLOv8 target detection model and simultaneously output the bounding boxes, confidence scores and category IDs of four types of targets: person, head, mask and welding spark.
[0016] S4. Perform human-centric related target cropping processing to generate a cropped image that includes the worker's entire body and surrounding potential related areas;
[0017] S5. Input the cropped image into a pre-trained multi-label attribute classification model and output the classification results and confidence scores of the mask wearing attribute and the work status attribute.
[0018] The S6 and YOLOv8 target detection models and multi-label attribute classification models are weighted and fused together to jointly determine the mask wearing attribute and the work status attribute.
[0019] S7: Assign unique IDs to personnel for tracking, implement dynamic window-based differentiated voting for judgment, and stably trigger unsafe work status.
[0020] S8. Use the alarm cache table to deduplicate alarm cycles, generate alarm events with attribute details, and output the final result.
[0021] Furthermore, S2 specifically includes the following steps:
[0022] S201. Adaptive histogram equalization technology is used to dynamically adjust the contrast of the image to be inspected, and Gaussian filtering is used for smoothing and noise reduction.
[0023] S202. Take the image to be inspected after contrast adjustment and smoothing and denoising in S201 as the source image, calculate the scaling factor of its resolution to the 640×640 network input resolution, and perform edge filling on the image that does not fill the 640×640 size after scaling to generate an affine transformation matrix.
[0024] S203. Extract the uint8 pixel values of the scaled and filled image in S202, convert them to float32 type, normalize them to the 0 to 1 range, subtract the mean according to the YOLOv8 object detection model training settings, divide by the standard deviation, and obtain the standardized input data.
[0025] Furthermore, S4 specifically includes the following steps:
[0026] S401. First, calculate the intersection-over-union (IoU) ratio between the bounding boxes of the workers and the bounding boxes of the associated targets.
[0027] The bounding box of the associated target is the bounding box of the protective mask or welding spark, and the intersection-union ratio (IoU) of the bounding boxes is calculated according to the following formula:
[0028] (1)
[0029] Where A is the bounding box area of the worker, and B is the bounding box area of the associated target (protective mask or welding sparks). Let the area be the union of A and B. Let A be the area of the intersection of A and B;
[0030] S402. If there are protective masks or welding sparks around the worker's bounding box, and the association judgment condition of IoU ≥ preset threshold is met, then the worker's bounding box is expanded based on the bounding rectangle of the worker's bounding box and the bounding box of the associated target, so that the clipping area completely includes the worker and all associated targets. The boundary coordinates of the expanded worker's bounding box are calculated according to the following formula:
[0031] The horizontal coordinate x of the top left corner of the expanded worker's bounding box min =min(the horizontal coordinate of the top left corner of the expanded worker's bounding box x) min The horizontal coordinate x of the top-left corner of the bounding box of all associated targets after expansion. min );
[0032] The vertical coordinate y of the top left corner of the expanded worker's bounding box min =min(vertical coordinate y of the top left corner of the expanded worker's bounding box) min The vertical coordinate y of the top-left corner of the bounding box of all associated targets after expansion. min );
[0033] The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =max(the horizontal coordinate of the bottom right corner of the expanded worker bounding box x) max The horizontal coordinate x of the bottom right corner of the bounding box of all associated targets after expansion. max );
[0034] The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =max(the vertical coordinate of the bottom right corner of the expanded worker bounding box) max The vertical coordinate y of the bottom right corner of the bounding box of all associated targets after expansion. max );
[0035] S403. If there are no protective masks or welding spark targets around the worker's bounding box, or if all protective masks and welding spark targets do not meet the association judgment condition of IoU ≥ preset threshold, then the length and width of the worker's bounding box are each proportionally increased by 2 times, with the center of the worker's bounding box as a fixed point. The boundary coordinates of the enlarged worker's bounding box are calculated according to the following formula:
[0036] The width W of the original worker's bounding box is equal to the horizontal coordinate x of the bottom right corner of the original worker's bounding box. max - The horizontal coordinate x of the top left corner of the original operator's bounding box min ;
[0037] The original worker's bounding box height H; the vertical coordinate y of the bottom right corner of the original worker's bounding box. max - The vertical coordinate y of the top left corner of the original operator's bounding box min ;
[0038] The expanded worker's bounding box width W' = the original worker's bounding box width W × 2;
[0039] The height H' of the expanded worker's bounding box is equal to the height H × 2 of the original worker's bounding box.
[0040] The horizontal coordinate x of the top left corner of the expanded worker's bounding box min =Horizontal coordinates of the center point of the original operator's bounding box - W' / 2;
[0041] The vertical coordinate y of the top left corner of the expanded worker's bounding box min =Vertical coordinates of the original worker's bounding box center point - H' / 2;
[0042] The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =Horizontal coordinates of the center point of the original operator's bounding box + W' / 2;
[0043] The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =Vertical coordinates of the center point of the original worker's bounding box + H' / 2;
[0044] S404. If the bounding box coordinates of the enlarged worker exceed the original image pixel range, i.e., x... min <0, y min <0, x max >Original image width, y max If the original image height is greater than the specified value, then the coordinates of the excess portion will be corrected to the original image boundary coordinates, i.e., x... min Corrected to 0, y min Corrected to 0, x max Corrected to the original image width, y max The image height is corrected to the original height, resulting in a cropped image that includes the worker's entire body and surrounding potentially related areas.
[0045] Furthermore, S5 specifically includes the following steps:
[0046] S501. Input the cropped image into a pre-trained multi-label attribute classification model. The multi-label attribute classification model adopts a ResNet50 dual-branch architecture. The ResNet50 backbone network is initialized with weights pre-trained on the ImageNet dataset. The parameters of the first 30 layers are frozen and the last 19 feature extraction layers are fine-tuned. A global average pooling layer is connected after the backbone network to output a 2048-dimensional feature vector. The classification head is set to two parallel classification branches, one for mask wearing attributes and the other for work status attributes. The model has a fixed input size of 640×640 pixels.
[0047] S502. Perform single-frame or batch inference. Input the preprocessed 640×640×3 image tensor into the multi-label attribute classification model. Extract features through the backbone network to generate a 2048-dimensional feature vector. Then, calculate the original prediction score in parallel through the two classification branches. Convert the original prediction score into a confidence level with a value range of 0-1 through the Softmax function.
[0048] S503. Output the classification results and corresponding confidence scores of the mask wearing attribute and the work status attribute. Among them, the mask wearing attribute outputs two categories: no mask and mask wearing, and their respective confidence scores. The work status attribute outputs two categories: work status and non-work status, and their respective confidence scores.
[0049] Furthermore, S6 specifically includes the following steps:
[0050] S601. Extract preliminary judgment results from the detection results of the YOLOv8 target detection model. The work status is judged as "whether sparks are detected in the area affected by personnel's work" to determine whether they are working or not. The mask wearing status is judged as "whether the IoU between the head and the mask bounding box is ≥ a preset threshold when sparks are detected" to determine whether they are not wearing a mask or wearing a mask. Extract the highest confidence and corresponding category of the two types of attributes from the results of the multi-label attribute classification model.
[0051] S602. Set the weight of the YOLOv8 object detection model to 'a' and the weight of the multi-label attribute classification model to 'b', and a + b = 1; calculate the fusion confidence of the work status and the mask wearing status respectively. The fusion confidence of the work status = (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b), and the fusion confidence of the mask wearing status = (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b); normalize the fusion confidence to ensure that the sum of the fusion confidence of each attribute is 1.
[0052] S603. The work status is determined by the category with the highest fusion confidence, and the mask wearing status is determined by the category with the highest fusion confidence. If the work status is not in work, the final status is non-work status. If the work status is in work and the mask status is wearing, the final status is safe work status. If the work status is in work and the mask status is not wearing, the final status is dangerous work status.
[0053] Furthermore, S7 specifically includes the following steps:
[0054] S701. A cross-frame tracking algorithm based on target detection bounding box coordinates, category ID, and motion trajectory features is adopted to assign a unique and fixed tracking ID to each worker.
[0055] S702. Maintain a dynamic sliding window for each tracking ID. The window size is set to N frames. The window is dynamically updated as new frames are input. The final state data of the last N frames is always retained.
[0056] S703. Differentiated voting rules are set for the different characteristics of mask wearing attributes and work status attributes. When the proportion of frames in the dynamic window that are determined to have the same mask wearing attribute reaches a preset high proportion threshold, the mask wearing is confirmed to be a stable state. When the proportion of frames in the dynamic window that are determined to be in the "working" state reaches a preset low proportion threshold, the work status is confirmed to be a stable state.
[0057] S704. Combining the stable mask-wearing status and stable work status confirmed by the above differentiated voting rules, a comprehensive safety status judgment is made. If the vote confirms "working" and "no mask", it is judged as "unsafe work status"; if the vote confirms "working" and "wearing a mask", it is judged as "safe work status"; if the vote confirms "not working", it is judged as "non-working status".
[0058] S705. Configure an independent status counter for each tracking ID, with an initial value of 0. When the overall judgment is "unsafe work status", the value of the status counter is incremented by 1. When the overall judgment is "safe work status" or "non-work status", the value of the status counter is cleared to zero. When the value of the status counter continuously accumulates to reach the preset threshold, it is confirmed that the unsafe work status has been stably occurring, and the tracking ID, status and related attribute data of the person are transmitted to the alarm module.
[0059] Furthermore, S8 specifically includes the following steps:
[0060] S801. Establish an alarm cache table to record the most recent alarm timestamp and the most recent alarm mask status corresponding to each operator's tracking ID.
[0061] S802. Set an alarm cooling-off period. When an unsafe work status information is received from the alarm module, query the last alarm record corresponding to the tracking ID in the alarm cache table and calculate the time difference between the current system time and the last alarm timestamp.
[0062] S803. If the time difference between the current system time and the last alarm timestamp is greater than or equal to the alarm cooling-off period, or if the mask-wearing status of the last alarm is different from the current mask-wearing status, then an alarm event is generated that includes the worker tracking ID, mask-wearing status, work status, detection timestamp, target bounding box coordinates, fusion confidence score, and IoU calculation result. The alarm cache table is updated and output. If the time difference between the current system time and the last alarm timestamp is less than the alarm cooling-off period and the mask-wearing status of the last alarm is the same as the current mask-wearing status, then no new alarm event is generated.
[0063] Compared with the prior art, the present invention has the following beneficial effects:
[0064] Faced with complex construction site environments and multi-person operation scenarios, YOLOv8 target detection, dual-model fusion judgment, and multi-person independent tracking mechanism can achieve accurate collaborative detection of the mask wearing status and operation status of workers. This can effectively solve the problems of poor adaptability to complex environments, interference in multi-person scenarios, and high false alarm and false alarm rates in existing technologies. Attached Figure Description
[0065] To more clearly illustrate the technical solutions of this embodiment, the accompanying drawings used in the description of the embodiments will be briefly described below. Obviously, the following drawings are only some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.
[0066] Figure 1 This is an example of an overall flowchart of an automatic detection method for the wearing status of welding masks at construction sites, based on the present invention.
[0067] Figure 2 This is a flowchart illustrating an exemplary embodiment of the associated target clipping process of the present invention;
[0068] Figure 3 This is a flowchart illustrating an exemplary embodiment of the reasoning and result output of the multi-label attribute classification model of the present invention.
[0069] Figure 4 This is a flowchart illustrating an exemplary embodiment of the dual-model weighted fusion calculation and joint determination of two types of attributes of the present invention. Detailed Implementation
[0070] To facilitate a clearer understanding of the technical solutions of this application by those skilled in the art, the technical solutions are described clearly and completely below with reference to the accompanying drawings of the embodiments of this application. Obviously, the described embodiments are only some embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments disclosed in this specification without creative effort should be included within the protection scope of this application.
[0071] Please see Figure 1 This is an exemplary embodiment of the present invention of an automatic detection method for the wearing status of welding masks at construction sites, comprising the following steps:
[0072] S1. Real-time video stream of welding operations is collected through cameras at the construction site. Frames are extracted at a preset analysis frame rate of 5-10 FPS to generate images to be inspected. Multiple continuously extracted frames are combined into batch processing data.
[0073] In practice, high-definition network cameras are deployed at the construction site. The camera installation height should be set to ensure coverage of the entire welding operation area without blind spots. Considering hardware computing power and real-time detection requirements, the preset analysis frame rate is 5-10 FPS, meaning 5 frames per second are extracted from the real-time video stream as the images to be inspected. To improve batch inference efficiency, 20 consecutively extracted images are combined into a batch processing dataset (batch size can be adjusted according to GPU memory). Batch input reduces the time overhead of model loading and inference, enabling simultaneous parallel detection of multiple personnel and multiple states.
[0074] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0075] S2. Perform an adaptive preprocessing procedure for welding scene on the image to be inspected to obtain standardized input data.
[0076] Specifically, in this embodiment, the real-time video stream is extracted into frames at a preset analysis frame rate of 5-10 FPS as described in step S1 to generate images to be inspected, and the extracted multiple frames are preprocessed.
[0077] As a specific embodiment, step S2, the method for performing adaptive preprocessing on the image to be inspected, is as follows:
[0078] S201. Adaptive histogram equalization technology is used to dynamically adjust the contrast of the image to be inspected (i.e., images of different resolutions such as 1280×720 and 800×600 commonly seen at construction sites). This step can enhance the recognition of mask edges and spark details in outdoor strong light overexposure scenes; and can improve the grayscale difference between the target and the background in indoor low light scenes. At the same time, Gaussian filtering is used to smooth and denoise the sensor noise under low light conditions, so as to avoid noise interfering with the detection accuracy of small targets such as welding sparks.
[0079] S202. Take the image to be inspected (i.e., the image to be inspected with different resolutions such as 1280×720 and 800×600 commonly seen at construction sites) after contrast adjustment and smoothing and noise reduction in S201 as the source image, and uniformly calculate the scaling factor k of it and the 640×640 network input resolution, k=min(640 / source image width, 640 / source image height), and generate an affine transformation matrix by edge filling. That is, after scaling the image to be inspected according to the scaling factor k, black borders are filled on the edges of the image to be inspected to a size of 640×640 to ensure that the target geometry is not distorted.
[0080] S203. Extract the uint8 type pixel values of the scaled and filled image in S202, convert them to float32 type, normalize them to the 0-1 range, and then subtract the mean and divide by the standard deviation according to the YOLOv8 object detection model training settings to obtain standardized input data.
[0081] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0082] S3. Input the standardized input data into the pre-trained YOLOv8 target detection model and simultaneously output the bounding boxes, confidence scores, and category IDs of four target categories: person, head, mask, and welding spark.
[0083] Specifically, in this step, the YOLOv8 target detection model uses the CSPDarknet-53 backbone network, PANet neck network, and multi-scale detection head. During model inference, the standardized 640×640 image data is input into the model, and the bounding box coordinates (xmin, ymin, xmax, ymax), confidence scores, and category IDs of four types of targets, namely, person, head, mask, and welding spark, are output simultaneously.
[0084] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0085] S4. Perform human-centric related target cropping processing to generate a cropped image that includes the worker's entire body and surrounding potential related areas.
[0086] Specifically, this step involves performing associated target clipping according to the following process:
[0087] S401. For each frame of the image, iterate through all detected worker targets. For each worker target, calculate the intersection-union ratio (IoU) between the worker's bounding box and the bounding boxes of associated targets in the image (i.e., the bounding boxes of all protective masks and welding spark targets). The IoU is calculated using the formula IoU = ( ) / ( Calculate, where A is the bounding box area of the operator, and B is the bounding box area of the associated target (mask or spark). Let A be the area of the intersection of A and B. Let the area be the union of A and B;
[0088] S402. If there are protective masks or welding sparks around the worker's bounding box, and the association judgment condition of IoU ≥ preset threshold is met, then the worker's bounding box is expanded based on the bounding rectangle of the worker's bounding box and the bounding box of the associated target, so that the clipping area completely includes the worker and all associated targets. The boundary coordinates of the expanded worker's bounding box are calculated according to the following formula:
[0089] The horizontal coordinate x of the top left corner of the expanded worker's bounding box min=min(the horizontal coordinate of the top left corner of the expanded worker's bounding box x) min The horizontal coordinate x of the top-left corner of the bounding box of all associated targets after expansion. min );
[0090] The vertical coordinate y of the top left corner of the expanded worker's bounding box min =min(vertical coordinate y of the top left corner of the expanded worker's bounding box) min The vertical coordinate y of the top-left corner of the bounding box of all associated targets after expansion. min );
[0091] The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =max(the horizontal coordinate of the bottom right corner of the expanded worker's bounding box x) max The horizontal coordinate x of the bottom right corner of the bounding box of all associated targets after expansion. max );
[0092] The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =max(the vertical coordinate of the bottom right corner of the expanded worker's bounding box) max The vertical coordinate y of the bottom right corner of the bounding box of all associated targets after expansion. max );
[0093] S403. If there are no protective masks or welding spark targets around the worker's bounding box, or if all protective masks and welding spark targets do not meet the association determination conditions in S402, then the length and width of the worker's bounding box are each proportionally doubled, using the center of the worker's bounding box as a fixed point. The boundary coordinates of the enlarged worker's bounding box are calculated using the following formula:
[0094] The width W of the original worker's bounding box is equal to the horizontal coordinate x of the bottom right corner of the original worker's bounding box. max - The horizontal coordinate x of the top left corner of the original operator's bounding box min ;
[0095] The height H of the original worker's bounding box is equal to the vertical coordinate y of the bottom right corner of the original worker's bounding box. max - The vertical coordinate y of the top left corner of the original operator's bounding box min ;
[0096] The expanded worker's bounding box width W' = the original worker's bounding box width W × 2;
[0097] The height H' of the expanded worker's bounding box is equal to the height H × 2 of the original worker's bounding box.
[0098] The horizontal coordinate x of the top left corner of the expanded worker's bounding box min =Horizontal coordinates of the center point of the original operator's bounding box - W' / 2;
[0099] The vertical coordinate y of the top left corner of the expanded worker's bounding box min =Vertical coordinates of the original worker's bounding box center point - H' / 2;
[0100] The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =Horizontal coordinates of the center point of the original operator's bounding box + W' / 2;
[0101] The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =Vertical coordinates of the center point of the original worker's bounding box + H' / 2;
[0102] S404. If the bounding box coordinates of the expanded operator exceed the pixel range of the original image (i.e., xmin < 0, ymin < 0, xmax > original image width, ymax > original image height), then the coordinates of the excess part are corrected to the original image boundary coordinates. Specifically, xmin is corrected to 0, ymin is corrected to 0, xmax is corrected to the original image width, and ymax is corrected to the original image height. Then, the image is cropped to obtain the cropped image.
[0103] To further illustrate the execution of cropping images that include the entire body of a person and potentially related surrounding areas, please refer to [link to relevant documentation]. Figure 2 This is a flowchart of an exemplary embodiment of the associated target clipping process of the present invention.
[0104] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0105] S5. Input the cropped image into a pre-trained multi-label attribute classification model and output the classification results and confidence scores of the mask wearing attribute and the work status attribute.
[0106] Specifically, this step uses a pre-trained multi-label attribute classification model to classify the mask wearing status and work status in the cropped image, and outputs the confidence scores of various attributes to provide data support for subsequent rule fusion.
[0107] More specifically, this step involves performing related target clipping according to the following process:
[0108] S501. Input the cropped image into a pre-trained multi-label attribute classification model. The multi-label attribute classification model adopts a ResNet50 dual-branch architecture. The ResNet50 backbone network is initialized with weights pre-trained on the ImageNet dataset. The parameters of the first 30 layers are frozen and the last 19 feature extraction layers are fine-tuned. A global average pooling layer is connected after the backbone network to output a 2048-dimensional feature vector. The classification head is set to two parallel classification branches, one for mask wearing attributes and the other for work status attributes. The model has a fixed input size of 640×640 pixels.
[0109] S502. Perform single-frame or batch inference. Input the preprocessed 640×640×3 image tensor into the multi-label attribute classification model. The backbone network extracts features to generate a 2048-dimensional feature vector. Then, the original prediction score is output through parallel calculation by two parallel classification branches. The original prediction score is converted into a confidence level with a value range of 0-1 through the Softmax function.
[0110] S503 outputs the classification results and corresponding confidence scores for two types of attributes: mask wearing attribute and work status attribute. Specifically, the mask wearing attribute outputs two results: no mask and mask wearing, along with their respective confidence scores. The work status attribute outputs two results: work status and non-work status, along with their respective confidence scores.
[0111] To further illustrate the classification results and confidence scores of the pre-trained multi-label attribute classification model that inputs cropped images, and outputs mask-wearing attributes and job status attributes, please refer to [link to relevant documentation]. Figure 3 The flowchart below is an exemplary embodiment of the reasoning and result output of the multi-label attribute classification model of the present invention.
[0112] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0113] The S6 and YOLOv8 target detection models and multi-label attribute classification models are weighted and fused together to jointly determine the mask wearing attribute and the work status attribute.
[0114] Specifically, this step follows the procedure of integrating the rules:
[0115] S601. Extract preliminary judgment results from the YOLOv8 target detection model detection results. The operation status is judged as "whether sparks are detected in the operation influence area" to determine whether the operation is in progress or not. The mask wearing status is judged as "whether the IoU between the head and the mask bounding box is ≥ the preset threshold" to determine whether the mask is not worn or the mask is worn. Extract the highest confidence level and corresponding category of the mask wearing attribute and the operation status attribute from the multi-label attribute classification model results.
[0116] S602. Set the weight of the YOLOv8 object detection model to a, the weight of the multi-label attribute classification model to b, and a+b=1. Calculate the fusion confidence of the work status and the mask wearing status respectively. The fusion confidence of the work status is calculated as follows: (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b). The fusion confidence of the mask wearing status is calculated as follows: (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b). Then, normalize the fusion confidence.
[0117] S603. The work status is determined by the category with the highest fusion confidence, and the mask wearing status is determined by the category with the highest fusion confidence. If the work status is not in work, the final status is non-work status. If the work status is in work and the mask status is wearing, the final status is safe work status. If the work status is in work and the mask status is not wearing, the final status is dangerous work status.
[0118] To further explain the dual-model weighted fusion calculation and the joint determination of the two types of attributes, please refer to [link to relevant documentation]. Figure 4 The above is a flowchart of an exemplary embodiment of the dual-model weighted fusion calculation and joint determination of two types of attributes of the present invention.
[0119] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0120] S7 assigns unique IDs to personnel for tracking, uses dynamic window-based differentiated voting for judgment, and triggers stable operation status for unsafe work conditions.
[0121] Specifically, the implementation method for this step is as follows:
[0122] S701. A cross-frame tracking algorithm based on target detection bounding box coordinates, category ID, and motion trajectory features is adopted to assign a unique and fixed tracking ID to each worker.
[0123] S702. Maintain a dynamic sliding window for each tracking ID. The window size is set to N frames. The window is dynamically updated as new frames are input. The final state data of the last N frames is always retained.
[0124] S703. Differentiated voting rules are set for the different characteristics of mask wearing attributes and work status attributes. When the proportion of frames in the dynamic window that are determined to have the same mask wearing attribute reaches a preset high proportion threshold, the mask wearing is confirmed to be a stable state. When the proportion of frames in the dynamic window that are determined to be in the "working" state reaches a preset low proportion threshold, the work status is confirmed to be a stable state.
[0125] S704. Combining the mask-wearing stability and operational stability confirmed by the differentiated voting rules in S703, a comprehensive safety status judgment is made. If the vote confirms "working" and "no mask", it is judged as "unsafe working status"; if the vote confirms "working" and "wearing a mask", it is judged as "safe working status"; if the vote confirms "not working", it is judged as "non-working status".
[0126] S705. Configure an independent status counter for each tracking ID, with an initial value of 0. When the overall judgment is "unsafe work status", the value of the status counter is incremented by 1. When the overall judgment is "safe work status" or "non-work status", the value of the status counter is cleared to zero. When the value of the status counter continuously accumulates to reach the preset threshold, it is confirmed that the unsafe work status has been stably occurring, and the tracking ID, status and related attribute data of the person are transmitted to the alarm module.
[0127] Please continue reading Figure 1 An exemplary embodiment of the present invention, a method for automatically detecting the wearing status of welding masks at construction sites, further includes the following steps:
[0128] S8. Use the alarm cache table to deduplicate alarm cycles, generate alarm events with attribute details, and output the final result.
[0129] Specifically, the implementation method for this step is as follows:
[0130] S801. Establish an alarm cache table to record the most recent alarm timestamp and the most recent alarm mask wearing status corresponding to each worker's tracking ID.
[0131] S802. Set an alarm cooling-off period. When an unsafe work status information is received from the alarm module, query the last alarm record corresponding to the tracking ID in the alarm cache table and calculate the time difference between the current system time and the last alarm timestamp.
[0132] S803. If the time difference between the current system time and the last alarm timestamp is greater than or equal to the alarm cooling-off period, or if the mask-wearing status of the last alarm is different from the current mask-wearing status, then an alarm event is generated that includes the worker tracking ID, mask-wearing status, work status, detection timestamp, target bounding box coordinates, fusion confidence score, and IoU calculation result. The alarm cache table is updated and output. If the time difference between the current system time and the last alarm timestamp is less than the alarm cooling-off period and the mask-wearing status of the last alarm is the same as the current mask-wearing status, then no new alarm event is generated.
[0133] Finally, it should be noted that the above are merely specific embodiments of the present invention and do not limit the scope of patent protection of the present invention. Any equivalent structural or procedural modifications made based on the description and drawings of the present invention, or direct or indirect applications to other related technical fields, shall similarly be included within the scope of patent protection of the present invention.
Claims
1. An automatic detection method for the wearing status of welding masks at construction sites, characterized in that, Includes the following steps: S1. Real-time video stream of welding operations is collected through cameras at the construction site. Frames are extracted at a preset analysis frame rate of 5-10 FPS to generate images to be inspected. Multiple continuously extracted frames are combined into batch processing data. S2. Perform an adaptive preprocessing procedure for welding scenarios on the image to be inspected to obtain standardized input data; S3. Input the standardized input data into the pre-trained YOLOv8 target detection model and simultaneously output the bounding boxes, confidence scores and category IDs of four types of targets: workers, heads, masks and welding sparks. S4. Perform human-centric related target cropping processing to generate a cropped image that includes the worker's entire body and surrounding potential related areas; S5. Input the cropped image into a pre-trained multi-label attribute classification model and output the classification results and confidence scores of the mask wearing attribute and the work status attribute. The S6 and YOLOv8 target detection models and multi-label attribute classification models are weighted and fused together to jointly determine the mask wearing attribute and the work status attribute. S7: Assign unique IDs to personnel for tracking, implement dynamic window-based differentiated voting for judgment, and stably trigger unsafe work status. S8. Use the alarm cache table to deduplicate alarm cycles, generate alarm events with attribute details, and output the final result.
2. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S2 specifically includes the following steps: S201. Adaptive histogram equalization technology is used to dynamically adjust the contrast of the image to be inspected, and Gaussian filtering is used for smoothing and noise reduction. S202. Take the image to be inspected after contrast adjustment and smoothing and denoising in S201 as the source image, calculate the scaling factor of its resolution to the 640×640 network input resolution, and perform edge filling on the image that does not fill the 640×640 size after scaling to generate an affine transformation matrix. S203. Extract the uint8 pixel values of the scaled and filled image in S202, convert them to float32 type, normalize them to the 0 to 1 range, subtract the mean according to the YOLOv8 object detection model training settings, divide by the standard deviation, and obtain the standardized input data.
3. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S4 specifically includes the following steps: S401. First, calculate the intersection-union ratio (IoU) between the bounding box of the operator and the bounding box of the associated target; The bounding box of the associated target is the bounding box of the protective mask or welding spark, and the intersection-union ratio (IoU) is calculated according to the following formula: （1） Where A is the bounding box area of the operator, and B is the bounding box area of the associated target. Let the area be the union of A and B. Let A be the area of the intersection of A and B; S402. If there are protective masks or welding sparks around the worker's bounding box, and the association judgment condition of IoU ≥ preset threshold is met, then the worker's bounding box is expanded based on the bounding rectangle of the worker's bounding box and the bounding box of the associated target, so that the clipping area completely includes the worker and all associated targets. The boundary coordinates of the expanded worker's bounding box are calculated according to the following formula: The horizontal coordinate x of the top left corner of the expanded worker's bounding box min =min(the horizontal coordinate of the top left corner of the expanded worker's bounding box x) min The horizontal coordinate x of the top-left corner of the bounding box of all associated targets after expansion. min ); The vertical coordinate y of the top left corner of the expanded worker's bounding box min =min(vertical coordinate y of the top left corner of the expanded worker's bounding box) min The vertical coordinate y of the top-left corner of the bounding box of all associated targets after expansion. min ); The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =max(the horizontal coordinate of the bottom right corner of the expanded worker's bounding box x) max The horizontal coordinate x of the bottom right corner of the bounding box of all associated targets after expansion. max ); The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =max(the vertical coordinate of the bottom right corner of the expanded worker's bounding box) max The vertical coordinate y of the bottom right corner of the bounding box of all associated targets after expansion. max ); S403. If there are no protective masks or welding spark targets around the worker's bounding box, or if all protective masks and welding spark targets do not meet the association determination conditions in S302, then the length and width of the worker's bounding box are each proportionally doubled, with the center of the worker's bounding box as a fixed point. The boundary coordinates of the expanded worker's bounding box are calculated according to the following formula: The width W of the original worker's bounding box is equal to the horizontal coordinate x of the bottom right corner of the original worker's bounding box. max - The horizontal coordinate x of the top left corner of the original operator's bounding box min ; The height H of the original worker's bounding box is equal to the vertical coordinate y of the bottom right corner of the original worker's bounding box. max - The vertical coordinate y of the top left corner of the original operator's bounding box min ; The expanded worker bounding box width W' = the original worker bounding box width W × 2; The height H' of the expanded worker bounding box is equal to the original worker bounding box height H × 2; The horizontal coordinate x of the top left corner of the expanded worker's bounding box min =Horizontal coordinates of the center point of the original operator's bounding box - W' / 2; The vertical coordinate y of the top left corner of the expanded worker's bounding box min =Vertical coordinates of the original worker's bounding box center point - H' / 2; The horizontal coordinate x of the bottom right corner of the expanded worker's bounding box max =Horizontal coordinates of the center point of the original operator's bounding box + W' / 2; The vertical coordinate y of the bottom right corner of the expanded worker's bounding box max =Vertical coordinates of the center point of the original worker's bounding box + H' / 2; S404. If the bounding box coordinates of the expanded worker exceed the pixel range of the original image, the coordinates of the excess part are corrected to the bounding coordinates of the original image, and finally a cropped image containing the worker's whole body and surrounding potential related areas is generated.
4. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S5 specifically includes the following steps: S501. Input the cropped image into a pre-trained multi-label attribute classification model. The multi-label attribute classification model adopts a ResNet50 dual-branch architecture. The ResNet50 backbone network is initialized with weights pre-trained on the ImageNet dataset. The parameters of the first 30 layers are frozen and the last 19 feature extraction layers are fine-tuned. A global average pooling layer is connected after the backbone network to output a 2048-dimensional feature vector. The classification head is set to two parallel classification branches, one for mask wearing attributes and the other for work status attributes. The model has a fixed input size of 640×640 pixels. S502. Perform single-frame or batch inference. Input the preprocessed 640×640×3 image tensor into the multi-label attribute classification model. Extract features through the backbone network to generate a 2048-dimensional feature vector. Then, calculate and output the original prediction score in parallel through the two parallel classification branches. Convert the original prediction score into a confidence level with a value range of 0-1 through the Softmax function. S503. Output the classification results and corresponding confidence scores for the mask wearing attribute and the work status attribute. The mask wearing attribute outputs two categories: no mask and mask wearing, along with their respective confidence scores. The work status attribute outputs two categories: work status and non-work status, along with their respective confidence scores.
5. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S6 specifically includes the following steps: S601. Extract preliminary judgment results from the YOLOv8 target detection model detection results. The operation status is judged as "whether sparks are detected in the operation influence area" to determine whether the operation is in progress or not. The mask wearing status is judged as "whether the IoU between the head and the mask bounding box is ≥ the preset threshold" to determine whether the mask is not worn or the mask is worn. Extract the highest confidence level and corresponding category of the mask wearing attribute and the operation status attribute from the multi-label attribute classification model results. S602. Set the weight of the YOLOv8 object detection model to a, the weight of the multi-label attribute classification model to b, and a+b=1. Calculate the fusion confidence of the work status and the mask wearing status respectively. The fusion confidence of the work status is calculated as follows: (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b). The fusion confidence of the mask wearing status is calculated as follows: (confidence of the corresponding status of the YOLOv8 object detection model × a) + (confidence of the corresponding status of the multi-label attribute classification model × b). Then, normalize the fusion confidence. S603. The work status is determined by the category with the highest fusion confidence, and the mask wearing status is determined by the category with the highest fusion confidence. If the work status is not in work, the final status is non-work status. If the work status is in work and the mask status is wearing, the final status is safe work status. If the work status is in work and the mask status is not wearing, the final status is dangerous work status.
6. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S7 specifically includes the following steps: S701. A cross-frame tracking algorithm based on target detection bounding box coordinates, category ID, and motion trajectory features is adopted to assign a unique and fixed tracking ID to each worker. S702. Maintain a dynamic sliding window for each tracking ID. The window size is set to N frames. The window is dynamically updated as new frames are input. The final state data of the last N frames is always retained. S703. Differentiated voting rules are set for the different characteristics of mask wearing attributes and work status attributes. When the proportion of frames in the dynamic window that are determined to have the same mask wearing attribute reaches a preset high proportion threshold, the mask wearing is confirmed to be a stable state. When the proportion of frames in the dynamic window that are determined to be in the "working" state reaches a preset low proportion threshold, the work status is confirmed to be a stable state. S704. Combining the mask-wearing stability and operational stability confirmed by the differentiated voting rules in S703, a comprehensive safety status judgment is made. If the vote confirms "working" and "no mask", it is judged as "unsafe working status"; if the vote confirms "working" and "wearing a mask", it is judged as "safe working status"; if the vote confirms "not working", it is judged as "non-working status". S705. Configure an independent status counter for each tracking ID, with an initial value of 0. When the overall judgment is "unsafe work status", the value of the status counter is incremented by 1. When the overall judgment is "safe work status" or "non-work status", the value of the status counter is cleared to zero. When the value of the status counter continuously accumulates to reach the preset threshold, it is confirmed that the unsafe work status has been stably occurring, and the tracking ID, status and related attribute data of the person are transmitted to the alarm module.
7. The automatic detection method for the wearing status of welding masks at construction sites according to claim 1, characterized in that, S8 specifically includes the following steps: S801. Establish an alarm cache table to record the most recent alarm timestamp and the most recent alarm mask wearing status corresponding to each worker's tracking ID. S802. Set an alarm cooling-off period. When an unsafe work status information is received from the alarm module, query the last alarm record corresponding to the tracking ID in the alarm cache table and calculate the time difference between the current system time and the last alarm timestamp. S803. If the time difference between the current system time and the last alarm timestamp is greater than or equal to the alarm cooling-off period, or if the mask-wearing status of the last alarm is different from the current mask-wearing status, then an alarm event is generated that includes the worker tracking ID, mask-wearing status, work status, detection timestamp, target bounding box coordinates, fusion confidence score, and IoU calculation result. The alarm cache table is updated and output. If the time difference between the current system time and the last alarm timestamp is less than the alarm cooling-off period and the mask-wearing status of the last alarm is the same as the current mask-wearing status, then no new alarm event is generated.