Detection and alarm system and method for protective equipment of rotating machinery operator

Abstract

The invention discloses a rotary machine operator protective equipment detection and alarm system and method.The rotary machine operator protective equipment detection and alarm system comprises a camera and an embedded system, the camera is used for collecting a real-time image of a rotary machine operator in a working space and sending RGB to the embedded system regularly, and the embedded system sends the real-time image of the rotary machine operator to the camera. The embedded system and the camera are connected to the same local network, the embedded system comprises an edge calculation module used for processing RGB frames and sending the RGB frames to an AI processing module, the AI processing module is internally provided with a pre-trained detection model and used for detecting whether an operator wears PPE in an image or not, and a monitor used for displaying the detection result of the AI processing module. According to the method, each incorrect PPE is displayed as a red bounding box, otherwise, each incorrect PPE is displayed as a green bounding box, images do not need to be continuously transmitted to a cloud system, and therefore bandwidth efficiency, reliability and privacy of workers are guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time image detection, and in particular to a detection and alarm system and method for protective equipment of rotating machinery operators. Background Art

[0002] In the field of industrial manufacturing, many operations have high rates of workplace injuries. However, many workplace injuries can be prevented through the use of personal protective equipment (PPE), such as hard hats, shoes, tights, and eye protection. For example, more than half of factory production accidents are caused by a lack of PPE equipment, while other studies have shown that proactive safety measures (including the continuous use of PPE) can effectively reduce accident rates. Typical examples of personal protective equipment ( Figure 1 ) are work hats, protective glasses, protective gloves, and tight-fitting protective clothing. Protective glasses: Protect the eyes from metal chips and dust. Protective gloves: Gloves are strictly prohibited to prevent hands from being caught in machinery. Work hats: Workers with long hair, in particular, must wear work hats to ensure head safety. Tight-fitting protective clothing: Wear tight-fitting protective clothing with buttoned cuffs and a closed hem to prevent clothing from being caught in machinery. Industrial environments are often a mixture of high-risk and low-risk areas. In the former, the use of different types of protective equipment is often mandatory, while in the latter, the use of protective equipment is discretionary. Despite their effectiveness, it is difficult to ensure that workers entering high-risk areas comply with and use personal protective equipment correctly. Monitoring by human supervisors is often implemented. However, manual monitoring is subject to underreporting, false alarms, and high labor costs.

[0003] Current PPE monitoring systems utilize real-time object detection technology, which has been used to automate specific tasks, such as car identification in road monitoring or person recognition in video surveillance. Most available monitoring systems adopt a cloud-based approach, where video frames are captured locally by cameras and transmitted to a cloud service, where object recognition (primarily based on machine learning) is performed. However, such centralized systems require a reliable network connection, as network outages can disrupt service. Furthermore, these systems consume a large amount of bandwidth as data is continuously offloaded to the cloud infrastructure over the internet. In addition, since images are analyzed on external infrastructure, which is often owned and operated by a third company, workers' privacy cannot be protected. Summary of the Invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of this application to avoid blurring the purpose of this section, the abstract and the title of the invention, and such simplifications or omissions should not be used to limit the scope of the present invention.

[0005] Therefore, the object of the present invention is to provide a rotating machinery operator protective equipment detection and alarm system and method, which does not require continuous image transmission to the cloud system, thereby ensuring bandwidth efficiency, reliability and worker privacy.

[0006] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0007] A rotating machinery operator protective equipment detection and alarm system, comprising:

[0008] A camera is used to collect real-time images of rotating machinery operators in the workspace and send RGB images to the embedded system at regular intervals;

[0009] An embedded system connected to the same local network as the camera, the embedded system comprising:

[0010] Edge computing module, which processes RGB frames and sends them to the AI processing module;

[0011] An AI processing module with a pre-trained detection model to detect whether the operator in the image is wearing PPE;

[0012] The monitor, used to display the detection results of the AI processing module, shows each incorrect PPE use as a red bounding box, otherwise it is displayed as a green bounding box.

[0013] As a preferred solution of the rotating machinery operator protective equipment detection and alarm system described in the present invention, the camera adopts Intel RealSense D415.

[0014] As a preferred solution of the rotating machinery operator protective equipment detection and alarm system described in the present invention, the AI processing module is NCS2.

[0015] As a preferred solution of the rotating machinery operator protective equipment detection and alarm system described in the present invention, the pre-trained detection model is deployed into the embedded system through the model optimizer and the inference engine. The specific deployment steps are as follows: the model optimizer imports the trained detection model and converts it into an intermediate representation IR to optimize the execution in the embedded system, and the inference engine loads the IR into the memory during runtime.

[0016] As a preferred solution of the rotating machinery operator protective equipment detection and alarm system described in the present invention, the images of the data set of the pre-trained detection model include six different categories, namely: head without helmet, head with helmet, no tight work clothes, wearing tight work clothes, wearing goggles, not wearing goggles, hands without gloves, and hands with gloves.

[0017] As a preferred embodiment of the rotating machinery operator protective equipment detection and alarm system of the present invention, the images in the data set of the pre-trained detection model are filtered to remove images with low resolution or images in which the operator is too far or too close to the camera;

[0018] After filtering, duplicates were removed and the mean structural similarity index (MSSIM) was used to detect duplicates. Images with an MSSIM index higher than 0.90 were marked as duplicates. For each pair of images detected as duplicates, the image with the higher resolution was retained.

[0019] As a preferred solution of the rotating machinery operator protective equipment detection and alarm system described in the present invention, the detection model is YOLOv5s, and the PyTorch framework is used for pre-training and evaluation.

[0020] A method for detecting and alarming a protective equipment for rotating machinery operators, comprising the following steps:

[0021] S1, the camera collects real-time images of the rotating machinery operator in the workspace and periodically sends RGB to the embedded system;

[0022] S2. The edge computing module processes the RGB frames received by the embedded system and sends them to the AI processing module. The built-in detection model in the AI processing module detects whether the operator in the image is wearing PPE;

[0023] S3. The edge computing module sends the detection results of the AI processing module to the monitor, which displays each incorrect PPE use as a red bounding box and the rest as green bounding boxes.

[0024] S4. When the AI processing module detects that the operator is missing at least one mandatory PPE, the edge computing module drives the alarm to sound an alarm and drives the controller of the rotating machinery to shut down the rotating machinery.

[0025] Compared with the prior art, the present invention has the following beneficial effects:

[0026] 1. Real-time PPE detection method based on edge computing: PPE detection is performed locally on embedded devices. Using edge computing models, image analysis and classification applications are deployed on embedded systems installed near and directly connected to the camera, eliminating the need for cloud computing and the need to continuously upload image data, saving bandwidth resources.

[0027] 2. Deep learning model optimization: Select the lightweight YOLOv5s model to balance detection accuracy and real-time performance.

[0028] 3. Real-time safety response mechanism: Control the operation of rotating machinery through alarms or power outages to prevent industrial accidents.

[0029] 4. Privacy protection design: Image data is only analyzed locally and not transmitted to external systems, eliminating the risk of privacy leakage. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be derived from these drawings without inventive effort. Among them:

[0031] Figure 1 A schematic diagram of an operator wearing personal protective equipment provided by the present invention;

[0032] Figure 2 This is a framework diagram of the rotating machinery operator protective equipment detection and alarm system provided by the present invention;

[0033] Figure 3 OpenVINO workflow diagram for deploying pre-trained detection models provided by the present invention;

[0034] Figure 4 This is an example image from the PPE Detection Dataset provided by the present invention. DETAILED DESCRIPTION

[0035] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0036] The present invention provides a rotating machinery operator protective equipment detection and alarm system and method, which does not require continuous image transmission to the cloud system, thereby ensuring bandwidth efficiency, reliability and worker privacy.

[0037] The structure of the rotating machinery operator protective equipment detection and alarm system of the present invention is as follows Figure 2 As shown, it includes a camera and an embedded system, and the embedded system includes an edge computing module, an AI processing module and a monitor.

[0038] The camera is used to collect real-time images of rotating machinery operators in the workspace and periodically send RGB images to the embedded system;

[0039] The embedded system and the camera are connected to the same local network. The image analysis and classification application is deployed on the embedded system near the camera. Using the local network, there is no need to continuously upload image data, thus getting rid of the dependence on cloud computing and saving bandwidth resources.

[0040] The edge computing module is used to process the RGB frames and send them to the AI processing module;

[0041] The AI processing module has a built-in pre-trained detection model to detect whether the operator in the image is wearing PPE;

[0042] The monitor is used to display the detection results of the AI processing module, showing each incorrect PPE use as a red bounding box, otherwise it is displayed as a green bounding box.

[0043] The specific steps of the above-mentioned rotating machinery operator protective equipment detection and alarm system are as follows:

[0044] S1, the camera collects real-time images of the rotating machinery operator in the workspace and periodically sends RGB to the embedded system;

[0045] S2. The edge computing module processes the RGB frames received by the embedded system and sends them to the AI processing module. The built-in detection model in the AI processing module detects whether the operator in the image is wearing PPE;

[0046] S3. The edge computing module sends the detection results of the AI processing module to the monitor, which displays each incorrect PPE use as a red bounding box and the rest as green bounding boxes.

[0047] S4. When the AI processing module detects that the operator is missing at least one mandatory PPE, the edge computing module drives the alarm to sound an alarm and drives the controller of the rotating machinery to shut down the rotating machinery.

[0048] In this implementation, a Raspberry Pi 5 development board is used as the edge computing node board for the rotating machinery operator protective equipment detection and alarm system. The edge computing module is equipped with an NCS2 as the AI processing module. The NCS2 is a USB device that supports hardware acceleration for inference-based computing. An Intel RealSense D415 is used as the camera. The Raspberry Pi 5 is the latest version of the Raspberry Pi single-board computer. Due to the Raspberry Pi's low computing power, real-time image object detection is supported by the NCS2. The NCS2 can support object detection through DL through hardware acceleration.

[0049] To measure the worker’s position in the area, the Intel RealSense Depth Camera D415 is used. This is a low-cost camera with infrared stereo support that can measure the distance between the detected object and the camera. The rotating machinery operator protective equipment detection and alarm system can use this additional information to enable / disable the alarm based on the worker’s location, that is, the alarm is only enabled when the worker is in a high-risk area.

[0050] Furthermore, in this embodiment, the pre-trained detection model deployed in the AI processing module is deployed using the OpenVINO framework. The process of deploying the pre-trained detection model in the OpenVINO framework is as follows: Figure 3 As shown, the OpenVINO framework includes a model optimizer and an inference engine that simplify the transition from the training phase to application deployment. The model optimizer imports a trained detection model and optimizes its execution on the target device by converting it into an intermediate representation (IR, a representation optimized for the inference engine). The latter loads the IR into memory at runtime to infer the network results for a given input. User applications must be developed to integrate the inference engine, ultimately reading the model and running inference on the target device.

[0051] In the present invention, in order to select the most appropriate detection model to be deployed in the rotating machinery operator protective equipment detection and alarm system, the applicant analyzed the following object detectors: YOLOv5s, RetinaNet, CornerNet and MobileNet-SSD v2, and selected the most appropriate detection model to be deployed in the system of the present invention based on the analysis results.

[0052] The training and evaluation of the above-mentioned YOLOv5s, RetinaNet, CornerNet and MobileNet-SSD v2 detection models are as follows:

[0053] 1. Deep Learning Models for the Rotating Machinery Operator Protective Equipment Detection and Alarm System. To automatically detect personal protective equipment (PPE) worn by rotating machinery operators, the system relies on deep neural network models to identify and classify objects in images. Typically, neural network training is performed on high-performance servers, using mainstream deep learning frameworks such as TensorFlow and PyTorch to build and train the model. For embedded deployment, the trained model is converted to an intermediate representation (IR) using the Model Optimizer in the OpenVINO toolchain to adapt it to the inference engine on the edge device. This conversion is lossless; for the same input, the original trained model and the IR model will produce consistent inference results. We obtained several pretrained networks trained on the COCO dataset from public repositories as initial models for the object detection task. The pretrained weights for YOLOv5s were obtained from the Ultralytics GitHub repository, while other models are available through the TensorFlow Model Zoo. In this study, we performed transfer learning and fine-tuning on these pretrained models based on the specific requirements of the PPE detection task. The following will further introduce the dataset used and compare and analyze the performance of five deep learning networks in this task.

[0054] 2. Pre-trained model dataset

[0055] Typically, creating large labeled datasets is very time-consuming: First, a set of images suitable for a specific task must be identified, and then such images must be labeled. The latter is often performed manually, which can also lead to errors.

[0056] The proposed rotating machinery operator protective equipment detection and alarm system is designed to identify the presence or absence of three types of PPE: helmet, vest, and gloves. Therefore, the dataset images should include six distinct classes: head without helmet, head with helmet, chest without vest, chest with vest, hands without gloves, and hands with gloves. For simplicity, these classes will be referred to as helmet, no_helmet, gloves, no_glove, glasses, no_glasses, clothing, and no_clothing, respectively.

[0057] This study used Figure 4The publicly available PPE Detection Dataset (hosted on RoboflowUniverse) is used to construct an experimental dataset. This dataset contains 7,035 images from industrial and construction sites, covering workers wearing or not wearing different personal protective equipment (PPE), such as helmets, vests, and gloves. The image scenes include indoor and outdoor scenes, complex backgrounds, and uneven object sizes and distances.

[0058] To improve data quality, we performed the following cleaning and enhancement operations on the original images: removing images with a resolution lower than 200×200 pixels; removing images where objects were too close or too far away, which affected recognition performance; using MSSIM (Mean Structural Similarity Index) to remove duplicates (with a threshold of 0.9) to retain higher-quality images; supplementing and correcting the original annotations, and reviewing the quality of all bounding boxes. The resulting dataset totaled 6,832 images, encompassing a variety of PPE categories and fully annotated.

[0059] We evaluated the performance of four mainstream lightweight object detection networks in the PPE detection task: YOLOv5s, RetinaNet, CornerNet, and MobileNet-SSD v2.

[0060] All models were transferred from pre-trained weights on the COCO dataset and fine-tuned on a public dataset. YOLOv5s was trained using the PyTorch framework, while the remaining models were built and evaluated using the TensorFlow Object Detection API. Experiments were conducted on a server equipped with an AMD EPYC (8 cores, 2.195GHz), 32GB of RAM, and an NVIDIA Tesla T4 GPU (16GB of video memory). The following table lists the model architecture parameters and fine-tuning settings:

[0061] Table 1 Parameter settings

[0062]

[0063] We used 5-fold cross-validation (stratified k=5) on the dataset to evaluate the detection performance of different models for various PPEs. The metrics used were the average precision (AP@50) for each category and the overall mAP@50.

[0064] Table 2 Cross-validation results of dataset (AP@50%)

[0065]

[0066] To evaluate the performance of different deep learning object detection models in the PPE detection task, we employed five-fold cross-validation on the public dataset D1, using AP@50 as the evaluation metric, and compared the average accuracy across eight PPE categories (worn / not worn). Table 2 summarizes the detection performance of the four models: YOLOv5s, RetinaNet, CornerNet, and MobileNet-SSD v2.

[0067] Experimental results show that YOLOv5s performs best among all models, with an mAP@50 of 91.0%, achieving the highest accuracy in all 8 categories. In particular, in the recognition tasks of key protective parts such as helmet (98.3%), no_helmet (96.7%), glove (83.5%), and no_glove (81.6%), YOLOv5s' accuracy is significantly better than other models. This is due to its good balance between lightweight structure and accuracy. It has excellent recognition capabilities while taking into account computational efficiency, and is suitable for deployment on edge devices to achieve real-time PPE detection. CornerNet has an mAP@50 of 88.2%, which is very competitive in some small target categories (such as glasses, no_glasses, no_clothing), thanks to its key point detection mechanism and high input resolution, but its overall recognition performance is slightly lower than YOLOv5s, and its inference speed is relatively slow. MobileNet-SSD v2 performed stably, with an mAP@50 of 86.9%. It performed well in categories such as glove and no_glove, making it suitable as a lightweight candidate for deployment on low-power or resource-constrained devices. However, its recognition capabilities were slightly inferior in the high-complexity PPE detection scenario. RetinaNet was the weakest performing model, with an mAP@50 of only 71.9%. Although Focal Loss was used to address the foreground / background imbalance, it lacked sensitivity to small objects in the actual PPE detection task, resulting in low overall accuracy. This is presumably due to its relatively low input size (512×512) and anchor configuration.

[0068] In summary, YOLOv5s has the best overall performance in terms of accuracy, stability, and real-time performance, and is the preferred target detection model deployed in this system. CornerNet and MobileNet-SSD v2 can be used as alternative models depending on specific deployment requirements. RetinaNet is not suitable for the efficient PPE detection scenario of this invention.

[0069] 4 Conclusion

[0070] This paper addresses the operational risks faced by rotating machinery operators in industrial settings by proposing a protective equipment detection and alarm system and method based on edge computing and deep learning. By processing video streams in real time on a local embedded device, the system automatically identifies and issues warnings regarding the presence of key PPE (personal protective equipment) such as helmets, goggles, gloves, and protective clothing worn by operators, effectively enhancing intelligent oversight capabilities in hazardous areas. This system eliminates the need to upload images to the cloud, avoiding bandwidth consumption and privacy risks, and offers excellent practicality and deployment flexibility. The overall system architecture comprises camera acquisition, edge computing analysis, AI recognition, and front-end alarm response modules, and can be deployed in typical rotating machinery operation scenarios, such as factory workshops, laboratories, and around printing equipment.

[0071] To select the optimal detection algorithm for deployment in the system, this paper conducted a five-fold cross-validation experiment on the public PPE dataset. The performance of four models, YOLOv5s, CornerNet, RetinaNet, and MobileNet-SSD v2, on a multi-class PPE recognition task was systematically evaluated. The experimental results showed that YOLOv5s achieved the best overall performance in terms of accuracy (91.0% mAP@50), inference speed, and adaptability to edge devices. Therefore, it was selected as the final object detection model for deployment.

[0072] The actual deployment and verification of this research system also proves that YOLOv5s can achieve high frame rate, low latency, and real-time image analysis on a low-power embedded platform, demonstrating its strong engineering feasibility. In the future, the system can be expanded to more operational scenarios, supporting multi-angle camera analysis, multi-target behavior recognition, and wearable compliance tracking, further enhancing intelligent industrial supervision.

[0073] Although the present invention has been described above with reference to embodiments, various modifications may be made thereto and equivalent components may be substituted without departing from the scope of the present invention. In particular, as long as there are no structural conflicts, the various features of the embodiments disclosed herein may be combined with each other in any manner, and the omission of an exhaustive description of such combinations in this specification is solely for the sake of space and resource conservation. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A rotating machinery operator protective equipment detection and alarm system, characterized in that: include: A camera is used to collect real-time images of rotating machinery operators in the workspace and send RGB images to the embedded system at regular intervals; An embedded system connected to the same local network as the camera, the embedded system comprising: Edge computing module, which processes RGB frames and sends them to the AI processing module; An AI processing module with a pre-trained detection model to detect whether the operator in the image is wearing PPE; The monitor, used to display the detection results of the AI processing module, shows each incorrect PPE use as a red bounding box, otherwise it is displayed as a green bounding box.

2. A rotating machinery operator protective equipment detection and alarm system according to claim 1, characterized in that: The camera uses Intel RealSense D415.

3. A rotating machinery operator protective equipment detection and alarm system according to claim 1, characterized in that: The AI processing module is NCS2.

4. A rotating machinery operator protective equipment detection and alarm system according to claim 1, characterized in that: The pre-trained detection model is deployed to the embedded system through the model optimizer and inference engine. The specific deployment steps are as follows: the model optimizer imports the trained detection model and converts it into an intermediate representation (IR) to optimize execution in the embedded system. The inference engine loads the IR into memory at runtime.

5. A rotating machinery operator protective equipment detection and alarm system according to claim 1, characterized in that: The images of the dataset of the pre-trained detection model include six different categories, namely: head without helmet, head with helmet, chest without vest, chest with vest, hand without gloves, and hand with gloves.

6. A rotating machinery operator protective equipment detection and alarm system according to claim 5, characterized in that: Filter the images in the dataset of the pre-trained detection model to remove images with low resolution or where the operator is too far or too close to the camera; After filtering, duplicates were removed and the mean structural similarity index (MSSIM) was used to detect duplicates. Images with an MSSIM index higher than 0.9 were marked as duplicates. For each pair of images detected as duplicates, the image with the higher resolution was retained.

7. A rotating machinery operator protective equipment detection and alarm system according to claim 1, characterized in that: The detection model is YOLOv5s, and the PyTorch framework is used for pre-training and evaluation.

8. A method for detecting and alarming a rotating machinery operator's protective equipment according to any one of claims 1 to 7, characterized in that: Here are the steps: S1, the camera collects real-time images of the rotating machinery operator in the workspace and periodically sends RGB to the embedded system; S2. The edge computing module processes the RGB frames received by the embedded system and sends them to the AI processing module. The built-in detection model in the AI processing module detects whether the operator in the image is wearing PPE; S3. The edge computing module sends the detection results of the AI processing module to the monitor, which displays each incorrect PPE use as a red bounding box and the rest as green bounding boxes. S4. When the AI processing module detects that the operator is missing at least one mandatory PPE, the edge computing module drives the alarm to sound an alarm and drives the controller of the rotating machinery to shut down the rotating machinery.

Images