A safety monitoring method based on smart helmet and computer vision

Abstract

The application discloses a safety monitoring method based on an intelligent helmet and computer vision, relies on an intelligent helmet terminal integrated with an image acquisition module, a multi-modal sensor group and edge computing main control, and realizes multi-dimensional safety monitoring functions such as personal protective equipment wearing state detection, dangerous behavior identification, multi-target tracking and personnel relative positioning through an entire process technical architecture of environment self-adaptive calibration, multi-modal data preprocessing, lightweight multi-task visual model edge parallel inference, multi-source heterogeneous data fusion graded early warning, edge cloud collaborative iterative optimization, and the like. The application can realize low-delay and high-precision on-body dynamic safety monitoring at an edge end, and can be applied to personnel safety management and control in high-risk operation scenes such as building construction, mine exploitation and power maintenance.

Description

Technical Field

[0001] This invention belongs to the field of industrial safety protection technology, specifically relating to a safety monitoring method based on smart helmets and computer vision. Background Technology

[0002] With the continuous improvement of safety management requirements in high-risk operations such as construction, mining, and power maintenance, the integrated application of computer vision, edge computing, and wearable device technologies has become an important development direction for personnel safety monitoring in industrial sites. Smart helmets, as wearable mobile sensing terminals, integrate image acquisition, multiple sensors, and edge computing units, enabling dynamic safety monitoring on-site by following the movement of workers. Currently, several patented technologies are being researched in this direction.

[0003] Patent CN120047760A discloses a safety early warning method for a mining smart helmet based on image recognition. This solution is designed for high-dust, low-light conditions in mines. It uses reinforcement learning algorithms to dynamically adjust camera exposure parameters and employs a semantic segmentation model to complete underground scene recognition and violation operation judgment. However, this solution only focuses on optimizing image acquisition parameters and semantic segmentation recognition for a single path, lacking a cross-validation mechanism for multimodal data such as visual and inertial sensing. The monitoring method relies entirely on visual images, resulting in a relatively single monitoring dimension.

[0004] Patent CN122132698A discloses a data processing method for a smart safety helmet for high-altitude operations based on edge computing and end-cloud collaboration. This solution constructs an end-cloud collaborative data processing link, designs a risk identification mode that combines rule-based and model-based discrimination in parallel, and relies on the cloud to complete parameter updates and distribution. However, this solution does not fully utilize the perceptual capabilities of computer vision and makes insufficient use of visual information from personnel at the work site.

[0005] Patent CN122226872A discloses a mining smart safety helmet system based on modular hot-swappable and multi-protocol fusion. The advantage of this patent lies in hardware expansion and multi-protocol data parsing, but it does not build a lightweight computer vision algorithm link. It mainly relies on environmental and positioning sensor data for risk assessment, and its visual monitoring capabilities are weak.

[0006] Patent CN121861586A discloses a collaborative early warning method for safety helmets based on AI visual detection and a smart safety helmet. This solution enables collaborative perception between the helmet and an external camera. However, the system is highly dependent on the external fixed monitoring equipment deployed on-site. A large amount of complex risk identification logic is handled by the cloud, which is highly dependent on network communication bandwidth and connectivity.

[0007] Based on the above existing technologies, it can be seen that current industrial safety monitoring solutions based on smart helmets still have the following technical shortcomings:

[0008] (1) It is difficult to balance edge computing power and monitoring dimensions. Due to the limitations of the computing power of the helmet embedded hardware, most existing solutions can only achieve a few tasks such as helmet wearing detection or simple scene recognition. It is difficult to complete multi-dimensional safety monitoring such as PPE multi-type protective equipment wearing detection, dangerous behavior posture recognition, multi-target personnel tracking, and personnel relative positioning in parallel at the edge. It is also impossible to balance inference latency and monitoring coverage.

[0009] (2) Insufficient robustness of identification under complex working conditions. High-risk work scenarios generally have problems such as drastic changes in lighting, dust, personnel occlusion, and severe background interference. Most existing technologies rely on a single visual path or a single type of sensor data to complete risk assessment. They lack a cross-validation mechanism for multi-source heterogeneous data from vision, IMU, and environmental parameters, which can easily lead to missed detections and false alarms.

[0010] (3) Weak environmental adaptability. Most solutions use fixed hardware acquisition parameters and fixed algorithm detection thresholds, lacking a dynamic calibration mechanism that links multiple factors such as light, dust, and temperature. They cannot automatically adjust image enhancement strategies and recognition confidence thresholds according to the on-site environment, nor can they automatically switch model versions to adapt to different working conditions, resulting in limited cross-scene generalization effects.

[0011] (4) Insufficient local autonomous monitoring capability at the edge. Some solutions rely on external surveillance cameras or cloud servers to complete core recognition calculations, with the helmet terminal only serving as a data collection device. Under conditions of unstable network or obstructed and ineffective external monitoring, the monitoring capability is greatly reduced, making it difficult to achieve complete personal safety monitoring independently by the edge of the helmet.

[0012] Therefore, there is a need for a safety monitoring method based on smart helmets and computer vision that can take into account low latency operation at the edge, high-precision detection for multiple tasks, multi-source fusion to reduce false alarms, and strong generalization ability for environmental adaptation. Summary of the Invention

[0013] This invention addresses the problems of existing technologies, such as large coverage blind spots, single detection dimensions, poor environmental robustness, and weak cross-scene adaptability. It proposes a safety monitoring method based on smart helmets and computer vision through an integrated technical architecture that incorporates lightweight multi-task visual perception at the edge, multi-source heterogeneous data fusion judgment, environmental adaptive dynamic calibration, hierarchical differentiated early warning, and edge-cloud collaborative iterative optimization. The method includes the following steps:

[0014] S1. Multimodal sensing system initialization and environmental adaptive calibration, complete hardware self-test, multi-source time synchronization and environmental parameter acquisition, and dynamically adjust hardware acquisition parameters and algorithm operation configuration;

[0015] S2. Multimodal data synchronous acquisition and preprocessing: synchronously acquire visual data and multi-sensor data from the work site, perform image adaptive enhancement and sensor data filtering feature extraction, and generate standardized input data.

[0016] S3 is a lightweight, multi-task computer vision model with parallel inference at the edge. It adopts a multi-threaded asynchronous scheduling architecture to execute multiple types of visual monitoring tasks in parallel.

[0017] S4. Security incident judgment and graded early warning of multi-source heterogeneous data fusion: Multi-dimensional fusion and verification of visual reasoning results and sensor data; After confirming the validity of the security incident, graded early warning response is executed according to the risk level.

[0018] S5 enables data interaction and system iteration optimization through edge-cloud collaboration. It achieves bidirectional data interaction between the edge and the cloud platform through hierarchical data flow scheduling, and completes continuous system optimization by combining cloud model iteration and remote updates.

[0019] Preferably, S1 includes:

[0020] S11. Hardware system self-test and basic parameter configuration: After the smart helmet is powered on, each functional module performs a power-on self-test. If the self-test passes, the default configuration parameters are loaded. If the self-test fails, a fault prompt is triggered and fault information is uploaded.

[0021] S12. High-precision time synchronization of multiple source devices: It adopts a dual synchronization mechanism of network time protocol and local hardware clock secondary calibration to bind the timestamp of a unified clock source to image frames and sensor data.

[0022] S13. Environmental adaptive dynamic calibration: Collect the light intensity, dust concentration and ambient temperature parameters of the current scene, and adjust the image acquisition hardware parameters, image enhancement algorithm strength and detection confidence threshold accordingly, while correcting the sensor temperature drift error.

[0023] Preferably, S2 includes:

[0024] S21. Multi-source data synchronous acquisition and caching: First-person view image data is acquired according to the preset frame rate, and motion sensor data and environmental sensor data are acquired according to the corresponding sampling frequency. All data are bound to timestamps and stored in a circular cache queue.

[0025] S22. Image adaptive enhancement preprocessing: The original image is sequentially processed with distortion correction, adaptive brightness correction, gamma correction, image dehazing and size normalization to improve image quality in complex environments.

[0026] S23. Sensor data filtering and feature extraction: Combined filtering and noise reduction of raw sensor data, solve head posture data and generate standardized environmental feature vectors.

[0027] Preferably, S3 includes:

[0028] S31. Real-time detection of personal protective equipment wearing status: A lightweight detection model is used to identify the wearing status of protective equipment of workers and output the corresponding category and confidence results.

[0029] S32. Dangerous behavior detection and posture temporal modeling: Identify multiple high-risk behaviors by extracting human key points and modeling temporal actions, and perform joint verification by combining motion sensor data.

[0030] S33. Multi-target tracking and relative personnel positioning enables stable tracking and identity maintenance of multiple personnel, and achieves two-dimensional relative positioning and boundary detection of personnel through planar projection mapping.

[0031] S34. Dynamic frame rate scheduling and computing resource optimization: The detection frame rate is dynamically adjusted according to the scene status. A multi-threaded parallel architecture is adopted to optimize computing power utilization, and a fault tolerance and degradation mechanism is set up to ensure the continuous operation of core monitoring functions.

[0032] Preferably, in step S31, the detection model backbone network introduces the GhostConv module, and the neck adopts a lightweight multi-scale feature fusion structure; the model removes redundant channels through structured channel pruning, and then compresses data precision through integer symmetric quantization, and achieves low-latency inference at the edge end with the inference acceleration framework; the detection outputs the category of protective equipment and the corresponding confidence level, and determines the wearing status by combining the spatial position binding relationship of the personnel detection box.

[0033] Preferably, in step S32, key points of human joints are extracted through a lightweight posture network, and a sequence of key points from multiple consecutive frames is input into a bidirectional long short-term memory network to extract temporal action features and output dangerous behavior classification results. Motion sensing data is introduced to jointly verify the recognition results, and corresponding motion feature judgment rules are set for falls and abnormal climbing behaviors, and false detections and false negatives are reduced through cross-validation.

[0034] Preferably, in step S33, an integrated network for unified detection and re-identification is used, which combines trajectory matching algorithm and Kalman filtering to achieve stable tracking and identity maintenance of multiple personnel; based on a pre-calibrated affine transformation matrix, the pixel coordinates of the personnel's feet in the image are mapped to the world coordinate system on site to achieve two-dimensional relative positioning of personnel, and cross-boundary operation detection is achieved by combining preset multi-level regional boundaries.

[0035] Preferably, S4 includes:

[0036] S41. Sliding window confidence smoothing and multi-condition fusion verification: The confidence output of the visual model is processed by time-series smoothing. Multi-condition linkage judgment rules are set for different types of security events. The validity of the event is confirmed by cross-validation of multi-dimensional information.

[0037] S42. Three-level early warning response execution: Based on the risk level of the security incident, three levels of early warning are divided, and differentiated local alarms and cloud-based linkage responses are executed. The accompanying early warning power-off mechanism reduces invalid interference.

[0038] Preferably, in step S41, a sliding window filter is used to smooth the visual confidence score over time, suppressing confidence score fluctuations caused by instantaneous environmental interference; multiple linkage judgment conditions are set for each safety event, and the safety event is confirmed when all conditions are met simultaneously; the safety events include not wearing a safety helmet, falling, and working outside designated areas;

[0039] In S42, Level 1 prompts correspond to minor violations, triggering only short-term local voice and vibration prompts; Level 2 alarms correspond to moderate violations, triggering continuous local audible and visual alarms and pushing information to on-site management personnel; Level 3 alarms correspond to high-risk emergency scenarios, triggering strong local audible and visual alarms and simultaneously pushing emergency information to on-duty and emergency personnel; a warning power-off mechanism is introduced, with the alarm interval time for the same uncorrected event increasing progressively.

[0040] Preferably, S5 includes:

[0041] S51 features hierarchical data flow scheduling and cloud-based security posture aggregation, which differentiates transmission frequency and conditions based on data priority. It also has a built-in seamless multi-mode communication switching mechanism, and locally caches data when the signal is poor, waiting for the network to recover before retransmission. The cloud performs global aggregation and analysis of multi-terminal data to generate security posture statistics and management reports.

[0042] S52. Remote model updates and scene-adaptive optimization: The cloud continuously optimizes the model based on the backflow of difficult examples, generates customized model versions for multiple scenarios, and pushes updates to the terminal remotely through differential upgrades. The terminal can automatically identify the scenario and switch the corresponding model parameters.

[0043] The present invention has the following beneficial effects:

[0044] (1) This invention performs multi-level lightweight adaptation optimization for the entire-link vision algorithm. The backbone network introduces the GhostConv module to replace ordinary convolution, which significantly reduces the number of parameters while retaining the feature extraction capability. Structured channel pruning is used to remove redundant channels, and then INT8 symmetric quantization and TensorRT inference acceleration framework are used to reduce the latency of model edge inference. At the same time, a multi-threaded asynchronous pipeline scheduling architecture is provided to realize parallel and non-blocking processing of image acquisition preprocessing, model inference, data fusion alarm, and data upload.

[0045] (2) This invention constructs a multi-dimensional fusion verification system with visual recognition results as the primary factor, IMU motion sensing data as a secondary factor, and environmental parameters and historical trajectories as supplementary factors. A sliding window filter is used to smooth the visual output confidence level over time, effectively suppressing jumps in detection confidence caused by instantaneous light fluctuations, dust obstruction, and personnel movement. For different types of safety events such as not wearing a safety helmet, falls, and working outside designated areas, multi-condition linkage judgment rules are set, and cross-validation is performed using multi-dimensional information such as head posture angle, peak acceleration, and trajectory continuity, improving the environmental robustness and reliability of the monitoring system.

[0046] (3) This invention designs an environment-adaptive dynamic calibration mechanism. Through the multi-modal sensor group integrated in the helmet, it collects environmental parameters such as light intensity, PM2.5 dust concentration, and ambient temperature in the current working scene in real time, and automatically adjusts the hardware acquisition parameters and algorithm operation strategy. At the same time, in conjunction with the multi-scene customized model version generated in the cloud, the helmet can automatically identify the current environment type and switch the corresponding model parameters.

[0047] In summary, this invention reduces the manpower cost of safety management, improves the efficiency of hazard discovery and handling, and constructs an intelligent safety management system from front-end perception and early warning to back-end control and decision-making. It can be widely adapted to various high-risk operating environments such as building construction, mining, and power maintenance. Attached Figure Description

[0048] Figure 1 This is a diagram illustrating the method steps of an embodiment of the present invention;

[0049] Figure 2 This is a hardware system architecture block diagram according to an embodiment of the present invention;

[0050] Figure 3 This is a parallel inference logic diagram at the edge of the computer vision model in an embodiment of the present invention.

[0051] Figure 4 This is a hierarchical early warning logic diagram of an embodiment of the present invention;

[0052] Figure 5 This is a cloud collaborative architecture diagram according to an embodiment of the present invention. Detailed Implementation

[0053] To further understand the present invention, embodiments of the present invention are described below in conjunction with examples. However, it should be understood that these descriptions are only for further illustrating the features and advantages of the present invention, and not for limiting the scope of the claims of the present invention.

[0054] The method described in this embodiment relies on an integrated smart helmet terminal, such as... Figures 1-5As shown, the smart helmet is an edge sensing and computing node, paired with a cloud management platform to achieve edge-cloud collaboration. The hardware system of the smart helmet includes:

[0055] (1) Protective shell: It is made of ABS engineering plastic and TPU thermoplastic polyurethane integrated injection molding, which conforms to the GB2811-2019 national standard for head protection safety helmets. The shell has a pre-reserved modular installation slot to realize the embedded layout of various electronic components. The center of gravity is located on the vertical center line of the helmet, so there is no forward tilting discomfort when wearing it.

[0056] (2) Image acquisition module: It adopts a 1 / 2.7-inch industrial-grade CMOS image sensor, supports 1920×1080@30fps high-definition acquisition, 120° wide-angle field of view, minimum illumination of 0.01Lux, and supports automatic exposure, automatic white balance, and automatic gain control. With an 850nm infrared fill light, it can clearly acquire images of people within a 5-meter range in the absence of visible light at night.

[0057] (3) Edge computing main controller: It adopts the NVIDIA Jetson Nano core module, equipped with a quad-core Cortex-A57 processor and a 128-core Maxwell GPU, and comes standard with 4GB of memory and 16GB of storage. It supports the TensorRT inference acceleration framework, which can realize edge-side accelerated inference of deep learning models.

[0058] (4) Multimodal sensor array. Six-axis IMU: Employs an MPU6050 six-axis motion sensor, integrating a three-axis accelerometer and a three-axis gyroscope. The accelerometer range is ±16g, and the maximum sampling frequency is 1kHz. In this embodiment, it is configured for 100Hz sampling. Environmental sensors: Integrates a temperature and humidity sensor, as well as a PM2.5 laser particulate matter sensor.

[0059] (5) Communication module: integrates dual-band Wi-Fi (2.4G / 5.8G), 4G LTE Cat.4 full network compatibility, Bluetooth 5.0 BLE multi-mode communication, and supports real-time network status monitoring and seamless switching.

[0060] (6) Power Management Module: Equipped with a high-density lithium battery, supporting QC3.0 fast charging. Paired with a low-power power management chip, it supports dynamic power consumption adjustment, with a continuous working time of ≥8 hours in standard monitoring mode and a battery life of ≥72 hours in standby mode.

[0061] The method in this embodiment includes multiple steps, such as... Figure 1 As shown, the specific explanation is as follows:

[0062] S1. Multimodal Sensing System Initialization and Environmental Adaptive Calibration. This step is the pre-start phase of monitoring, completing hardware self-test, time synchronization, and environmental adaptation calibration to ensure the consistency and environmental adaptability of subsequent multi-source data. It includes the following sub-steps:

[0063] S11. Hardware System Self-Test and Basic Parameter Configuration. After the smart helmet is powered on, the main control unit performs power-on self-tests on the image acquisition module, sensor group, communication module, and power module in sequence. If all modules pass the self-test, the default configuration parameters are loaded, with the camera configured at 1280×720 resolution and 20fps acquisition frame rate. The IMU is configured at a 100Hz sampling frequency. If any module self-test anomaly is detected, a local fault prompt is triggered, and a fault code is uploaded to the cloud, without entering normal monitoring mode.

[0064] S12. High-precision time synchronization across multiple devices. A dual synchronization mechanism, employing both the NTP network time protocol and secondary calibration of the local hardware clock, ensures precise alignment of image frames with the timestamps of all sensor data. When the helmet is connected to the network, it obtains standard time from a cloud time server via the NTP protocol to calibrate the local RTC real-time clock, achieving a synchronization accuracy better than 10ms. In offline, network-free mode, a local high-precision RTC clock is used as the reference. A unified clock source timestamp is applied to each image frame and each set of sensor data via hardware interrupt triggering, achieving a timestamp accuracy of 1ms. All data is bound to the corresponding timestamp when stored in the local circular buffer, providing a time reference for subsequent multi-source data fusion.

[0065] S13. Environmental Adaptive Dynamic Calibration. A multi-modal sensor array is used to collect real-time data on light intensity, PM2.5 dust concentration, and ambient temperature parameters of the current work environment. Based on a preset environment-parameter mapping model, the system automatically adjusts the image acquisition hardware parameters, image enhancement algorithm intensity, and detection confidence threshold, while simultaneously correcting sensor temperature drift errors. This achieves dynamic adaptation across all scenarios. The specific process is as follows:

[0066] (1) Real-time acquisition and filtering of environmental parameters. The current ambient light intensity is acquired through a light sensor. (Unit: lux), sampling frequency 1Hz. Dust concentration was collected using a PM2.5 laser particulate matter sensor. (Unit: μg / m³), sampling frequency 1Hz. Ambient temperature is collected via a temperature and humidity sensor. (Unit: °C), sampling frequency 1 Hz. A moving average filter with a window size of 3 was applied to the collected raw parameters to eliminate transient fluctuations and obtain smoothed environmental parameters as the basis for calibration.

[0067] (2) Illumination-driven adaptive adjustment of camera parameters. Based on illumination intensity. A piecewise linear mapping model is used to dynamically adjust the camera's exposure time. (Unit: s) Analog Gain (Unit: dB) and gamma correction factor The mapping rules are as follows:

[0068] when (Low-light scene): Automatically turn on the 850nm infrared fill light, setting the fill light power to 80% of the rated power. Exposure time is calculated using the following formula: ,in, The baseline exposure time (default 1 / 120s). For the exposure adjustment factor, this embodiment takes... The calculated upper limit of the exposure time is 1 / 30s to avoid motion blur. Simultaneously, the simulation gain is increased. ,in The base gain is set to 0dB (default), and the upper limit of the gain is set to 24dB. The gamma coefficient is... Increase the brightness of dark areas.

[0069] when (Regular lighting scenarios): Exposure time remains at the baseline value. Gain Gamma coefficient It uses the standard mode with automatic exposure and automatic white balance.

[0070] when (Overexposed scenes in strong light): Shorten the exposure time to suppress overexposure, calculated using the following formula: ,in The intensity adjustment coefficient is taken in this embodiment. The minimum exposure time is 1 / 1000s. Simultaneously reduce the analog gain to -6dB, and set the gamma factor to... Compress the dynamic range of the highlighted area.

[0071] (3) Dust-driven image algorithm and adaptive adjustment of detection threshold. Based on dust concentration. The intensity coefficient of the image dehazing algorithm and the confidence threshold of the visual detection model are dynamically adjusted. The dust impact factor is defined. The value range is [0,1], representing the degree of attenuation of the image by dust. Image dehazing intensity and Atmospheric light values ​​with linear positive correlation and dark channel prior dehazing algorithm Correct according to the following formula: ,in This represents the baseline atmospheric light value under dust-free conditions. Lower limit of transmittance. Correct according to the following formula: ,in This is the lower limit of the baseline transmittance (default 0.1). When At that time, the defogging pretreatment process was officially started.

[0072] Dynamic adjustment of detection confidence threshold: For different detection categories such as safety helmets and reflective vests, the baseline confidence threshold is... The adjusted threshold is: In other words, the higher the dust concentration, the lower the detection threshold should be to avoid missed detections caused by target feature attenuation due to high dust levels. The upper limit for threshold reduction is 0.05 to ensure basic accuracy.

[0073] (4) Temperature-driven sensor temperature drift calibration. A linear temperature drift compensation model is used to correct the sensor output values ​​to address the temperature drift errors of the IMU accelerometer and gyroscope. Taking the accelerometer as an example, the zero-bias temperature drift coefficient of each axis is pre-calibrated through high and low temperature tests. (Unit: g / ℃) and sensitivity temperature drift coefficient (Unit: ppm / ℃), the compensation formula is: ,in, This is the raw output value of the accelerometer. The output value after compensation, The reference calibration temperature (default 25℃). The ambient temperature is used. The gyroscope uses the same linear compensation model to eliminate attitude calculation drift errors caused by temperature changes.

[0074] S2. Synchronous Acquisition and Preprocessing of Multimodal Data. After initial calibration, the system enters normal monitoring mode, synchronously acquiring visual data and data from multiple sensors, and performing targeted preprocessing to provide standardized input for subsequent model inference. This includes the following sub-steps:

[0075] S21. Multi-source data synchronous acquisition and caching. Image data acquisition: The camera acquires first-person RGB images of the work site at a preset frame rate and transmits them directly to the GPU memory of the edge controller via the MIPI CSI high-speed interface. Each frame is bound with a precise timestamp of the acquisition time. Sensor data acquisition: The IMU acquires three-axis acceleration and three-axis angular velocity data at a frequency of 100Hz, while temperature, humidity, and PM2.5 sensors acquire environmental data at a frequency of 1Hz. All data is timestamped and stored in a circular buffer queue. Buffer scheduling: A FIFO circular buffer mechanism is adopted. The image buffer retains the most recent 30 frames of data, and the IMU buffer retains the most recent 10 seconds of data for abnormal event backtracking and multi-source data alignment.

[0076] S22. Adaptive Image Enhancement Preprocessing. This involves performing multi-step enhancement processing on the original acquired image to improve image quality in complex environments, including:

[0077] (1) Distortion correction: Based on the internal distortion coefficients calibrated by the camera at the factory, barrel distortion correction is performed on the wide-angle image to eliminate edge deformation and ensure subsequent positioning accuracy.

[0078] (2) Adaptive brightness correction: The Limit Contrast Adaptive Histogram Equalization (CLAHE) algorithm is adopted to divide the image into 8×8 sub-blocks. Histogram equalization is performed on each sub-block, and then bilinear interpolation is used to eliminate the boundaries between blocks, thereby improving the details in the dark areas while avoiding global overexposure.

[0079] (3) Gamma correction: The gamma value is dynamically adjusted according to the current ambient brightness. In dark environments Increase brightness, in bright environments To suppress overexposure, the gamma conversion formula is: ,in, For input image pixels grayscale value, To output grayscale values, is the gamma coefficient.

[0080] (4) Image dehazing: In high dust scenes, the dark channel prior dehazing algorithm is activated to remove the image blur caused by dust and improve the clarity of the target.

[0081] (5) Size normalization: The preprocessed image is scaled to the model input size of 640×640 pixels, the pixel value is normalized, divided by 255 to map to the 0~1 range, and sent to the inference queue.

[0082] S23. Sensor Data Filtering and Feature Extraction. The raw sensor data is filtered and denoised, and features are extracted to provide effective features for subsequent fusion and determination, including:

[0083] (1) IMU data filtering: A combination of sliding window mean filtering and first-order low-pass filtering is used to eliminate high-frequency noise caused by mechanical vibration. The sliding window size is set to 5, and the mean of 5 consecutive sampling points is taken to remove impulse noise. The cutoff frequency of the first-order low-pass filter is set to 10Hz to filter out high-frequency vibration interference. The filtering formula is as follows: ,in, The original acceleration value at time t. This is the filtered output value. The output value from the previous time step. For the filter coefficients, in this embodiment, we can take... .

[0084] (2) Attitude calculation: Based on the filtered acceleration and angular velocity data, the pitch angle, roll angle and yaw angle of the head are calculated by complementary fusion algorithm, and the attitude calculation accuracy is better than 1°.

[0085] (3) Environmental data normalization: The temperature, humidity and PM2.5 data are linearly mapped to the 0~1 range according to the measurement range to form a standardized environmental feature vector.

[0086] S3. Lightweight Multi-Task Computer Vision Model with Parallel Inference at the Edge. The system adopts a multi-threaded asynchronous scheduling architecture to execute multiple vision tasks in parallel, including PPE (Personal Protective Equipment) wearing detection, dangerous behavior and pose recognition, multi-target tracking, and personnel localization. All models have been lightweighted and optimized to adapt to edge computing power, achieving low-latency inference while ensuring accuracy. This includes the following sub-steps:

[0087] S31. Real-time detection of PPE wear status. A lightweight PPE detection model based on improved YOLO is adopted. Through multi-level lightweight solutions such as structure optimization, channel pruning, and parameter quantization, efficient inference at the edge is achieved.

[0088] (1) Model structure optimization. The backbone network introduces the GhostConv module, which significantly reduces the number of parameters and computational cost while ensuring feature extraction capability. GhostConv first generates basic feature maps through a small number of ordinary convolutions, and then generates more feature maps through inexpensive linear transformations. Finally, the output features are obtained by splicing them together. Its computational cost is only about 1 / 3 of that of ordinary convolutions. The neck network adopts a lightweight PANet structure, which retains the ability to fuse multi-scale features while reducing the number of convolutional layers. The detection head retains 3 detection scales, which correspond to the detection of large, medium and small targets, respectively, and are suitable for PPE targets of different sizes such as safety helmets, reflective vests and protective gloves.

[0089] (2) Model channel pruning. A structured channel pruning algorithm based on the BN layer scaling factor is adopted to prune redundant channels of the trained PPE detection model, compressing the model size and computational load under the premise of controllable accuracy loss. The specific implementation process is divided into several stages, including sparsity training, channel pruning and fine-tuning recovery.

[0090] (a) Sparsification training phase. Based on the normal training loss of the model, a scaling factor is applied to each batch normalization (BN) layer of the convolutional layer. Introducing L1 regularization constraints makes unimportant channels... The values ​​approach 0, achieving sparsity of channel weights. The total loss function is: ,in, The loss function for the original detection task (including classification loss and regression loss). For the sparse regularization coefficient, this embodiment takes... . This is the sum of the L1 norms of the scaling factors for all Batch Normalization (BN) layers in the model. During training, the redundant channels correspond to... The value gradually decays to near 0, and the effective channel's The values ​​remain relatively large, resulting in a clear polarization of the weight distribution.

[0091] (b) Channel pruning stage. After sparsification training is completed, all convolutional layers and BN layers of the statistical model are... The absolute values ​​are sorted in ascending order. Based on the preset pruning ratio. Determine the global pruning threshold : All After sorting the values, the first The value corresponding to the quantile is the clipping threshold.

[0092] For each convolutional layer, if a certain channel corresponds to a BN layer... The value satisfies If a channel is deemed redundant, it is pruned. Simultaneously, the input channels of the next convolutional layer corresponding to that channel are also pruned to ensure tensor dimension matching. To avoid a sharp drop in accuracy due to excessive pruning in a single layer, a minimum channel count constraint is set for each convolutional layer: the number of channels retained after pruning is no less than 30% of the original number of channels. After pruning, the number of model parameters and floating-point operations (FLOPs) decrease simultaneously.

[0093] (c) Fine-tuning and recovery phase. The pruned model is fine-tuned and trained using a subset of the original training dataset for 50 iterations. The learning rate is set to 1 / 10 of the initial training learning rate to recover the accuracy loss caused by channel pruning. After fine-tuning, the model's accuracy loss is controlled within 1.5 percentage points, while the inference speed is significantly improved.

[0094] (3) INT8 Symmetric Quantization. The INT8 symmetric quantization scheme provided by TensorRT is used to quantize the model's 32-bit floating-point (FP32) weights and activation values ​​into 8-bit integers. The quantization process employs entropy calibration (KL divergence minimization) to determine the quantization threshold for each layer. By extracting the distribution histogram of activation values ​​for each layer from the calibration dataset, the optimal quantization threshold is found to minimize the KL divergence between the quantized distribution and the original FP32 distribution, thus reducing accuracy loss to the greatest extent. The mapping formula for symmetric quantization is: ,in, These are the original values ​​for FP32. This is the INT8 quantization value. As a quantitative scaling factor, . For truncation functions, the value is restricted to a certain range. Within the acceptable range. After quantization, the model weight volume is significantly reduced, memory bandwidth usage during inference is lowered, and computation speed is improved by 2 to 3 times. The overall approach employs a combined scheme of channel pruning and INT8 quantization, balancing accuracy and speed.

[0095] (4) Model inference and output rules. The model input is the preprocessed image frame. The forward inference outputs a set of detection results, and the detection formula is as follows: ,in: Let N be the bounding box pixel coordinates of the detected targets. The coordinates of the top left corner of the box. The coordinates are the bottom right corner of the box. This is a collection of PPE category labels, consisting of four main categories. The confidence level corresponds to the detection result, representing the probability that the target exists.

[0096] During model training, a Mosaic data augmentation strategy is employed, using randomly stitched, scaled, and cropped four images as training input to improve the model's ability to detect small and occluded targets. A Focal Loss function is introduced to reduce the weight of easily classified samples and improve the training effect on difficult-to-classify and minority class samples, addressing the class imbalance problem of PPE. The Focal Loss formula is: ,in, This represents the model's predicted probability for the target category. Category balancing weights, To focus on the parameters, this embodiment can take... , Set the sample size inversely to the number of samples per category.

[0097] (5) Wearing Status Judgment Rules. The detected PPE targets are spatially bound to the personnel detection frame. The head area corresponds to the safety helmet detection, the torso area to the reflective vest detection, the waist area to the seat belt detection, and the hand area to the protective gloves detection. If no PPE target is detected at the corresponding part of the personnel, or the detection confidence level is low... If the PPE corresponding to the person is not worn, then the person is marked as "not wearing". The not wearing status is then bound to the person's unique trajectory ID and stored in the anomaly candidate queue, awaiting subsequent fusion verification.

[0098] In this embodiment, the model's average inference time per frame on the Jetson Nano platform is approximately 41ms, with a frame rate exceeding 20 FPS. The average accuracy for PPE detection is 90.1%, and the average recall is 86.7%, with a helmet detection accuracy of 94.2% and a recall of 91.5%, meeting the accuracy and real-time requirements for edge deployment.

[0099] S32. Hazardous Behavior Detection and Attitude Temporal Modeling. A multi-level approach is adopted, which combines key point attitude estimation, LSTM temporal modeling, and IMU data fusion, to achieve accurate identification of five high-risk behaviors: falls, failure to wear safety belts, leaning against objects at heights, abnormal climbing, and working outside designated areas.

[0100] (1) Human body key point extraction. The lightweight PoseNet network was used to extract 17 joint key points of the human body. The network used MobileNetV2 as the backbone and depthwise separable convolution to reduce the amount of computation. The input was an image frame. The output is a set of coordinates of key points of human joints, as shown in the following formula: ,in For the information of the j-th joint, For pixel coordinates, The confidence level of the key point. The 17 key points include: nose, left and right eyes, left and right ears, left and right shoulders, left and right elbows, left and right wrists, left and right hips, left and right knees, and left and right ankles.

[0101] (2) Temporal action feature modeling. After scaling the sequence of human keypoint coordinates of consecutive T frames, the data is input into a two-layer bidirectional long short-term memory network (Bi-LSTM) to model the temporal evolution of the action and extract temporal action features. In this embodiment, the temporal window length is... .

[0102] (a) Keypoint Coordinate Normalization. To eliminate the influence of personnel distance and detection box size on the absolute values ​​of coordinates, the pixel coordinates of 17 keypoints in each frame are normalized. Let the coordinates of the top-left corner of the personnel detection box in the current frame be... The coordinates of the lower right corner are Frame width Frame height The coordinates of the j-th key point The normalized coordinates are: After normalization, the coordinate values ​​range from [0,1]. Each frame outputs a 34-dimensional feature vector (17 points × 2 coordinates). The dimensions of T consecutive frames are... The temporal feature sequence is used as the input to the LSTM network.

[0103] (b) Two-layer bidirectional LSTM network structure. The network adopts a two-layer bidirectional stacked structure, with each layer containing a forward LSTM and a backward LSTM, extracting features in forward and reverse time order respectively. The bidirectional outputs are then concatenated to obtain the complete temporal features. The internal computation process of a single LSTM unit is as follows, where the input at time t is... The previous hidden state was Cell state is .

[0104] Forget gate: controls the degree to which the cell's state from the previous moment is retained. Input gate: Controls the degree to which the current input information is written into the cell state. Cell state update: Output gate: Controls the degree to which the current cell state is output to a hidden state. ,in, It is the Sigmoid activation function. The hyperbolic tangent activation function is used. For element-wise product, This is the weight matrix corresponding to the gating. This is the bias vector.

[0105] The first bidirectional LSTM layer has a hidden layer dimension of 128. After concatenating the forward and backward outputs, it outputs a 256-dimensional feature at each time step. The second bidirectional LSTM layer also has a hidden layer dimension of 128. It further encodes the temporal features output from the first layer and finally takes the hidden state output at the last time step to obtain a 256-dimensional global temporal action feature vector.

[0106] (c) Classification output. The 256-dimensional temporal feature vector is fed into a fully connected layer. It is first transformed nonlinearly by the ReLU activation function, and then output by the Softmax classifier to output the probability distribution of 6 types of behavior: falling, not wearing a seat belt, leaning on a height, abnormal climbing, working outside the boundary, and normal behavior, as well as normal behavior. The category with the highest probability is taken as the behavior recognition result of the current temporal window.

[0107] (3) Joint verification of IMU data. Acceleration and posture data from the head IMU are introduced to jointly verify the visual recognition results, improving the recognition accuracy in complex scenes and reducing false positives and false negatives caused by occlusion. Fall behavior verification: If the visual posture recognition detects a continuous downward movement of the human body's center of gravity and an angle less than 30° between the torso and the ground for three consecutive frames, and the IMU detects a peak vertical acceleration exceeding 1.8g and a sudden change in posture angle greater than 60°, then it is determined to be a fall event. Abnormal climbing verification: If the visual keypoints detect a continuous upward movement of the human body and a climbing posture of the limbs, and the IMU detects a continuous periodic change in pitch angle, then it is determined to be abnormal climbing behavior.

[0108] In this embodiment, the average accuracy rate for identifying dangerous behaviors is 91.1%, and the average recall rate is 88.6%. Among them, the accuracy rate for identifying falling behaviors is 95.6%, and the recall rate is 93.8%. The false negative rate is controlled within 5%.

[0109] S33. Multi-target tracking and relative personnel positioning. A unified detection and re-identification framework based on FairMOT is adopted, combined with the DeepSORT trajectory matching algorithm, to achieve stable tracking and ID maintenance of multiple personnel. Relative personnel positioning and boundary crossing detection are achieved through planar affine projection.

[0110] (1) Multi-target tracking algorithm. The FairMOT model performs target detection and Re-ID feature extraction simultaneously in one network, avoiding the accuracy loss of the two-stage algorithm and improving inference speed. The detection branch outputs the person detection box and confidence score. The Re-ID branch outputs a 128-dimensional feature vector for each target, which is used for target matching between different frames.

[0111] In the trajectory matching stage, the Hungarian algorithm is used to achieve optimal matching between the target in the current frame and historical trajectories. The matching metric is a weighted fusion of Mahalanobis distance and apparent feature distance, where Mahalanobis distance is used to measure the matching degree of motion states, and the calculation formula is as follows: Where: x is the motion state vector composed of the detection box coordinates and velocity of the target in the current frame, and y is the motion state vector predicted from the historical trajectory. Let be the covariance matrix of the state vector. The smaller the Mahalanobis distance, the higher the degree of matching between the motion state of the target and the trajectory.

[0112] Combining the cosine distance of apparent features, the final matching cost is a weighted sum of the two. The optimal match is solved using the Hungarian algorithm to achieve stable maintenance of the trajectory ID. For short-term occlusion scenarios, Kalman filtering is used to predict the trajectory position, maintaining the ID unchanged for 3 frames, and tracking resumes after the occlusion disappears.

[0113] (2) Personnel Relative Positioning. Based on the pre-calibrated on-site planar calibration results from the camera, the planar affine projection method is used to map the pixel coordinates of the personnel's feet in the image to the on-site world coordinate system, thereby realizing the two-dimensional relative positioning of the workers. The specific calibration and solution process is as follows:

[0114] (a) On-site calibration. Select four or more non-collinear calibration feature points (such as ground markers, equipment corners, etc.) within the working plane, and use a measuring tape or total station to measure the two-dimensional coordinates of each feature point in the on-site world coordinate system. , , Simultaneously, the helmet-mounted camera captures images containing all calibrated feature points, identifying the pixel coordinates corresponding to each feature point. This forms n sets of coordinate pairs. The origin of the world coordinate system is selected from a fixed reference point within the work area. The X-axis is along the long side of the work area, and the Y-axis is along the short side, with the unit being meters.

[0115] (b) Solving for the affine transformation matrix. The expression for the planar affine transformation in homogeneous coordinates is: ,in, This is a 3×3 affine transformation matrix. Since the last row of the affine transformation is [0,0,1], it contains a total of [0,0,1]. There are a total of 6 unknown parameters. Substituting the i-th set of coordinates into the above equation, we can expand it to obtain two linear equations: , .

[0116] n sets of coordinate pairs constitute 2n linear equations, which can be solved when n≥3. To improve calibration accuracy, this embodiment uses n=6 sets of calibration feature points and solves the overdetermined system of equations using the least squares method to obtain the optimal affine transformation matrix. Write the system of equations in matrix form. ,in: It is a coefficient matrix of 2n×6, with each row corresponding to the coefficient of an equation. Given a 6-dimensional parameter vector to be determined, . Let be a 2n-dimensional observation vector, composed of all world coordinate values. The least squares solution is: The optimal affine transformation parameters are obtained through matrix inversion, thus completing the calibration.

[0117] (c) Calibration accuracy verification and positioning calculation. After calibration, the reprojection error is calculated using the two reserved sets of verification points: the pixel coordinates of the verification points are substituted into... The predicted world coordinates are obtained through matrix calculation, and the Euclidean distance between these and the measured world coordinates is the reprojection error. In this embodiment, the average reprojection error is less than 0.3m, meeting the positioning accuracy requirements. During real-time inference, the coordinates of the bottom midpoint of the personnel detection frame are extracted. Using the foot pixel coordinates as the base coordinates, the world coordinates are calculated using the affine transformation formula. It achieves two-dimensional relative positioning of personnel, with an overall positioning accuracy better than 0.5m.

[0118] (3) Boundary Crossing Detection. Different levels of area boundaries, such as high-risk restricted areas, ordinary restricted areas, and warning areas, are preset in the on-site world coordinate system and stored as a set of polygon coordinates. The system determines in real-time whether the personnel's location coordinates fall within the corresponding area, and performs boundary crossing detection based on the dwell time. If a personnel's trajectory enters the restricted area boundary and lasts for more than 3 seconds, it is determined as an abnormal boundary crossing operation and stored in the abnormal candidate queue. Multi-level area configuration is supported, with different levels of warnings triggered for different areas.

[0119] In this embodiment, after INT8 quantization and TensorRT acceleration, the tracking model achieves a frame rate of 18 FPS on the Jetson Nano platform, with low single-frame inference latency and low personnel ID switching rate, meeting the tracking requirements in densely populated scenarios.

[0120] S34. Dynamic Frame Rate Scheduling and Computational Resource Optimization. To balance real-time performance and power consumption, the system employs dynamic frame rate control and multi-threaded parallel scheduling mechanisms to optimize the utilization of computing resources at the edge.

[0121] (1) Dynamic frame rate control: Under normal and abnormal scenarios, the detection frame rate is maintained at 5 FPS to reduce power consumption. When a suspected abnormal event is detected, personnel enter a key monitoring area, or the scene changes drastically, the frame rate is automatically increased to 20 FPS to ensure the real-time response to the abnormality. The low frame rate mode is automatically restored 10 seconds after the abnormal event is resolved.

[0122] (2) Multi-threaded asynchronous scheduling: A 4-thread parallel architecture is adopted, which is responsible for image acquisition and preprocessing, model inference, data fusion and alarm, data upload and communication respectively. Each thread interacts with data through a queue to avoid blocking, realize pipeline processing, and effectively compress the overall response latency.

[0123] (3) Fault tolerance and degradation mechanism: The built-in lightweight fault tolerance mechanism automatically degrades to the basic detection mode when a certain visual reasoning module fails or has insufficient computing power, while retaining the core functions of safety helmet detection and fall alarm, ensuring that the system's basic monitoring capabilities operate uninterruptedly.

[0124] S4. Security Incident Judgment and Graded Early Warning Based on Multi-Source Heterogeneous Data Fusion. This step performs multi-dimensional fusion verification of visual reasoning results and sensor data, eliminates false detections from single-path sources, and, after confirming a security incident, executes graded early warnings according to risk level. This includes the following sub-steps:

[0125] S41, Sliding window confidence smoothing and multi-condition fusion verification.

[0126] (1) Sliding window confidence smoothing. First, a sliding window filter is applied to the frame-by-frame confidence output of the visual model to smooth the confidence level, reducing fluctuations caused by instantaneous changes in illumination, occlusion, and dust, thereby reducing instantaneous false alarms. The smoothing formula is: ,in: The sliding window size can be set to [size] in this embodiment. . Let be the original recognition confidence level of the i-th frame. Let be the confidence level after smoothing at time t.

[0127] (2) Multi-source fusion triggering mechanism. A multi-source fusion judgment mechanism is constructed, with visual results as the primary factor, sensor data as the secondary factor, and historical trajectories as the supplementary factor. Corresponding multi-condition verification rules are set for different types of security events. An event is confirmed only when all conditions are met simultaneously, effectively reducing the false alarm rate. The verification rules for typical events are as follows:

[0128] (a) No helmet incident. Condition 1: Confidence of helmet detection after smoothing 3 consecutive frames. The system detects the head area of ​​the person. Condition 2: The head IMU acceleration is within the normal range of 0.5g to 1.2g, excluding detection deviations caused by falls or strenuous exercise. Condition 3: The ambient light intensity is higher than 1 lux, excluding detection failures in extremely dark environments. When all three conditions are met, the event of not wearing a safety helmet is confirmed.

[0129] (b) Fall event. Condition 1: Attitude recognition detects a drop in center of gravity and an angle of less than 30° between the torso and the ground for three consecutive frames. Condition 2: The IMU detects a peak vertical acceleration exceeding 1.8g, and the acceleration direction reverses. Condition 3: A sudden change in attitude angle greater than 60° occurs, followed by a state of stillness. When all three conditions are met simultaneously, a fall event is confirmed.

[0130] (c) Outbound Operation Event. Condition 1: Personnel positioning coordinates fall within the preset restricted area. Condition 2: The trajectory remains within the restricted area for more than 3 seconds. Condition 3: Three consecutive positioning frames are all within the restricted area, eliminating positioning jump misjudgment. When all three conditions are met simultaneously, the outbound operation event is confirmed.

[0131] Through this multi-source fusion verification mechanism, the system has a low overall false alarm rate under extreme scenarios such as dust interference, sudden changes in lighting, and dense crowds, which is better than single-path visual detection solutions.

[0132] S42. Three-tiered early warning response execution. Based on the risk level and urgency of safety incidents, early warnings are divided into three levels: Level 1 alert, Level 2 alarm, and Level 3 alarm. Differentiated local and cloud-based linkage response strategies are implemented to balance the warning effect with the experience of operators.

[0133] (1) Low-risk Level 1 alert. Triggering scenario: Single instance of improper PPE wearing, brief entry into the warning area, minor violation that can be quickly corrected. Local response: The smart helmet plays a short 1-second voice prompt and a short vibration reminder, without triggering an audible or visual alarm. Cloud response: Only the event log is recorded locally, without uploading to the cloud or notifying management personnel. Design purpose: To remind workers to correct minor violations themselves and avoid frequent interference with normal operations.

[0134] (2) Medium-risk Level 2 Alarm. Triggering scenarios: Failure to correct PPE violations for 5 consecutive seconds, working at height without a safety belt, intrusion into a restricted area, abnormal climbing behavior, etc. Local response: The smart helmet activates a flashing red LED, continuous vibration, and a looping voice alarm until the violation is corrected. Cloud response: Records alarm logs and keyframe images, and pushes alarm information to the on-site safety supervisor's terminal via the MQTT protocol, including the violator's ID, location, event type, and time. Design purpose: To promptly stop moderate violations and notify on-site management personnel to intervene.

[0135] (3) High-risk Level 3 Alarm. Triggering Scenarios: Falls, intrusion into high-risk work areas, prolonged serious violations of regulations, and emergency dangerous situations. Local Response: The smart helmet triggers a high-decibel audio-visual alarm, continuously outputting warning sounds to alert nearby personnel for rescue. Cloud Response: Immediately uploads event information, 10-second video clips before and after the event, and synchronizes IMU data to the cloud platform, automatically pushing alarm information to on-duty managers and emergency response personnel, and supports one-click dialing of emergency contacts. Design Purpose: To achieve the fastest emergency response for high-risk emergencies and reduce accident injuries.

[0136] At the same time, an early warning and power-off mechanism is introduced: if the operator does not correct the situation after the same event is triggered for the first time, the interval between subsequent reminders will increase, such as 2 seconds, 4 seconds, 8 seconds, etc., with a maximum interval of 30 seconds, to avoid frequent alarms interfering with normal operations.

[0137] S5. Edge-Cloud Collaborative Data Interaction and System Iterative Optimization. Through the collaborative linkage between the edge and cloud platforms, global security control, data aggregation and analysis, and continuous model iterative optimization are achieved, ensuring the system's long-term adaptability to different operational scenarios. This includes the following sub-steps:

[0138] S51, hierarchical data flow scheduling and cloud security posture aggregation.

[0139] (1) Layered Data Stream Scheduling Mechanism. The smart helmet adopts a layered data stream scheduling mechanism to distinguish the transmission priority and frequency of different data types, reducing network bandwidth usage and system power consumption. High-priority data: Early warning event data and emergency alarm information are uploaded in real time using a TLS encrypted channel with a latency of less than 1 second. Medium-priority data: Personnel status statistics, alarm logs, and key frame images are uploaded incrementally at a frequency of 1 time / minute. Low-priority data: Raw video streams and full sensor data are automatically cached and uploaded only in Wi-Fi environments, or retrieved on demand. A differential encoding mechanism is adopted, and under normal conditions, only incremental data of status changes is uploaded, rather than full data. When an abnormal event occurs, only key data before and after the event is uploaded, significantly reducing network traffic.

[0140] (2) Redundant Network Switching Strategy. To address signal obstruction and weak coverage issues at the work site, a multi-mode communication redundancy switching strategy is built-in. Real-time monitoring of Wi-Fi and 4G LTE signal strength, latency, and packet loss rate is implemented. When the current network signal strength is below a threshold or the packet loss rate is above 10%, automatic and seamless switching to the communication link with better signal strength is achieved. Data is locally cached during the switching process and automatically resumed transmission after network recovery, ensuring the continuity and reliability of early warning data transmission.

[0141] (3) Cloud-based security situation aggregation. After receiving data from all helmet terminals, the cloud platform performs global aggregation analysis to build an integrated security management system. It displays the location distribution, movement trajectory, and PPE wearing status of all workers in real time, generating a personnel distribution heat map. It statistically analyzes the occurrence frequency, location, and personnel involved in various abnormal events, generating statistical charts of abnormal behavior types and distribution maps of high-risk work areas. All alarm events are bound and stored with corresponding video and sensor data, supporting retrieval and traceability by time, personnel, and event type, for accident investigation, responsibility auditing, and safety management optimization. It supports the automatic generation of multi-dimensional safety reports to assist managers in formulating safety management strategies.

[0142] S52, Model OTA Remote Update and Scene Adaptive Optimization.

[0143] (1) OTA Differential Hot Update Mechanism. The cloud platform supports remote hot updates of models and full lifecycle version control, requiring no manual operation and not affecting normal operations. Pre-update detection: The cloud platform first detects the operating status and computing load of all online helmets, and selects idle or low-load devices to push updates in batches. Differential push: Differential upgrade technology is adopted, only pushing the differences between the old and new models, greatly reducing the size of the upgrade package and reducing traffic consumption and transmission time. Integrity verification: After transmission, the integrity of the model file is verified by MD5 checksum, and automatic retransmission is performed if the verification fails. Seamless switching: After the verification passes, the new version model is automatically switched during the model inference interval, and the upgrade process is imperceptible and does not interrupt normal monitoring functions. Version rollback: If the new version model runs abnormally, it automatically rolls back to the previous stable version to ensure system reliability.

[0144] (2) Continuous Model Iteration and Domain Generalization Optimization. The cloud platform continuously optimizes the model based on newly collected data from various sites, improving generalization ability and detection accuracy. Data Feedback: Difficult sample and false positive / false negative sample from various sites are manually labeled and fed back to the cloud dataset. Domain Generalization Training: MixStyle training and domain adversarial training methods are used to train the model by mixing data from different scenarios, including buildings, mines, tunnels, and power, to improve the model's cross-scenario generalization ability. Scene Customized Models: Customized model versions are generated for typical scenarios, such as tunnel night mode, open-air construction bright mode, and indoor factory mode, and the model parameters and detection thresholds are optimized according to the characteristics of the scenarios. Adaptive Switching: The helmet identifies the current environment type in real time and automatically switches to the corresponding customized model for the scenario, further improving detection accuracy and environmental adaptability.

[0145] This embodiment underwent field testing in three typical high-risk scenarios: construction sites, tunnel work areas, and power maintenance platforms. Real-time performance: The end-to-end total latency, from image acquisition to alarm output, was consistently controlled within 230ms, as shown in Table 1, meeting real-time early warning requirements. Accuracy indicators: The average accuracy rate for PPE detection was 90.1%, the average recall rate for hazardous behavior identification was 88.6%, and the overall false alarm rate was ≤3.5%.

[0146] Table 1 Real-time performance test of the example

[0147] Module Name Average latency (ms) Percentage of total latency (%) Image acquisition and transmission 38 16.5 Model Inference 47 20.4 Data fusion and judgment 62 27.0 Local alarm response 83 36.1 total 230 100.0

[0148] The test results of this implementation show that the method described in this embodiment has the advantages of high precision, low latency, high robustness and low power consumption, which can effectively replace manual supervision, reduce the incidence of safety accidents and improve the level of intelligent safety management in industrial sites.

[0149] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

Claims

1. A safety monitoring method based on smart helmets and computer vision, comprising the following steps: S1. Multimodal sensing system initialization and environmental adaptive calibration, complete hardware self-test, multi-source time synchronization and environmental parameter acquisition, and dynamically adjust hardware acquisition parameters and algorithm operation configuration; S2. Multimodal data synchronous acquisition and preprocessing: synchronously acquire visual data and multi-sensor data from the work site, perform image adaptive enhancement and sensor data filtering feature extraction, and generate standardized input data. S3 is a lightweight, multi-task computer vision model with parallel inference at the edge. It adopts a multi-threaded asynchronous scheduling architecture to execute multiple types of visual monitoring tasks in parallel. S4. Security incident judgment and graded early warning of multi-source heterogeneous data fusion: Multi-dimensional fusion and verification of visual reasoning results and sensor data; After confirming the validity of the security incident, graded early warning response is executed according to the risk level. S5 enables data interaction and system iteration optimization through edge-cloud collaboration. It achieves bidirectional data interaction between the edge and the cloud platform through hierarchical data flow scheduling, and completes continuous system optimization by combining cloud model iteration and remote updates.

2. The safety monitoring method based on smart helmet and computer vision according to claim 1, characterized in that, S1 includes: S11. Hardware system self-test and basic parameter configuration: After the smart helmet is powered on, each functional module performs a power-on self-test. If the self-test passes, the default configuration parameters are loaded. If the self-test fails, a fault prompt is triggered and fault information is uploaded. S12. High-precision time synchronization of multiple source devices adopts a dual synchronization mechanism of network time protocol and local hardware clock secondary calibration to bind the timestamp of a unified clock source to image frames and sensor data. S13. Environmental adaptive dynamic calibration: Collect the light intensity, dust concentration and ambient temperature parameters of the current scene, and adjust the image acquisition hardware parameters, image enhancement algorithm strength and detection confidence threshold accordingly, while correcting the sensor temperature drift error.

3. The safety monitoring method based on smart helmet and computer vision according to claim 1, characterized in that, S2 includes: S21. Multi-source data synchronous acquisition and caching: First-person view image data is acquired according to the preset frame rate, and motion sensor data and environmental sensor data are acquired according to the corresponding sampling frequency. All data are bound to timestamps and stored in a circular cache queue. S22. Image adaptive enhancement preprocessing: The original image is sequentially processed with distortion correction, adaptive brightness correction, gamma correction, image dehazing and size normalization to improve image quality in complex environments. S23. Sensor data filtering and feature extraction: Combined filtering and noise reduction of raw sensor data, solve head posture data and generate standardized environmental feature vectors.

4. The safety monitoring method based on smart helmet and computer vision according to claim 1, characterized in that, S3 includes: S31. Real-time detection of personal protective equipment wearing status: A lightweight detection model is used to identify the wearing status of protective equipment of workers and output the corresponding category and confidence results. S32. Dangerous behavior detection and posture temporal modeling: Identify multiple high-risk behaviors by extracting human key points and modeling temporal actions, and perform joint verification by combining motion sensor data. S33. Multi-target tracking and relative personnel positioning enables stable tracking and identity maintenance of multiple personnel, and achieves two-dimensional relative positioning and boundary detection of personnel through planar projection mapping. S34. Dynamic frame rate scheduling and computing resource optimization: The detection frame rate is dynamically adjusted according to the scene status. A multi-threaded parallel architecture is adopted to optimize computing power utilization, and a fault tolerance and degradation mechanism is set up to ensure the continuous operation of core monitoring functions.

5. A safety monitoring method based on a smart helmet and computer vision according to claim 4, characterized in that, In step S31, the GhostConv module is introduced into the backbone network of the detection model, and a lightweight multi-scale feature fusion structure is adopted for the neck area. The model removes redundant channels through structured channel pruning, and then compresses the data precision through integer symmetric quantization. It is combined with an inference acceleration framework to achieve low-latency inference at the edge. The detection outputs the category of protective equipment and its corresponding confidence level, and determines the wearing status by combining the spatial position binding relationship of the personnel detection box.

6. A safety monitoring method based on a smart helmet and computer vision according to claim 4, characterized in that, In step S32, key points of human joints are extracted through a lightweight posture network, and the sequence of key points in multiple consecutive frames is input into a bidirectional long short-term memory network to extract temporal action features and output dangerous behavior classification results. Motion sensing data is introduced to jointly verify the recognition results, and corresponding motion feature judgment rules are set for falls and abnormal climbing behaviors. Cross-validation is used to reduce false detections and false negatives.

7. A safety monitoring method based on a smart helmet and computer vision according to claim 4, characterized in that, In S33, an integrated network for unified detection and re-identification is adopted, which combines trajectory matching algorithm and Kalman filtering to achieve stable tracking and identity maintenance of multiple personnel; based on the pre-calibrated affine transformation matrix, the pixel coordinates of the personnel's feet in the image are mapped to the world coordinate system on site to achieve two-dimensional relative positioning of personnel, and cross-boundary operation detection is achieved by combining preset multi-level area boundaries.

8. A safety monitoring method based on a smart helmet and computer vision according to claim 1, characterized in that, S4 includes: S41. Sliding window confidence smoothing and multi-condition fusion verification: The confidence output of the visual model is processed by time-series smoothing. Multi-condition linkage judgment rules are set for different types of security events. The validity of the event is confirmed by cross-validation of multi-dimensional information. S42. Three-level early warning response execution: Based on the risk level of the security incident, three levels of early warning are divided, and differentiated local alarms and cloud-based linkage responses are executed. The accompanying early warning power-off mechanism reduces invalid interference.

9. A safety monitoring method based on a smart helmet and computer vision according to claim 8, characterized in that, In step S41, a sliding window filter is used to smooth the visual confidence score over time, suppressing confidence score fluctuations caused by instantaneous environmental interference; multiple linkage judgment conditions are set for each safety event, and the safety event is confirmed when all conditions are met; the safety events include not wearing a safety helmet, falling, and working outside the designated area; In S42, the first-level prompt corresponds to a minor violation scenario, which only triggers a short-term local voice and vibration prompt. Level 2 alarms correspond to moderate violations, triggering continuous local audible and visual alarms and pushing information to on-site management personnel; Level 3 alarms correspond to high-risk emergency scenarios, triggering strong local audible and visual alarms and simultaneously pushing emergency information to on-duty and emergency personnel; a warning power-off mechanism is introduced, with the alarm interval time for the same uncorrected event increasing progressively.

10. A safety monitoring method based on a smart helmet and computer vision according to any one of claims 1-9, characterized in that, S5 includes: S51 features hierarchical data flow scheduling and cloud-based security posture aggregation, which differentiates transmission frequency and conditions based on data priority. It also has a built-in seamless multi-mode communication switching mechanism, and locally caches data when the signal is poor, waiting for the network to recover before retransmission. The cloud performs global aggregation and analysis of multi-terminal data to generate security posture statistics and management reports. S52. Remote model updates and scene-adaptive optimization: The cloud continuously optimizes the model based on the backflow of difficult examples, generates customized model versions for multiple scenarios, and pushes updates to the terminal remotely through differential upgrades. The terminal can automatically identify the scenario and switch the corresponding model parameters.

Images