Industrial safety monitoring system and method

Abstract

The present invention discloses an industrial safety monitoring system and method, which includes an image acquisition module, a core image processing module and a feedback control module. The image acquisition module captures worker behavior data through an industrial camera; the core image processing module is based on an FPGA parallel architecture, and implements dynamic illumination adaptive threshold binarization, skin color recognition, head posture analysis and multi-level morphological filtering to solve the problems of sudden illumination changes, dust interference and target segmentation in industrial scenes; fatigue and distraction are accurately determined through periodic eye closure duration statistics and multi-angle posture analysis. The feedback control module adopts a hierarchical alarm strategy to distinguish risk levels and link equipment shutdowns, supports integration with the MES system through the OPC UA protocol, uploads data in real time and generates safety reports. The present invention improves the real-time performance, anti-interference capability and multi-system collaboration efficiency of industrial monitoring, and reduces the misjudgment rate and accident risk.

Description

Technical Field

[0001] The present invention relates to the technical field of industrial safety monitoring, and in particular to an industrial safety monitoring system and method. Background Art

[0002] With the increasing degree of industrial automation, safety issues in high-risk occupations (such as chemical, metallurgical, and mechanical manufacturing) are receiving increasing attention. Safety accidents caused by fatigue, distraction, or improper operation of production line workers are frequent. Traditional manual inspections or single-sensor monitoring models suffer from poor real-time performance and high false positive rates. Existing vision-based industrial safety monitoring systems often utilize general-purpose processor architectures (such as CPUs or GPUs). However, these systems face the following technical bottlenecks in complex industrial scenarios: insufficient adaptability to dynamic environments. Industrial sites often experience alternating strong light and shadows, as well as dust interference. Traditional image processing algorithms (such as fixed-threshold binarization and single morphological filtering) struggle to accurately extract the target area of ​​worker movement, resulting in target segmentation errors or residual noise. For example, fixed-threshold binarization is prone to losing target details when illumination changes suddenly, while conventional filtering methods cannot effectively eliminate high-frequency dust noise. Limited real-time performance and parallel processing capabilities, existing systems rely on serial computing architectures, making them unable to meet the high-speed processing requirements of multiple video streams. Especially in scenarios such as facial feature recognition (such as eye closure and yawn detection) and head posture analysis, algorithm latency is high, making it impossible to detect abnormal conditions within milliseconds, potentially delaying safety responses. Furthermore, traditional systems lack dynamic optimization strategies for FPGA hardware resources, which can lead to problems such as uneven logic unit utilization. Feedback control accuracy is insufficient. Existing alarm strategies often use a single trigger mode (such as only sound and light alarms) and lack a graded response mechanism. They are unable to distinguish between different risk levels, such as fatigue and illegal operation. Furthermore, the coordinated control with production equipment (such as PLCs) relies on low-speed communication protocols, resulting in delays in the transmission of shutdown commands. Summary of the Invention

[0003] In view of this, in order to solve the above-mentioned problems in the prior art, the present invention proposes an industrial safety monitoring system and method, which can improve the real-time performance, anti-interference ability and multi-system coordination efficiency of industrial monitoring, and reduce the misjudgment rate and accident risk.

[0004] The present invention solves the above problems through the following technical means:

[0005] In one aspect, the present invention provides an industrial safety monitoring system, comprising:

[0006] Image acquisition module, used to capture real-time video streams of the faces and behaviors of production line workers in high-risk positions;

[0007] The core image processing module is used to perform the following operations on the video stream:

[0008] a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area;

[0009] b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The frequency of eye closure, blinking, and / or yawning is counted over a certain period. If the frequency of eye closure, blinking, and / or yawning reaches a certain threshold, fatigue is determined;

[0010] c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference;

[0011] d) Real-time statistics of attention distraction indicators, judging whether the worker has deviated from the work area based on the head posture angle. If the head posture angle reaches the preset threshold, the worker is considered distracted;

[0012] The feedback control module is used to trigger an alarm signal of a corresponding level or send a control instruction to the production equipment to suspend the production equipment according to the judgment result.

[0013] Preferably, the dynamic illumination adaptive threshold binarization includes:

[0014] By using histogram equalization to analyze the ambient light intensity in real time, the binary grayscale threshold is dynamically adjusted to adapt to factory scenes with alternating strong light and shadows. The adjustment formula for the binary grayscale threshold T is:

[0015]

[0016] Among them, μ is the average brightness of the image, σ is the standard deviation, and α and β are illumination compensation coefficients.

[0017] Preferably, dynamically adjusting the binarization grayscale threshold specifically includes:

[0018] Under dynamic lighting conditions, histogram equalization is first performed on the input video frame to eliminate brightness deviations caused by non-uniform lighting. Subsequently, the average brightness μ and standard deviation σ are calculated based on the pixel brightness values ​​of the current frame. Combined with the preset α and β coefficients, a binary threshold T is generated through linear combination. This threshold can dynamically reflect the overall intensity and local fluctuations of the ambient light. In areas with strong lighting, the μ value increases significantly, and the threshold is increased accordingly to avoid over-segmentation. In shadowed areas, the μ value decreases, but the σ value may increase due to local contrast enhancement. Threshold adjustment can preserve target edge details.

[0019] Preferably, in the core image processing module, facial feature recognition includes face detection, facial feature extraction and facial expression analysis;

[0020] Facial detection uses Haar feature classifier or HOG feature and SVM support vector machine to locate the face area;

[0021] Haar feature classifier: The Haar feature classifier represents the local features of the image by weighted rectangular areas and uses the following formula to calculate the Haar feature: ;

[0022] Among them, H(x, y) is the Haar eigenvalue of the image at position (x, y), A i (x,y) is the local area of ​​the image, w i is the weight of the corresponding area, N is the number of rectangular areas, (x, y) is the coordinate of the image, and i is the index of the i-th rectangular area;

[0023] HOG feature and SVM support vector machine: HOG feature obtains local features by calculating the direction and intensity of image gradient, and combines it with SVM support vector machine for classification. The formula for calculating image gradient is: ;

[0024] Among them, G x and G y Represent the gradients in the horizontal and vertical directions respectively, and I(x,y) is the grayscale value of the image;

[0025] The goal of facial feature extraction is to extract key points or texture information from facial images. The algorithm used is LBPH feature extraction or SIFT feature extraction;

[0026] LBPH feature extraction generates a binary pattern by comparing the grayscale values ​​of a pixel with those of its neighbors. The formula is: ;

[0027] Where LBP(x, y) is the local binary pattern value at position (x, y), I n is the neighborhood pixel, I c is the center pixel, s() is the sign function, n is the index of the neighboring pixels, N' is the number of neighboring pixels, 2 n is the weight factor;

[0028] SIFT feature extraction is done by finding key points in the scale space and extracting descriptors. The scale space is established using the following formula: ;

[0029] Among them, L(x,y,σ) is the image in the scale space, G(x,y,σ) is the Gaussian filter, I'(x,y) is the original image, represents the convolution operation, σ is the standard deviation of Gaussian blur;

[0030] Facial expression analysis infers emotional states by analyzing changes in facial features. The algorithm used is a convolutional neural network (CNN). The convolutional neural network (CNN) extracts high-level features through stacked convolutional layers and performs classification.

[0031] The convolution operation formula is: ;

[0032] Among them, X(i,j) is the pixel of the input image, K(m,n) is the convolution kernel, Y(i,y) is the output feature map, (i,j) represents the position in the image, and (m,n) represents the position of the convolution kernel.

[0033] Preferably, the multi-stage morphological filtering specifically includes:

[0034] The binary image is subjected to three erosion operations in sequence to remove noise points, and then two dilation operations are performed to restore the target contour. The filter kernel size is 5×5;

[0035] The corrosion operation formula is as follows:

[0036] ; Where E is the result after corrosion, A is the input binary image, and B is the corrosion kernel, which is a 5×5 matrix;

[0037] The expansion operation formula is as follows:

[0038] ; Where D is the result after expansion, A is the input binary image, and B is the expansion kernel, which is a 5×5 matrix.

[0039] Preferably, when determining the fatigue state, the eye closing frequency formula is as follows: ;

[0040] in, F is the eye closing frequency for fatigue status determination, T blink Is the duration of eye closure within 10 minutes, T cycle It is a total duration of 10 minutes;

[0041] The formula for calculating blink frequency is: ;

[0042] Among them, N blinks is the number of blinks, T total is the total monitoring time, f blink Indicates blink frequency;

[0043] When determining whether a worker has deviated from the work area during distraction, the formula based on the head posture angle is as follows:

[0044] ;

[0045] Where P is the total attitude deviation angle, θ pitch is the pitch angle of the head, θ yaw is the yaw angle of the head;

[0046] The rotation matrix of the head is expressed as:

[0047] ;

[0048] Among them, θ yaw is the yaw angle, and R is the rotation matrix.

[0049] Preferably, in order to comprehensively evaluate the fatigue state, the core image processing module combines multiple detection indicators to determine, including eye closing frequency, blinking frequency and head posture angle, the fatigue evaluation value F fatigue The calculation is performed by weighted sum:

[0050] ;

[0051] Among them, w1, w2 and w3 are the corresponding weight coefficients, f blink is the blink frequency, P is the head posture angle, F is the eye closing frequency.

[0052] Preferably, the core image processing module further includes:

[0053] The fatigue status determination unit is used to count the frequency of eye closure in a 10-minute cycle. If the eye closure duration accounts for ≥ 20%, fatigue is determined;

[0054] The attention distraction judgment unit is used to judge whether the worker has deviated from the work area by checking whether the head pitch angle is ≥30° or the yaw angle is ≥45°.

[0055] Preferably, the feedback control module is configured as follows:

[0056] Grading alarm strategy: the yellow LED flashes and low-frequency beeps are triggered in the "fatigue" state, and the red LED is always on and high-frequency beeps are triggered in the "illegal operation" state;

[0057] Send equipment shutdown instructions to PLC via Modbus-TCP protocol;

[0058] The industrial safety monitoring system supports integration with the MES system, including:

[0059] Upload worker status data to the MES database via the OPC UA protocol;

[0060] Generate security incident reports in real time and push them to management terminals;

[0061] The formula for generating a security incident report is as follows: ;

[0062] Where: R is the security incident report, E is the detected abnormal event data, and C is the supplementary information generated by the system. f (E, C) is a report generation function used to comprehensively analyze E and C.

[0063] In another aspect, the present invention provides an industrial safety monitoring method, comprising the following steps:

[0064] Real-time video streams capturing the faces and behaviors of production line workers in high-risk positions;

[0065] Perform the following operations on the video stream in sequence:

[0066] a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area;

[0067] b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The frequency of eye closure, blinking, and / or yawning is counted over a certain period. If the frequency of eye closure, blinking, and / or yawning reaches a certain threshold, fatigue is determined;

[0068] c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference;

[0069] d) Real-time statistics of attention distraction indicators, judging whether the worker has deviated from the work area based on the head posture angle. If the head posture angle reaches the preset threshold, the worker is considered distracted;

[0070] According to the judgment result, an alarm signal of corresponding level is triggered or a control instruction is sent to the production equipment to suspend the production equipment.

[0071] Compared with the prior art, the beneficial effects of the present invention include at least:

[0072] The present invention uses an image acquisition module to capture worker behavior data in real time, and combines it with an FPGA parallel architecture to implement dynamic lighting adaptive processing, multi-level morphological filtering, and abnormal state analysis. This solves the problems of insufficient real-time performance, poor environmental adaptability, and low multi-module collaboration efficiency of traditional systems in complex industrial scenarios. It has the advantages of improving the real-time performance of high-risk job safety monitoring, enhancing adaptability to complex industrial environments, improving the accuracy of worker status recognition, and optimizing system collaboration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0074] Figure 1 It is a structural diagram of the industrial safety monitoring system of the present invention;

[0075] Figure 2 It is the image processing flow chart of the present invention;

[0076] Figure 3 This is a fatigue state determination logic diagram of the present invention;

[0077] Figure 4 It is a schematic diagram of the feedback control strategy of the present invention;

[0078] Figure 5 is a flow chart of the industrial safety monitoring method of the present invention;

[0079] Figure 6 It is a structural block diagram of the electronic equipment of the present invention. DETAILED DESCRIPTION

[0080] To make the above-mentioned objectives, features, and advantages of the present invention more clearly understood, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are also within the scope of protection of the present invention. Example 1

[0081] Existing industrial safety monitoring systems often rely on general-purpose processor architectures for visual analysis, which suffer from insufficient adaptability to dynamic environments and limited real-time processing capabilities. For example, fixed-threshold binarization algorithms cannot accurately segment target areas when illumination changes dramatically, and serial computing architectures result in excessive latency in facial feature recognition. Furthermore, traditional systems struggle with high-speed caching and processing of video data due to bus protocol bandwidth limitations and cross-clock synchronization flaws. Furthermore, they lack a tiered alarm strategy, preventing them from quickly responding to abnormal conditions at varying risk levels.

[0082] To address these challenges, an efficient monitoring system adaptable to complex industrial environments is needed. Adaptive image processing algorithms are required to account for the impact of dynamic lighting and dust interference on target segmentation. To meet real-time requirements, a hardware acceleration architecture is needed to reduce processing latency. To achieve precise control, a hierarchical alarm and device linkage mechanism is required. To address these requirements, a comprehensive solution is developed by combining dynamic threshold adjustment, a parallel computing architecture, and multi-stage filtering.

[0083] Therefore, if Figure 1 As shown, the present invention proposes an industrial safety monitoring system, which includes an image acquisition module, a core image processing module and a feedback control module.

[0084] The image acquisition module is used to capture real-time video streams of the faces and behaviors of production line workers in high-risk positions.

[0085] The core image processing module is used to perform the following operations on the video stream:

[0086] a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area;

[0087] b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The frequency of eye closure, blinking, and / or yawning is counted over a certain period. If the frequency of eye closure, blinking, and / or yawning reaches a certain threshold, fatigue is determined;

[0088] c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference;

[0089] d) Real-time statistics of attention distraction indicators are collected, and the head posture angle is used to determine whether the worker has deviated from the work area. If the head posture angle reaches the preset threshold, the worker is considered distracted.

[0090] The feedback control module is used to trigger an alarm signal of a corresponding level or send a control instruction to the production equipment to suspend the production equipment according to the judgment result.

[0091] The image acquisition module is equipped with an industrial-grade camera and a driver circuit to capture the worker's facial and behavioral video stream in real time; the core image processing module is based on an FPGA parallel architecture and sequentially performs grayscale conversion and dynamic illumination adaptive threshold binarization, skin color recognition and head posture analysis, multi-level morphological filtering, and attention distraction index statistics, such as Figure 2 As shown; the feedback control module includes an audible and visual alarm unit and a device control interface to trigger an alarm or pause the device.

[0092] The image acquisition module is the hardware unit used to acquire video data. Specifically, it can be implemented using an industrial-grade camera in conjunction with a driver circuit, ensuring the continuity of the target's movements through high-frame-rate acquisition. The core image processing module is the computational unit that performs image analysis. Specifically, it can utilize an FPGA parallel architecture to accelerate algorithms and improve computational efficiency through pipeline processing. The feedback control module is the execution unit that executes safety responses. Specifically, it can utilize an audible and visual alarm device and a PLC interface to achieve a multi-level response to abnormal conditions.

[0093] Specifically, the image acquisition module continuously captures the video stream and transmits it to the storage module for caching; the core processing module first performs grayscale conversion and dynamic threshold segmentation to extract the moving target area, then locates the facial area through skin color recognition and analyzes the head posture, and then uses multi-level morphological filtering to eliminate environmental noise. Finally, the attention index is statistically analyzed to determine the abnormal state; the feedback module triggers the corresponding level of alarm signal or sends control instructions to the production equipment based on the judgment result.

[0094] Compared with the existing technology, the traditional system uses fixed threshold binarization and is easily affected by light interference, while the present invention adapts to environmental changes by dynamically adjusting the threshold; the existing method relies on serial calculations, resulting in processing delays, while the present invention uses FPGA parallel architecture to achieve synchronous execution of multiple algorithms; conventional filtering methods have difficulty eliminating high-frequency noise, while the present invention improves anti-interference capabilities through multi-level morphological operations; traditional alarm strategies lack a grading mechanism, while the present invention matches different response modes according to the type of anomaly.

[0095] Through the above technical solution, the present invention can effectively overcome the influence of sudden changes in industrial lighting and dust interference on target segmentation, accurately identify workers' facial features and behavioral status; shorten algorithm delays through parallel processing architecture, and achieve millisecond-level abnormal state judgment; utilize high-speed bus protocols and circular storage mechanisms to ensure continuous processing of video data; establish a hierarchical alarm strategy to improve the accuracy and timeliness of safety response.

[0096] The present invention further proposes a dynamic illumination adaptive threshold binarization method, which includes real-time analysis of ambient light intensity through histogram equalization and dynamic adjustment of the binarization grayscale threshold to adapt to factory scenes with alternating strong light and shadows. The threshold adjustment formula is: , where μ is the average brightness of the image, σ is the standard deviation, and α and β are illumination compensation coefficients.

[0097] Among them, histogram equalization refers to a processing technology that enhances image contrast by redistributing the brightness values ​​of image pixels. It can be implemented by calculating the brightness histogram in real time and generating a mapping function to eliminate local overexposure or underexposure caused by sudden changes in illumination. Dynamic adjustment of the binarization grayscale threshold refers to automatically updating the segmentation threshold based on the brightness statistical characteristics of the current scene. It can be implemented by calculating the mean and standard deviation of the image area and combining them with empirical coefficients for linear combination, and is used to maintain the stability of target segmentation in an environment with alternating strong light and shadow. The illumination compensation coefficients α and β refer to weight parameters used to balance the overall brightness and local contrast of the image. They can be determined through offline calibration or online learning, and are used to adaptively adjust the illumination characteristics of different industrial scenes.

[0098] Specifically, under dynamic lighting conditions, histogram equalization is first performed on the input video frame to eliminate brightness deviations caused by non-uniform lighting. Subsequently, the average brightness μ and standard deviation σ are calculated based on the pixel brightness values ​​of the current frame. Combined with the preset α and β coefficients, a binary threshold T is generated through linear combination. This threshold dynamically reflects the overall intensity and local fluctuations of the ambient light. For example, in areas of strong illumination, the μ value increases significantly, and the threshold is increased accordingly to avoid over-segmentation. In shadowed areas, the μ value decreases, but the σ value may increase due to local contrast enhancement. Threshold adjustment preserves target edge details. As a result, the binarization process can accurately distinguish between moving workers and background interference areas.

[0099] Compared to existing technologies, traditional fixed-threshold binarization methods are prone to incomplete target segmentation or false noise detection when sudden changes in lighting occur. For example, target details are lost in strong light or shadows are misidentified as foreground. However, this invention utilizes a dynamic threshold adjustment mechanism, combining both the mean and standard deviation of brightness to effectively adapt to the frequently changing lighting conditions in factory environments while simultaneously suppressing the impact of high-frequency noise, such as dust, on the binarization results.

[0100] Through the above technical solution, the present invention can achieve robust motion target extraction in complex industrial lighting environments, significantly reduce the target segmentation error rate caused by sudden changes in illumination, and provide accurate binary image input for subsequent worker status recognition algorithms, thereby improving the reliability of the entire monitoring system in real industrial scenarios.

[0101] Facial feature recognition algorithms aim to extract key facial information from input images, including face detection, feature extraction, expression analysis, etc.

[0102] Facial detection is the basis of facial feature recognition, and Haar feature classifier or HOG (Histogram of Oriented Gradients) + SVM (Support Vector Machine) is usually used to locate the face area.

[0103] Haar feature classifier: The Haar feature classifier represents the local features of the image by weighted rectangular areas. The Haar feature can be calculated using the following formula:

[0104]

[0105] Among them, H(x, y) is the Haar eigenvalue of the image at position (x, y), A i (x,y) is the local area of ​​the image, w i is the weight of the corresponding area, N is the number of rectangular areas, (x, y) is the coordinate of the image, and i is the index of the i-th rectangular area.

[0106] HOG features and SVM: HOG features obtain local features by calculating the direction and intensity of image gradients and are combined with support vector machines for classification. The formula for calculating image gradients is:

[0107]

[0108] Among them, G x and G y They represent the gradients in the horizontal and vertical directions respectively, and I(x,y) is the grayscale value of the image.

[0109] The goal of facial feature extraction is to extract key points or texture information from facial images. Commonly used algorithms include LBPH and SIFT.

[0110] LBPH (Local Binary Pattern Histograms): LBPH generates a binary pattern by comparing the grayscale values ​​of a pixel with those of its neighbors. The formula is:

[0111]

[0112] Where LBP(x, y) is the local binary pattern value at position (x, y), I n is the neighborhood pixel, I c is the center pixel, s() is the sign function, n is the number index of the neighborhood pixel, which is used to compare each neighborhood pixel around the center pixel one by one when calculating LBP, N' is the number of neighborhood pixels, 2 n is the weighting factor.

[0113] SIFT feature extraction: SIFT (Scale Invariant Feature Transform) finds key points in the scale space and extracts descriptors. The scale space is established using the following formula:

[0114]

[0115] Among them, L(x,y,σ) is the image in the scale space, G(x,y,σ) is the Gaussian filter, I'(x,y) is the original image, represents the convolution operation, and σ is the standard deviation of Gaussian blur.

[0116] Facial expression analysis infers emotional states by analyzing changes in facial features. A commonly used algorithm is the convolutional neural network (CNN). CNN extracts high-level features through stacked convolutional layers and performs classification.

[0117] The convolution operation formula is:

[0118]

[0119] Among them, X(i,j) is the pixel of the input image, K(m,n) is the convolution kernel, Y(i,y) is the output feature map, (i,j) represents the position in the image, and (m,n) represents the position of the convolution kernel.

[0120] Fatigue detection identifies fatigue status in real time by monitoring physiological and behavioral characteristics. Common methods include blink detection, eye closure detection, head posture detection and comprehensive evaluation models.

[0121] Blink detection detects fatigue by analyzing the state of eye opening and closing, usually by locating the eye area through Haar feature classifier. Calculate the blink frequency (f blink ) is:

[0122]

[0123] Among them, N blinks is the number of blinks, T total is the total monitoring time, f blink Indicates blink frequency.

[0124] The formula for eye closing frequency is as follows:

[0125]

[0126] in, F is the eye closing frequency for fatigue status determination, T blink Is the duration of eye closure within 10 minutes, T cycle It is a total duration of 10 minutes;

[0127] Blink rate reflects the number of blinks per unit time and is suitable for identifying attention fluctuations or early signs of fatigue. Eye closure rate (percentage of time closed) focuses on the duration of eye closure over the entire cycle and serves as the primary basis for determining fatigue. Combining these two metrics improves the sensitivity and robustness of fatigue detection.

[0128] When determining whether a worker has deviated from the work area during distraction, the formula based on the head posture angle is as follows:

[0129]

[0130] Where P is the total attitude deviation angle, θ pitch is the pitch angle of the head, θ yaw is the yaw angle of the head.

[0131] Head posture detection assesses fatigue by calculating the angles of the head (pitch, roll, and yaw). The rotation matrix of the head can be expressed as:

[0132]

[0133] Among them, θ yaw is the yaw angle, and R is the rotation matrix. The yaw angle, pitch angle, and roll angle are usually calculated by a 3D head pose estimation model.

[0134] In order to comprehensively evaluate the fatigue state, multiple detection indicators (such as blinking frequency, eye closing frequency, head posture, etc.) can be combined. Fatigue assessment value (F fatigue ) can be calculated by weighted sum:

[0135]

[0136] Among them, w1, w2 and w3 are the corresponding weight coefficients, f blink is the blink frequency, P is the head posture angle, F is the eye closing frequency.

[0137] To improve the accuracy and real-time performance of the algorithm, the following are possible optimization measures:

[0138] Data enhancement: Enhance the generalization ability of the model by rotating, flipping, scaling, and other operations on the training data.

[0139] Accelerated model: Use a lightweight network architecture (such as MobileNet) and use GPU for computational acceleration.

[0140] Multimodal fusion: Combining multiple sensors (such as infrared cameras, eye trackers, etc.) to perform multimodal information fusion to further improve the accuracy of fatigue detection.

[0141] The present invention further proposes a multi-level morphological filtering method, which specifically includes performing three corrosion operations on the binary image to remove noise points, and then performing two dilation operations to restore the target contour, and the filter kernel size is 5×5.

[0142] The corrosion operation formula is as follows:

[0143]

[0144] Among them, E is the result after corrosion, A is the input binary image, and B is the corrosion kernel, which is a 5×5 matrix;

[0145] The expansion operation formula is as follows:

[0146]

[0147] Among them, D is the result after expansion, A is the input binary image, and B is the expansion kernel, which is a 5×5 matrix.

[0148] The erosion operation involves performing a sliding window logical AND operation on the structuring element and the target image to eliminate isolated noise points. This can be achieved by traversing the binary image pixel by pixel using a 5×5 rectangular structuring element. This operation effectively removes dust and noise particles attached to the edges of the target. The dilation operation involves performing a sliding window logical OR operation on the structuring element and the target image to fill holes or connect broken areas. This can be achieved by performing a reverse compensation on the eroded image using a structuring element of the same size. This operation can reconstruct the complete outline of the worker's moving target after noise removal. The filter kernel size refers to the geometric dimensions of the structuring element in morphological operations. Specifically, a 5×5 pixel square template can be used. This size effectively covers the size range of typical dust particles in industrial scenarios while ensuring computational efficiency.

[0149] Specifically, in industrial environments with severe dust interference, the system first performs three consecutive erosion operations on the binary image, removing high-frequency dust interference by gradually removing small noise points. It then performs two dilation operations to restore the outline of over-eroded worker limbs or tools by expanding the pixel area. Using a fixed 5×5 filter kernel size, a parallel processing pipeline is constructed using shift registers at the FPGA hardware level to ensure millisecond-level image processing latency per frame.

[0150] Compared to existing technologies, existing morphological filtering methods often rely on a single erosion or dilation operation, which can easily lead to residual noise or distortion of target contours in industrial dust scenarios. Traditional techniques employ a single loop structure that alternates erosion and dilation, making it difficult to balance the conflicting requirements of noise elimination and target preservation. Three erosion operations completely eliminate dust particles, while two dilation operations accurately restore the target shape, avoiding the contour distortion problems often encountered with conventional methods due to inappropriate number of dilation operations.

[0151] Through the above technical solution, the present invention can accurately separate the worker's moving targets and noise in a strong dust interference environment, eliminate the image area missegmentation phenomenon caused by dust adhesion, provide high-completeness target area input data for subsequent fatigue detection and posture analysis, and effectively reduce the misjudgment rate of the safety monitoring system.

[0152] like Figure 3 As shown, the core image processing module of the present invention further includes a fatigue state determination unit and an attention distraction determination unit. The fatigue state determination unit counts the eye closing frequency in a certain period, and determines fatigue if the eye closing duration reaches a specific threshold; the attention distraction determination unit determines whether the worker has deviated from the work area through the head posture angle.

[0153] The fatigue state determination unit refers to a calculation unit that analyzes the worker's fatigue level by periodically monitoring the proportion of eye closure time. This can be achieved by combining a timer with a state machine to periodically collect eye feature data and calculate the proportion of eye closure time. By quantifying the temporal distribution characteristics of eye closure behavior, this unit can eliminate instantaneous movement interference and improve the stability of fatigue determination. The attention distraction determination unit refers to a detection module that analyzes work concentration based on the spatial angle of the head. This can be achieved by using an inertial measurement unit and a visual fusion algorithm. By establishing a spatial mapping relationship between the head posture angle and the work area, this unit can distinguish between normal operations and violations, avoiding misjudgments due to perspective deviations.

[0154] Specifically, the fatigue state judgment unit continuously collects data on the worker's eye closure status by setting a periodic monitoring window, for example, at intervals of 10 minutes. When the cumulative duration of eye closure exceeds 20% of the window time, the fatigue judgment flag is triggered. This judgment method filters out accidental eye-closing movements, such as blinking or short-term eye closure, through the cumulative effect of time, so as to accurately identify the characteristics of persistent fatigue. The attention distraction judgment unit calculates the spatial offset of the head pitch angle and yaw angle. For example, when the pitch angle exceeds 30 degrees or the yaw angle exceeds 45 degrees, it determines that the worker's line of sight is out of the work area. This judgment logic is set through the three-dimensional spatial angle threshold, which can effectively distinguish between normal working postures and illegal operating behaviors.

[0155] Compared to existing technologies, existing systems typically use momentary eye closure detection or single-angle thresholds to determine distraction. For example, these systems only detect the duration of a single eye closure or set a fixed yaw angle threshold. These methods are susceptible to environmental interference or sudden changes in movement, such as dust causing misidentification of eye features or false alarms caused by a worker's temporary head turn. The present invention, through periodic statistics combined with a multi-angle joint judgment mechanism, eliminates momentary interference factors and achieves more reliable abnormal behavior identification.

[0156] Through this technical solution, the present invention solves the problem of fatigue misjudgment caused by instantaneous detection errors in existing systems, while also reducing the underreporting rate caused by single-angle threshold limitations in head posture analysis. By combining time window statistics with multi-angle judgment, it effectively distinguishes normal operating movements from dangerous behaviors, ensuring the accuracy of safety alarm triggering and avoiding equipment shutdowns or alarm delays caused by misjudgment.

[0157] like Figure 4 As shown, the present invention further proposes that the feedback control module is configured as a hierarchical alarm strategy, which triggers the yellow LED to flash and a low-frequency buzzer in a fatigue state, triggers the red LED to be constantly on and a high-frequency buzzer in an illegal operation state, and sends a device shutdown instruction to the PLC through the Modbus-TCP protocol.

[0158] The hierarchical alarm strategy involves triggering differentiated alarm signals based on the type of abnormal condition. This can be achieved using a multi-color LED and multi-frequency buzzer control circuit, with the status recognition results driving the switching of different alarm modes. The Modbus-TCP protocol is an industrial communication protocol based on a TCP / IP network. It can be implemented using a protocol stack hardware acceleration module, connecting to the PLC via an Ethernet physical layer interface.

[0159] Specifically, when the core image processing module detects worker fatigue, it triggers the yellow LED to flash at a preset frequency and simultaneously activates a low-frequency buzzer to emit intermittent warning sounds. If the worker's head posture is detected to be outside the safe range or an illegal operation is detected, the LED immediately switches to a continuous red light and a high-frequency buzzer to sound a continuous alarm. Simultaneously, the embedded protocol stack parses the abnormal event code, generates a data frame conforming to the Modbus-TCP format, and sends an equipment shutdown command to the PLC controller via the physical layer interface, enabling rapid, coordinated control of production equipment.

[0160] Compared with existing technologies, existing alarm systems often use a single trigger mode, unable to distinguish abnormal conditions of different risk levels, and device control relies on low-speed bus protocols such as RS485. This invention establishes a mapping relationship between status type and alarm mode to achieve precise abnormality classification warnings. It also uses the Modbus-TCP protocol instead of traditional serial communication to improve command transmission speed and network reliability.

[0161] Through the above technical solution, the present invention solves the problems of single alarm strategy and equipment control delay in traditional monitoring systems, realizes classified warnings of abnormal conditions and rapid response of production equipment, and avoids the expansion of safety accidents caused by confusion of alarm information or delay in command transmission.

[0162] This invention further proposes that the industrial safety monitoring system supports integration with the MES system, including uploading worker status data to the MES database via the OPC UA protocol, and generating real-time safety event reports and pushing them to the management terminal. The formula for generating safety event reports is as follows:

[0163]

[0164] Where: R is the security incident report, E is the detected abnormal event data, and C is the supplementary information generated by the system. f (E, C) is a report generation function used to comprehensively analyze E and C.

[0165] The OPC UA protocol is an industrial communication protocol that supports cross-platform data exchange. It can be implemented using a standardized client-server architecture. By defining a unified data model, it enables real-time data exchange between heterogeneous systems, addressing the inefficiency of collaboration caused by incompatibility between existing systems and the MES platform's communication protocols. Real-time security event reporting automatically generates structured log files based on timestamps and abnormal event types. Data can be encapsulated in XML or JSON format, and reports are generated and pushed through a preset event trigger mechanism. This addresses the issue of managers being unable to obtain accurate event information in a timely manner.

[0166] Specifically, the system uploads collected worker facial recognition results, attention indicators, and equipment control status to the MES database in real time via the OPC UA protocol, seamlessly integrating production safety data with enterprise-level management systems. When the core image processing module detects an abnormal worker condition, the system automatically triggers the event report generation process, extracting key information such as the time of occurrence, location coordinates, and abnormality type. It then generates a standardized report file based on a predefined template and pushes it to the manager's mobile terminal or workstation via the MQTT protocol. This creates a closed-loop information flow from on-site monitoring to management decision-making, ensuring that abnormal events are identified and handled in the shortest possible time.

[0167] Compared to existing technologies, existing systems typically use Modbus or HTTP protocols for data upload, which can lead to complex protocol conversions and poor real-time performance. This invention, however, uses the OPC UA protocol to directly connect to the MES database, reducing intermediate protocol conversion steps and improving data transmission efficiency. Furthermore, traditional security incident reporting relies on manual recording and aggregation, which can easily lead to information omissions or delays. This invention ensures the integrity and timeliness of incident information through automated report generation and push mechanisms.

[0168] Through the above technical solution, the present invention can realize the real-time integration of industrial safety monitoring data and enterprise information management systems, improve the response speed of abnormal events and the accuracy of management decisions, and at the same time eliminate communication barriers between different systems through standardized data formats, providing full-link data support for production safety management.

[0169] Through the above technical solution, the present invention realizes real-time synchronization of worker status data and the production management system, ensuring that abnormal events can be recorded and trigger management responses in a timely manner, effectively solving the problems of information islands and response delays in the existing system. Example 2

[0170] like Figure 5 As shown, the present invention provides an industrial safety monitoring method, comprising the following steps:

[0171] Real-time video streams capturing the faces and behaviors of production line workers in high-risk positions;

[0172] Perform the following operations on the video stream in sequence:

[0173] a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area;

[0174] b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The frequency of eye closure, blinking, and / or yawning is counted over a certain period. If the frequency of eye closure, blinking, and / or yawning reaches a certain threshold, fatigue is determined;

[0175] c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference;

[0176] d) Real-time statistics of attention distraction indicators, judging whether the worker has deviated from the work area based on the head posture angle. If the head posture angle reaches the preset threshold, the worker is considered distracted;

[0177] According to the judgment result, an alarm signal of corresponding level is triggered or a control instruction is sent to the production equipment to suspend the production equipment.

[0178] The other features of this embodiment are the same as those of embodiment 1, so they will not be repeated here. Example 3

[0179] Based on the same concept, the present invention also provides a schematic diagram of a physical structure, such as Figure 6As shown, the server may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the steps of the industrial safety monitoring method.

[0180] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks. Example 4

[0181] Based on the same concept, the present invention also provides a non-transitory computer-readable storage medium, which stores a computer program. The computer program includes at least one segment of code, which can be executed by a main control device to control the main control device to implement the steps of the industrial safety monitoring method.

[0182] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in this application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive).

[0183] Those skilled in the art will appreciate that all or part of the process steps in the above-described method embodiments can be implemented by a computer program instructing the relevant hardware. The program can be stored in a computer-readable storage medium. When executed, the program can include the process steps in the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

[0184] The above-described embodiments merely illustrate several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. An industrial safety monitoring system, characterized in that: include: Image acquisition module, used to capture real-time video streams of the faces and behaviors of production line workers in high-risk positions; The core image processing module is used to perform the following operations on the video stream: a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area; b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The eye closure frequency, blinking frequency, and / or yawning frequency are counted over a certain period. If the eye closure frequency, blinking frequency, and / or yawning frequency reaches a threshold, fatigue is determined; c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference; d) Real-time statistics of attention distraction indicators, judging whether the worker has deviated from the work area by head posture angle. If the head posture angle reaches the preset threshold, the worker is considered distracted; A feedback control module is used to trigger an alarm signal of a corresponding level or send a control instruction to the production equipment to suspend the production equipment according to the judgment result; The dynamic illumination adaptive threshold binarization includes: By using histogram equalization to analyze the ambient light intensity in real time, the binary grayscale threshold is dynamically adjusted to adapt to factory scenes with alternating strong light and shadows. The adjustment formula for the binary grayscale threshold T is: T=α·μ+β·σ Where μ is the average brightness of the image, σ is the standard deviation, and α and β are the illumination compensation coefficients; Dynamically adjusting the binary grayscale threshold specifically includes: Under dynamic lighting conditions, histogram equalization is first performed on the input video frame to eliminate brightness deviations caused by non-uniform lighting. Subsequently, the average brightness μ and standard deviation σ are calculated based on the pixel brightness values ​​of the current frame. Combined with the preset α and β coefficients, a binary threshold T is generated through linear combination. This threshold can dynamically reflect the overall intensity and local fluctuations of the ambient light. In areas with strong lighting, the μ value increases significantly, and the threshold is increased accordingly to avoid over-segmentation. In shadowed areas, the μ value decreases but the σ value increases due to local contrast enhancement. Threshold adjustment can preserve target edge details.

2. The industrial safety monitoring system according to claim 1, characterized in that: In the core image processing module, facial feature recognition includes face detection, facial feature extraction and facial expression analysis; Facial detection uses Haar feature classifier or HOG feature and SVM support vector machine to locate the face area; Haar feature classifier: The Haar feature classifier represents the local features of the image by weighted rectangular areas and uses the following formula to calculate the Haar feature: Among them, H(x,y) is the Haar eigenvalue of the image at position (x,y), A i (x,y) is the local area of ​​the image, w i is the weight of the corresponding area, N is the number of rectangular areas, (x, y) is the coordinate of the image, and i is the index of the i-th rectangular area; HOG feature and SVM support vector machine: HOG feature obtains local features by calculating the direction and intensity of image gradient, and combines it with SVM support vector machine for classification. The formula for calculating image gradient is: G x =I(x+1,y)-I(x-1,y),G y =I(x,y+1)-I(x,y-1) Among them, G x and G y Represent the gradients in the horizontal and vertical directions respectively, and I(x,y) is the grayscale value of the image; The goal of facial feature extraction is to extract key points or texture information from facial images. The algorithm used is LBPH feature extraction or SIFT feature extraction; LBPH feature extraction generates a binary pattern by comparing the grayscale values ​​of a pixel with those of its neighbors. The formula is: Where LBP(x,y) is the local binary pattern value at position (x,y), I n is the neighborhood pixel, I c is the center pixel, s() is the sign function, n is the number index of the neighborhood pixel, N' is the number of neighborhood pixels, 2 n is the weight factor; SIFT feature extraction is done by finding key points in the scale space and extracting descriptors. The scale space is established using the following formula: L(x,y,σ)=G(x,y,σ)*I'(x,y) Among them, L(x,y,σ) is the image in scale space, G(x,y,σ) is the Gaussian filter, and I'(x,y) is the original image, * represents the convolution operation, and σ is the Gaussian modulus. Facial expression analysis infers emotional state by analyzing changes in facial features. The algorithm used is the convolutional neural network (CNN). The convolutional neural network (CNN) extracts high-level features through stacked convolutional layers and performs classification. The convolution operation formula is: Among them, X(i,j) is the pixel of the input image, K(m,n) is the convolution kernel, and Y(i,y) is the output feature map. (i, j) represents the position in the image, and (m, n) represents the position of the convolution kernel.

3. The industrial safety monitoring system according to claim 1, characterized in that: The multi-stage morphological filtering specifically includes: The binary image is subjected to three erosion operations in sequence to remove noise points, and then two dilation operations are performed to restore the target contour. The filter kernel size is 5×5; The corrosion operation formula is as follows: Among them, E is the result after corrosion, A is the input binary image, and B is the corrosion kernel, which is a 5×5 matrix; The expansion operation formula is as follows: Among them, D is the result after expansion, A is the input binary image, and B is the expansion kernel, which is a 5×5 matrix.

4. The industrial safety monitoring system according to claim 1, characterized in that: When judging the fatigue state, the eye closing frequency formula is as follows: Among them, F is the eye closing frequency for fatigue status determination, T blink Is the duration of eye closure within 10 minutes, T cycle It is a total duration of 10 minutes; The formula for calculating blink frequency is: Among them, N blinks is the number of blinks, T total is the total monitoring time, f blink Indicates blink frequency; When determining whether a worker has deviated from the work area during distraction, the formula based on the head posture angle is as follows: Where P is the total attitude deviation angle, θ pitch is the pitch angle of the head, θ yaw is the yaw angle of the head; The rotation matrix of the head is expressed as: Among them, θ yaw is the yaw angle, and R is the rotation matrix.

5. The industrial safety monitoring system according to claim 4, characterized in that: In the core image processing module, in order to comprehensively evaluate the fatigue state, multiple detection indicators are combined to determine, including eye closing frequency, blinking frequency and head posture angle, and the fatigue evaluation value F fatigue The calculation is performed by weighted sum: F fatigue =w1·f blink +w2·P+w3·F Among them, w1, w2 and w3 are the corresponding weight coefficients, f blink is the blink frequency, P is the head posture angle, and F is the eye closing frequency.

6. The industrial safety monitoring system according to claim 1, characterized in that: The core image processing module further includes: Fatigue status determination unit, used to count the frequency of eye closure in a 10-minute cycle, and if the eye closure duration accounts for ≥ 20%, fatigue is determined; The attention distraction judgment unit is used to judge whether the worker has deviated from the work area by checking whether the head pitch angle is ≥30° or the yaw angle is ≥45°.

7. The industrial safety monitoring system according to claim 1, characterized in that: The feedback control module is configured as follows: A graded alarm strategy triggers a flashing yellow LED and a low-frequency buzzer in the "fatigue" state, and a solid red LED and a high-frequency buzzer in the "illegal operation" state; Send equipment shutdown instructions to PLC via Modbus-TCP protocol; The industrial safety monitoring system supports integration with the MES system, including: Upload worker status data to the MES database via the OPC UA protocol; Generate security incident reports in real time and push them to management terminals; The formula for generating a security incident report is as follows: R=f(E,C) Where: R is the security event report, E is the detected abnormal event data, C is the supplementary information generated by the system, and f(E,C) is the report generation function used to comprehensively analyze E and C.

8. An industrial safety monitoring method, characterized in that: The steps include: Real-time video streams capturing the faces and behaviors of production line workers in high-risk positions; Perform the following operations on the video stream in sequence: a) Perform grayscale conversion and dynamic illumination adaptive threshold binarization to extract the worker movement target area; b) Based on the worker's target movement area, skin color recognition and head posture analysis algorithms are used to locate the worker's face and identify features of eye closure, blinking, and / or yawning. The eye closure frequency, blinking frequency, and / or yawning frequency are counted over a certain period. If the eye closure frequency, blinking frequency, and / or yawning frequency reaches a threshold, fatigue is determined; c) Perform multi-level morphological filtering to eliminate environmental dust and light noise interference; d) Real-time statistics of attention distraction indicators, judging whether the worker has deviated from the work area by head posture angle. If the head posture angle reaches the preset threshold, the worker is considered distracted; According to the judgment result, an alarm signal of corresponding level is triggered or a control instruction is sent to the production equipment to suspend the production equipment; The dynamic illumination adaptive threshold binarization includes: By using histogram equalization to analyze the ambient light intensity in real time, the binary grayscale threshold is dynamically adjusted to adapt to factory scenes with alternating strong light and shadows. The adjustment formula for the binary grayscale threshold T is: T=α·μ+β·σ Where μ is the average brightness of the image, σ is the standard deviation, and α and β are the illumination compensation coefficients; Dynamically adjusting the binary grayscale threshold specifically includes: Under dynamic lighting conditions, histogram equalization is first performed on the input video frame to eliminate brightness deviations caused by non-uniform lighting. Subsequently, the average brightness μ and standard deviation σ are calculated based on the pixel brightness values ​​of the current frame. Combined with the preset α and β coefficients, a binary threshold T is generated through linear combination. This threshold can dynamically reflect the overall intensity and local fluctuations of the ambient light. In areas with strong lighting, the μ value increases significantly, and the threshold is increased accordingly to avoid over-segmentation. In shadowed areas, the μ value decreases but the σ value increases due to local contrast enhancement. Threshold adjustment can preserve target edge details.

Images