Non-contact intelligent monitoring method for fatigue state of operating personnel

Abstract

The invention relates to the technical field of personnel state monitoring, in particular to a non-contact intelligent monitoring method for a fatigue state of an operator, which comprises the following steps of: a, acquiring a face heat distribution image and a dynamic behavior video stream through a non-contact sensor array; b, extracting facial thermal features based on a multispectral fusion algorithm; c, analyzing behavior characteristics through a space-time attention neural network; d, fusing physiological and behavior characteristics, and outputting a fatigue level; and e, triggering a multi-level linkage early warning mechanism according to the fatigue level. A common direct contact type sensor brings discomfort to operators and affects normal work, manual observation has subjectivity, misjudgment is prone to occurring, and real-time and comprehensive monitoring cannot be achieved. Compared with the prior art, the non-contact physiological feature sensor and the non-contact behavior feature sensor can be used for detecting a worker together, direct contact with the worker is not needed, interference to the working state of the worker is reduced, long-time continuous monitoring can be achieved, and misjudgment is avoided.

Description

Technical Field

[0001] This invention relates to the field of personnel condition monitoring technology, and in particular to a non-contact intelligent monitoring method for the fatigue state of workers. Background Technology

[0002] In numerous fields such as industrial production and transportation, the working condition of workers has a crucial impact on production efficiency and safety. With the development of technology, people have gradually realized the importance of effectively monitoring the working condition of workers. Effective monitoring can help companies understand the working status of their employees in a timely manner, rationally allocate work tasks, improve production efficiency, and at the same time reduce the risk of safety accidents caused by poor worker condition, thus ensuring the safety of personnel and property.

[0003] In the past, monitoring worker fatigue often involved direct-contact sensors to acquire physiological signals, such as heart rate and blood pressure measurements via wearable devices. Additionally, manual observation was also relied upon, with managers using experience and subjective judgment to assess worker fatigue levels.

[0004] However, direct-contact sensors can cause discomfort to workers, affecting their normal work, and some workers also need to wear protective clothing, making sensor setup difficult. Manual observation, on the other hand, is highly subjective, prone to misjudgment, and cannot provide real-time and comprehensive monitoring of workers' conditions. Summary of the Invention

[0005] To address the technical problems of existing technologies, this invention provides a non-contact intelligent monitoring method for worker fatigue.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A non-contact intelligent monitoring method for worker fatigue includes: a) synchronously acquiring facial thermal distribution images and dynamic behavior video streams using a non-contact sensor array deployed in the work area; b) extracting facial thermal features based on a multispectral fusion algorithm: eyelid temperature gradient changes, perinasal blood flow fluctuations, and frontal lobe thermal radiation anomaly index; wherein the eyelid temperature gradient is calculated by locating the eyelid ROI region and calculating the dynamic slope of the highest / lowest temperature difference points within 10 consecutive frames, and marking an eye movement metabolic abnormality when the absolute value of the slope exceeds the baseline by 30% and lasts for more than 5 seconds; c) analyzing behavioral features using a spatiotemporal attention neural network. The evaluation criteria include: 1) Head micro-movement frequency, limb movement lag duration, and deviation of the tool's trajectory; the trajectory deviation is calculated in real time by establishing a B-spline curve model of the tool's standard operating path and calculating the Hausdorff distance between the tool tip and the standard curve. When the distance exceeds the safety tolerance for three consecutive operations and the acceleration variance decreases by more than 40%, it is determined as a decline in motor control; 2) Integrating physiological and behavioral characteristics, the fatigue level is output through a dual-channel adversarial generative network model; the generator maps physiological characteristics to the fatigue potential space, and the discriminator verifies the reliability of physiological characteristics based on behavioral characteristics; 3) Triggering a multi-level linkage early warning mechanism based on the fatigue level.

[0008] Furthermore, the non-contact sensor array described in step a includes a thermal imaging sensor and a visible light camera.

[0009] Furthermore, step a can also simultaneously acquire respiratory rhythm fluctuation coefficients and cardiac micro-vibration signals using millimeter-wave radar, and compensate for image feature errors caused by ambient light interference.

[0010] Furthermore, the multi-level linkage early warning mechanism described in step e includes: primary early warning: issuing a non-contact warning through the environmental control equipment in the work area; intermediate early warning: sending a feedback pulse through a small vibrator; and advanced early warning: locking the operation permissions of high-risk equipment and activating the emergency ventilation system.

[0011] Furthermore, it also includes: f. Establishing a personalized physiological parameter baseline through a 2-minute benchmark test before daily work, and dynamically adjusting the judgment threshold according to the work intensity.

[0012] Furthermore, it also includes: g. Adopting an edge computing-cloud platform collaborative architecture: edge nodes process more than 90% of the data in real time and generate low-latency early warning instructions, while the cloud platform aggregates multi-node data every 24 hours to train models and update edge parameters.

[0013] A non-contact intelligent monitoring system for worker fatigue, applied to a non-contact intelligent monitoring method for worker fatigue, includes: a non-contact sensor array: an infrared thermal imaging sensor, a visible light camera, and a millimeter-wave radar, deployed in the work area to acquire thermal distribution images, behavioral video streams, and physiological signals; a data acquisition node: controlling sensor switching and acquisition rate, monitoring ambient brightness and lighting conditions; a microprocessor: processing thermal images and video streams according to preset thresholds to initially identify fatigue levels; an edge computing node: verifying the reliability of fatigue determination and controlling the early warning mechanism; a cloud platform: aggregating historical data from edge nodes, training a multimodal fusion fatigue model, and predicting work risks; and optional early warning devices: a small vibrator and feedback software, connected to the edge computing node, enabling the feedback function during the early warning phase.

[0014] Furthermore, the edge computing nodes include: a benchmarking module for calibrating personalized physiological parameter baselines; a generative adversarial network module for cross-validating physiological and behavioral characteristics; and a multi-level early warning decision-making module for triggering graded responses based on fatigue levels.

[0015] The beneficial effects of this invention are: by using non-contact physiological and behavioral sensors to detect fatigue in workers, there is no need to directly contact the workers, thus reducing interference with their work status, and it can monitor continuously for a long time, avoiding misjudgment.

[0016] By using a generative adversarial network module, combined with physiological signals and behavioral characteristics, the accuracy of judgment is improved.

[0017] Through individual baseline calibration and workload correction, adjustments can be made according to the actual working environment to adapt to different jobs and scenarios. Attached Figure Description

[0018] Figure 1 : System architecture diagram of the present invention;

[0019] Figure 2 : Flowchart of the adversarial generative network model structure of this invention;

[0020] Figure 3 The tool trajectory deviation quantification flowchart of the present invention. Detailed Implementation

[0021] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0022] according to Figure 1-3This invention provides a non-contact intelligent monitoring method for worker fatigue, comprising: a) synchronously acquiring facial thermal distribution images and dynamic behavior video streams through a non-contact sensor array deployed in the work area; b) extracting facial thermal features based on a multispectral fusion algorithm: eyelid temperature gradient change values, perinasal blood flow fluctuation characteristics, and frontal lobe thermal radiation abnormality index; wherein the eyelid temperature gradient is calculated by locating the eyelid ROI region and calculating the dynamic slope of the highest / lowest temperature difference points within 10 consecutive frames, and when the absolute value of the slope exceeds the baseline by 30% and lasts for more than 5 seconds, it is marked as an eye movement metabolic abnormality; c) analyzing behavior through a spatiotemporal attention neural network. Features: Frequency of head micro-movements, duration of limb movement lag, and deviation of the tool's trajectory; the trajectory deviation is determined by establishing a B-spline curve model of the tool's standard operating path, calculating the Hausdorff distance between the tool tip and the standard curve in real time, and classifying it as a decline in motor control when the distance exceeds the safety tolerance for three consecutive operations and the tool's acceleration variance decreases by >40%; d. Fatigue level is output through a dual-channel adversarial generative network model by fusing physiological and behavioral features; the generator maps physiological features to the fatigue potential space, and the discriminator verifies the reliability of physiological features based on behavioral features; e. A multi-level linkage early warning mechanism is triggered based on the fatigue level; f. A personalized physiological parameter baseline is established through a 2-minute benchmark test before daily work, and the judgment threshold is dynamically adjusted according to the work intensity; g. An edge computing-cloud platform collaborative architecture is adopted: edge nodes process more than 90% of the data in real time and generate low-latency early warning commands, while the cloud platform aggregates multi-node data every 24 hours to train the model and update the edge parameters.

[0023] A non-contact intelligent monitoring system for worker fatigue, applied to a non-contact intelligent monitoring method for worker fatigue, includes: a non-contact sensor array; a data acquisition node; a microprocessor; an edge computing node; a cloud platform; and optional early warning devices such as a small vibrator or a feedback screen, connected to the edge computing node, which enable feedback functionality during the early warning phase.

[0024] I. Based on the above, the present invention can be configured with the following system architecture, including:

[0025] Non-contact sensor array: Infrared thermal imaging sensor (FLIR Lepton 3.5, for example): acquires facial thermal distribution images (640×480 resolution, 15fps); Visible light camera (Hikvision DS-2CD3T47WDV3, for example): acquires dynamic behavior video streams (1080P resolution, 30fps); Millimeter-wave radar (Infineon XENSIV, for example). TM 60GHz): Non-contact acquisition of respiratory rhythm and cardiac micro-vibration signals.

[0026] Data acquisition node: Based on Raspberry Pi 4B, it controls the sensor switch and acquisition rate, and monitors ambient brightness (0-10000 lux).

[0027] Microprocessor (NVIDIA Jetson Nano can be used): Runs multispectral fusion algorithms and preliminary fatigue assessment.

[0028] Edge computing node: Benchmark module: Before each day's work, staff are asked to perform a 2-minute sit-and-wait task to establish baselines for basal metabolic rate and respiratory rhythm;

[0029] Generative Adversarial Network Module: The generator maps physiological features to the fatigue latent space, and the discriminator verifies the reliability of the behavioral features;

[0030] Multi-level early warning decision module: Triggers graded responses based on fatigue level (L1-L4).

[0031] Cloud platform: Aggregates historical data from multiple edge nodes, trains a multimodal fusion fatigue model every 24 hours, and updates edge node parameters.

[0032] Optional early warning device: A small vibrator can be placed inside work clothes or other personal items. It can be wirelessly connected to a multi-level early warning decision module, occupying little space and eliminating the need for direct contact with workers for monitoring. Instead, it serves to provide early warning through vibration.

[0033] II. The key algorithm flow for determining fatigue state is as follows:

[0034] Physiological sign extraction (multispectral fusion algorithm):

[0035] Eyelid temperature gradient: Locate the eyelid ROI region (using Dlib facial landmark detection) and calculate the highest temperature difference (ΔT) over 10 consecutive frames. max ) and the lowest temperature difference (△T) min The dynamic slope of )

[0036]

[0037] When |S| exceeds 30% of the baseline and lasts for ≥5 seconds, it is marked as an oculomotor metabolic disorder.

[0038] Fluctuations in perinasal blood flow: characterized by the average rate of temperature change (ΔT / Δt) in the nasal alar region over a certain period of time in thermal imaging images.

[0039] Frontal lobe thermal radiation anomaly: Calculate the deviation ratio between the thermal radiation power density of the frontal lobe region and the baseline value.

[0040] Behavioral Feature Analysis (Spatiotemporal Attention Neural Network):

[0041] Head micro-motion frequency: calculated by the variance of the trajectory of key points on the head (such as the tip of the nose) in the video stream.

[0042]

[0043] Delayed limb movement: Compare the time difference between the expected movement and the actual movement (e.g., the speed of reaching out is lower than the threshold).

[0044] Tool trajectory deviation:

[0045] Establish a B-spline curve model of the standard tool path (control point spacing ≤ 5cm);

[0046] The Hausdorff distance (D) between the tip position of the real-time calculation tool and the standard curve;

[0047] When the D value exceeds the safety tolerance (e.g., 8 cm) for three consecutive operations and the tool's acceleration variance decreases by more than 40%, it is determined to be a decline in motion control.

[0048] Fatigue level determination (dual-channel adversarial generative network):

[0049] Generator channel: Input physiological characteristics (eyelid temperature gradient, perinasal blood flow, etc.), output candidate fatigue levels (L1-L4).

[0050] Discriminator channel: Input behavioral characteristics (head micro-movement frequency, trajectory deviation, etc.), verify the reliability of the generator output, and output the final fatigue level.

[0051] III. Based on the differences in fatigue levels, there is a multi-level linkage early warning mechanism as follows:

[0052] Different warning methods are available depending on the level of fatigue:

[0053] Primary warning (L2≤fatigue level<L3): Adjust the lighting in the work area to the warning spectrum (580-590nm) and send a text prompt to the monitoring terminal.

[0054] Intermediate warning (L3≤fatigue level<L4): Vibration feedback pulses are sent via a small vibrator (frequency 2Hz, intensity adjustable in 3 levels).

[0055] Advanced warning (fatigue level ≥ L4): Locks the operation permissions of high-risk equipment, activates the emergency ventilation system, and pushes warning information to the management personnel terminal.

[0056] When the fatigue level is L1, the operator still has normal operational capabilities. Triggering an early warning may cause unnecessary panic or interference. However, real-time updates of physiological and behavioral data can ensure that the fatigue level dynamically reflects the latest status and can be used for long-term fatigue trend analysis and model optimization.

[0057] IV. Depending on factors such as working environment conditions, the monitored fatigue levels may vary, allowing for dynamic baseline calibration and model iteration.

[0058] Threshold Correction: The fatigue judgment threshold (T) is dynamically adjusted based on the work intensity index (I, adjusted according to the handling weight and operation frequency). adj T is the corrected fatigue judgment threshold. base (Based fatigue judgment threshold):

[0059] T adj =T base ×(1+0.2I)

[0060] To avoid misjudgment of fixed thresholds in high and low intensity work scenarios.

[0061] Edge-cloud collaboration: Edge nodes process 90% of the data in real time (latency <200ms), while the cloud platform aggregates data every 24 hours to train the model and update the edge node parameters. Through individual baseline calibration and workload correction, the method of this invention can be adapted to different job types and scenarios.

[0062] In summary, this invention uses non-contact physiological and behavioral sensors to detect fatigue in workers without direct contact, thus reducing interference with their work performance. It also allows for continuous monitoring over long periods, preventing misjudgments.

[0063] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A non-contact intelligent monitoring method for fatigue state of workers, characterized in that: Comprise: a、Synchronous acquisition of facial thermal distribution images and dynamic behavior video streams through a non-contact sensor array deployed in the work area; b、Extraction of facial thermal features based on multi-spectral fusion algorithm: eyelid temperature gradient change value, nasal blood flow fluctuation characteristics, and frontal lobe thermal radiation abnormality index; wherein the eyelid temperature gradient is calculated by positioning the eyelid ROI region, and the dynamic slope of the highest / lowest temperature difference in 10 consecutive frames is calculated, and when the absolute value of the slope exceeds 30% of the baseline and lasts for more than 5 seconds, it is marked as abnormal eye movement metabolism; c、Analysis of behavior characteristics through spatio-temporal attention neural network: head micro-motion frequency, limb action lag time, and operation tool trajectory deviation; wherein the trajectory deviation is calculated by establishing a B-spline curve model of the standard operation path of the tool, and the Hausdorff distance between the tool tip position and the standard curve is calculated in real time, and when the operation distance exceeds the safety tolerance for 3 consecutive times and the acceleration variance decreases by more than 40%, it is determined that the motor control force has declined; d、Fusion of physiological and behavioral characteristics, output of fatigue level through a double-channel adversarial generative network model; the generator maps physiological characteristics to fatigue latent space, and the discriminator verifies the reliability of physiological characteristics based on behavioral characteristics; e、Triggering a multi-level linkage early warning mechanism according to the fatigue level.

2. The non-contact intelligent monitoring method for work personnel fatigue state according to claim 1, wherein: The non-contact sensor array in step a comprises a thermal imaging sensor and a visible light camera.

3. The non-contact intelligent monitoring method for work personnel fatigue state according to claim 1, wherein: Step a can also synchronously collect respiratory rhythm fluctuation coefficients and heart beat micro-vibration signals through a millimeter wave radar, and compensate for image feature errors caused by environmental light interference.

4. The non-contact intelligent monitoring method for work personnel fatigue state according to claim 1, wherein: The multi-level linkage early warning mechanism in step e comprises: Primary warning: issuing a non-contact warning through work area environmental control equipment; Intermediate warning: sending feedback pulses through a small vibrator; Advanced warning: locking high-risk equipment operation permissions and starting an emergency ventilation system.

5. The non-contact intelligent monitoring method for work personnel fatigue state according to claim 1, wherein: Further comprising: f、Establishing individual physiological parameter baseline through 2-minute benchmark test before daily work, and dynamically correcting the determination threshold according to work intensity.

6. The non-contact intelligent monitoring method for work personnel fatigue state according to claim 1, wherein: Further comprising: g、Adopting an edge computing-cloud platform collaborative architecture: edge nodes process more than 90% of data in real time and generate low-latency early warning instructions, and the cloud platform aggregates multi-node data every 24 hours to train the model and update the edge parameters.

7. A non-contact intelligent monitoring system for the fatigue state of a worker, applied to the non-contact intelligent monitoring method for the fatigue state of a worker according to claims 1-6, characterized in that: Comprise: Non-contact sensor array: infrared thermal imaging sensor, visible light camera, and millimeter wave radar, arranged in the work area to obtain thermal distribution images, behavior video streams, and physiological signals; Data acquisition node: control sensor switch and acquisition rate, monitor environmental brightness and lighting conditions; Microprocessor: process thermal images and video streams according to preset thresholds, preliminarily identify fatigue levels; Edge computing node: verify fatigue determination reliability and control early warning mechanism; Cloud platform: aggregate historical data from edge nodes, train multi-modal fusion fatigue model and predict work risk; Optional warning device: small vibrator, feedback screen, connected with edge computing node, enable feedback function in early warning stage.

8. The non-contact intelligent monitoring system for work personnel fatigue state according to claim 7, characterized in that: the edge computing node comprises: benchmarking module: calibrate personalized physiological parameter baseline; adversarial generative network module: cross-verify physiological and behavioral characteristics; multi-level early warning decision module: trigger graded response according to fatigue level.

Images