Eye movement and multi-mode sensing underground protective glasses and early warning method

Abstract

The invention relates to the technical field of underground safety monitoring and occupational health protection, and discloses an eye movement and multi-mode sensing underground protective glasses and an early warning method. And an elevation angle eye movement sensing module, an inertial sensor, a PPG and body temperature galvanic skin sensor, an acoustic component, a wireless communication module and a detachable glasses leg battery module are connected. The method comprises the steps that eye images and multi-source physiological data are collected in real time, and a multi-dimensional state judgment model is constructed; the logic contradiction between normal blood oxygen value and slow eye movement response is innovatively used for recognizing hidden carbon monoxide poisoning, accidental slipping and coma falling are distinguished through a posture and eye movement cooperation mechanism, and meanwhile the active noise reduction function is achieved. The problems that traditional equipment cannot recognize hidden poisoning and the falling false alarm rate is high are solved, and accurate sensing and early warning of physiological fatigue, poisoning symptoms and safety states of underground personnel are achieved.

Description

Technical Field

[0001] This invention relates to the field of downhole safety monitoring and occupational health protection technology, specifically to downhole protective glasses and early warning methods based on eye movement and multimodal perception. Background Technology

[0002] Currently, the underground coal mining environment is extremely complex and harsh, making operational safety a core focus of the industry. In-depth reviews of past accidents reveal that unsafe human behavior is the primary cause. The physiological and psychological state of workers, especially their level of fatigue, concentration, and awareness, directly determines the standardization and accuracy of equipment operation. Therefore, real-time individual monitoring of underground workers has become a crucial step in preventing human error and reducing accident risks at their source.

[0003] To address the aforementioned need for personnel status monitoring, existing applications mostly employ non-contact machine vision technology. A common implementation involves integrating a camera module into the brim of a mining helmet or attaching it externally to a miner's lamp assembly. During operation, the system continuously collects video stream data of the wearer's face, uses image processing algorithms to locate the eye area, and calculates the percentage of time the eyelids are closed within a certain period (PERCLOS value). When this value exceeds a preset threshold, the system determines that the person is in a state of fatigue or drowsiness and triggers a local audible and visual alarm or uploads an alarm message via a base station.

[0004] However, such existing technologies have limitations in practical applications. First, attaching monitoring devices externally to safety helmets or miners' lamps increases the load on the head and alters the center of gravity, leading to discomfort and strong resistance from miners. Furthermore, frequent head movements during operation result in unstable facial feature capture, leading to frequent false alarms. Second, relying solely on eyelid closure indicators has blind spots. In the specific environment underground, slowed reaction times are not only due to insufficient sleep but also to central nervous system depression caused by low-concentration carbon monoxide poisoning or hypoxia. Because carbon monoxide poisoning causes a cherry-red skin color, conventional visual monitoring or ordinary blood oxygen monitoring is insufficient to detect such pathological abnormalities, failing to distinguish between physiological fatigue and covert poisoning, and thus failing to provide comprehensive and effective life safety protection.

[0005] Therefore, the present invention provides downhole protective glasses and early warning methods based on eye movement and multimodal perception to address the shortcomings of the prior art. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides downhole protective glasses and an early warning method based on eye movement and multimodal perception. This solves the problems of poor wearing comfort, inability to distinguish between simple fatigue and hidden gas poisoning risks, and lack of comprehensive protection functions such as fall detection and hearing protection in existing downhole monitoring equipment.

[0007] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, the present invention provides downhole protective glasses with eye movement and multimodal perception, including a frame and temple assembly, wherein an embedded main control unit is provided inside the frame and temple assembly; The input terminals of the embedded main control unit are connected to an eye-tracking sensor module, an inertial sensor, a reflective PPG sensor, a body temperature and skin conductance sensor, and a microphone. The output terminals of the embedded main control unit are connected to a speaker and a wireless communication module. The eye-tracking sensing module is installed in the nose pad area or lower edge of the frame and temple assembly, and is configured to acquire image data of the eye using an upward-angle optical path; An inertial sensor is integrated inside the frame and temple assembly and is configured to collect head movement posture data; The reflective PPG sensor and the body temperature and skin conductance sensor are located on the inner wall of the frame and temple assembly, and the position corresponds to the wearer's temple or behind the ear area, and are configured to collect physiological signals. The microphone and speaker are respectively located on the outside of the frame and temple assembly and at the corresponding ear canal position; The embedded main control unit is configured to perform logical judgments on physiological fatigue, environmental poisoning, and fall accidents based on the collected data, and control the horn alarm.

[0008] By adopting the above technical solution, this invention highly integrates eye-tracking technology with various physiological and posture sensors into a wearable glasses carrier. This structure can not only acquire traditional vital sign data, but also obtain eye movement characteristics reflecting the brain's neural state through an elevation-angle optical path, thereby constructing a multi-dimensional status perception system for downhole workers. The layout of each sensor conforms to ergonomics; the reflective PPG sensor is located near the temporal artery to ensure signal strength, and the elevation-angle eye-tracking module avoids obstructing the field of vision, realizing accurate monitoring and proactive early warning of physiological fatigue, signs of poisoning, and fall status of personnel in complex downhole environments.

[0009] Preferably, the eye-tracking sensing module includes a miniature infrared CMOS image sensor and a near-infrared fill light; the near-infrared fill light is configured to emit near-infrared light with a wavelength range of 750 nm to 1000 nm; the miniature infrared CMOS image sensor is configured to operate in global exposure mode, and the optical axis of the miniature infrared CMOS image sensor is tilted upward relative to the horizontal line of sight to avoid the wearer's direct line of sight area.

[0010] By employing the above technical solution, near-infrared light in the 750-1000 nm wavelength range is used for supplementary illumination. This light can penetrate the dusty environment underground while remaining within the non-visible spectrum range for the human eye, thus avoiding interference with the operator's vision. Combined with a global exposure mode and an elevation-angle optical path design, it effectively captures high-speed eye movement characteristics and prevents image blurring, while ensuring that the optical components do not obstruct the operator's primary field of vision, guaranteeing operational safety.

[0011] Preferably, the embedded main control unit is also connected to an environmental sensing interface, which is configured to acquire data on carbon monoxide and methane concentrations in the environment; the environmental sensing interface is connected to a miniature gas sensor integrated into the frame and temple assembly, or to a portable gas detector carried by the wearer.

[0012] By adopting the above technical solution, the system can introduce environmental gas concentration as a key dimension for state determination, realize the data fusion of physiological characteristics and environmental parameters, and provide data support for subsequent identification of environmentally induced physiological abnormalities.

[0013] Preferably, a detachable temple battery module is provided at the end of the frame and temple assembly; the detachable temple battery module is connected by magnetic contacts or a snap-fit ​​structure, and is positioned behind the wearer's ear to form a gravitational balance weight for the eye-tracking module at the front; the circuit board on which the embedded main control unit is located integrates a backup power supply, configured to maintain system operation to support hot-swapping when the detachable temple battery module is disconnected.

[0014] By adopting the above technical solution, the weight of the battery module located behind the ear is used as a counterweight. Based on the lever balance principle, the gravitational torque generated by the eye-tracking sensing module at the front is counteracted, solving the problem of the smart glasses easily slipping off due to their heavy-set feet, thus improving wearing comfort. At the same time, combined with the onboard backup power supply design, the battery module can be hot-swapped and replaced without stopping the operation, breaking the limitation of physical battery capacity and meeting the continuous monitoring needs of ultra-long-term downhole operations.

[0015] Secondly, the present invention provides a downhole early warning method based on eye movement and multimodal perception, applied to the aforementioned downhole protective eyewear, comprising the following steps: Step S1, Real-time acquisition and preprocessing of multi-source heterogeneous data: The embedded main control unit synchronously acquires eye image data from the eye-tracking sensing module, photoplethysmography pulse wave signals from the reflective PPG sensor, skin conductance response signals from the body temperature and skin conductance sensors, head motion posture data from the inertial sensor, and environmental gas concentration data, and performs filtering and noise reduction preprocessing on the acquired data. Step S2, Deep extraction of eye features and physiological features: The embedded main control unit performs feature calculation on the preprocessed data and extracts PERCLOS value, pupil diameter, pupil light reaction speed, eye saccade speed, blood oxygen saturation value, heart rate variability index and skin conductance change trend. Step S3: Construct a three-dimensional state discrimination model based on eye movement features, blood oxygen data, and environmental data, and perform logical judgment: The embedded main control unit inputs the feature parameters extracted in step S2 into the preset logical discrimination model to determine whether the worker is in a state of simple physiological fatigue, hypoxia or overt poisoning, or covert carbon monoxide poisoning. Step S4, Fall detection and false alarm elimination based on posture and eye movement coordination: The embedded main control unit calculates the combined acceleration value based on head movement posture data and identifies the fall timing characteristics. After determining that a fall event has occurred, it uses synchronized eye movement feature data to determine the wearer's state of consciousness, thereby distinguishing between accidental slips and unconscious falls. Step S5: Perform active hearing protection, graded early warning and data reporting: Based on the judgment results of steps S3 and S4, the embedded main control unit controls the speaker to issue alarm prompts of the corresponding level, reports distress data packets through the wireless communication module, and uses the data collected by the pickup microphone to generate anti-phase sound waves for active noise reduction.

[0016] By adopting the above technical solution, this method establishes a complete closed loop from data acquisition and feature calculation to multimodal logic judgment. In particular, it introduces eye-tracking features as a direct feedback of the nervous system state, combined with circulatory system features such as blood oxygen and heart rate, as well as external environmental data, to solve the problem of false alarms or missed alarms caused by a single data source in the downhole environment, thereby improving the accuracy and reliability of safety warnings.

[0017] Preferably, in step S3, the logic for determining the state of concealed carbon monoxide poisoning is as follows: when the blood oxygen saturation value is greater than or equal to the preset lower limit threshold of normal blood oxygen, and the saccade speed of the eyeball is less than the preset threshold of slowed neural response, and the carbon monoxide concentration value in the ambient gas concentration data is greater than the minimum detection limit of the sensor, the embedded main control unit identifies the logical contradiction between the normal physiological blood oxygen reading and the suppressed neural response state, and determines it to be a state of concealed carbon monoxide poisoning.

[0018] By employing the above technical solution, this invention innovatively utilizes the characteristic that carboxyhemoglobin's cherry-red color causes conventional optical pulse oximeters to display falsely high readings. Combined with the physiological fact that carbon monoxide poisoning inhibits the central nervous system, leading to slower eye saccades, a mutually exclusive logic model is constructed. This effectively solves the industry pain point of traditional equipment failing to detect early carbon monoxide poisoning due to normal pulse oximeter readings, achieving timely early warning of covert poisoning.

[0019] Preferably, in step S4, the false alarm rejection and consciousness status confirmation mechanism includes: when a change in the timing of the combined acceleration that matches the characteristics of a fall is detected, the embedded main control unit reads the eye movement feature data during the static phase of the fall; if the eye movement feature data shows that the saccade speed of the eyeballs is greater than a preset active threshold or the blinking frequency is within the normal range, it is determined to be an accidental slip with full consciousness, and no emergency call is triggered; if the eye movement feature data shows that the saccade speed of the eyeballs approaches zero, the eyelids are closed for a long time, or the pupillary light reflex disappears, it is determined to be a fall with unconsciousness or serious injury, and an emergency call is triggered.

[0020] By employing the above technical solution and using eye-tracking signals as the gold standard for consciousness status, the initial fall detection results based on posture sensors are subjected to secondary verification. This mechanism can effectively distinguish between two distinct situations common in underground mining: immediately getting up and continuing work after a slip and falling unconscious. This reduces the false alarm rate caused by ordinary slips and avoids the waste of emergency rescue resources.

[0021] Preferably, in step S1, the preprocessing of the photoplethysmography (PPG) signal includes: the embedded main control unit uses the synchronously acquired head motion posture data as a reference signal, and uses the least mean square adaptive filtering algorithm to estimate and subtract the noise component related to head motion from the original PPG signal.

[0022] By adopting the above technical solution, the problem of PPG signal baseline drift caused by frequent head movements of downhole workers is addressed. IMU data is used to adaptively filter the photoelectric signal, effectively removing motion artifacts and ensuring the accuracy of heart rate and blood oxygen data in dynamic working scenarios.

[0023] Preferably, in step S5, the specific process of active hearing protection is as follows: the embedded main control unit performs spectrum analysis on the sound signal collected by the microphone; identifies the spectrum component with a frequency lower than the preset low frequency threshold and a stable amplitude as steady-state low-frequency noise; calculates an anti-phase sound wave signal with the same frequency and amplitude as the steady-state low-frequency noise but opposite phase; and digitally synthesizes the anti-phase sound wave signal with the human voice and alarm signal after pass-through processing and plays it through the speaker.

[0024] By adopting the above technical solution, the steady-state low-frequency noise generated by various large underground machinery is directionally eliminated, while preserving the human voice frequency band and high-frequency alarm signals. This protects the hearing health of workers and avoids the safety hazard of not being able to hear ambient warning sounds due to wearing traditional noise-canceling earplugs.

[0025] Preferably, in step S2, the PERCLOS value is calculated as follows: the upper and lower edges of the eyelids are located using a facial key point detection algorithm and the eyelid opening amplitude is calculated; the cumulative duration of closed frames with eyelid opening amplitude less than 20% of the full opening amplitude is counted within a preset time window; the cumulative duration is divided by the total duration of the preset time window to obtain the PERCLOS value.

[0026] By adopting the above technical solution, the proportion of eyelid closure time can be quantitatively calculated, providing objective and quantitative data indicators for the determination of fatigue state.

[0027] This invention provides downhole protective eyewear and an early warning method based on eye movement and multimodal perception. It offers the following advantages: 1. This invention adopts an integrated structural design, miniaturizing and integrating the eye-tracking sensing module into the nose pad area of ​​the frame, and embedding the main control unit and multimodal sensors within the temple assembly. This design does not change the physical form and center of gravity distribution of traditional mining protective glasses, eliminating the need for miners to bear additional wearing load during operation. It effectively solves the problem that existing underground monitoring equipment is cumbersome and bulky, causing workers to avoid wearing it due to its inconvenience. It balances wearing comfort and all-day monitoring needs without affecting normal working vision.

[0028] 2. This invention constructs a judgment model based on the logical mutual exclusion between oculomotor nerve response and blood oxygen data, overcoming the technical challenge of detecting concealed carbon monoxide poisoning. Addressing the deficiency of conventional optical blood oxygenation sensors in detecting falsely high readings due to the cherry-red color of carboxyhemoglobin in the early stages of carbon monoxide poisoning, this invention utilizes the physiological characteristic of reduced eye saccade speed caused by central nervous system inhibition resulting from poisoning. By identifying the logical contradiction between normal blood oxygen readings and slowed nerve responses, it provides miners with a biological-level safety barrier that can penetrate deception.

[0029] 3. This invention integrates vision protection, active hearing protection, vital sign monitoring, and environmental perception, achieving an intelligent upgrade of personal protective equipment for mining. By integrating a microphone and speaker to generate anti-phase sound waves, it effectively counteracts the hearing damage caused by low-frequency mechanical noise underground. Simultaneously, by utilizing a collaborative verification mechanism of head posture and eye movement characteristics, it effectively eliminates false alarms of accidental slips while the user is conscious, improving the accuracy of determining falls and unconsciousness in complex underground environments and enhancing emergency response efficiency. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the structure of the intelligent protective glasses for downhole drilling based on eye tracking and multimodal physiological perception coupling of the present invention; Figure 2 This is a flowchart of the downhole safety early warning method of the present invention; Figure 3This is a flowchart of step S1, data acquisition and preprocessing, of the present invention. Figure 4 This is a flowchart of step S2, feature extraction, of the present invention. Figure 5 This is the logic diagram of the state discrimination model for step S3 of the present invention; Figure 6 This is a flowchart of step S4, fall detection and false alarm rejection, of the present invention. Figure 7 This is a flowchart of step S5, active hearing protection, of the present invention.

[0031] The components include: 1. Eye-tracking sensor module; 2. Inertial sensor and temple assembly; 3. Embedded main control unit; 4. Reflective PPG sensor; 5. Body temperature and skin conductance sensor; 6. Microphone; 7. Speaker; and 8. Detachable temple battery module. Detailed Implementation

[0032] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] See attached document Figure 1 The present invention provides an intelligent protective glasses for downhole drilling based on eye tracking and multimodal physiological perception coupling. The intelligent protective glasses include an eye tracking module 1, an inertial sensor and temple assembly 2, an embedded main control unit 3, a reflective PPG sensor 4, a body temperature and skin conductance sensor 5, a microphone 6, a speaker 7, and a detachable temple battery module 8.

[0034] The main structure of these smart protective glasses adopts a highly integrated goggle design. The frame and temple assembly 2 are injection molded from TR90 memory polymer or polycarbonate material. The selection of these materials gives the glasses resistance to acid and alkali corrosion, enabling them to adapt to the high humidity, high dust, and corrosive gas environments underground. Simultaneously, the high toughness of the materials provides explosion-proof and impact-resistant capabilities, while their lightweight nature reduces the burden on workers during extended wear. The temple assembly 2 features a head-hugging ergonomic design, with its ends extending behind the wearer's ears. The detachable temple battery module 8 is located at the very end of the temple assembly 2, physically and electrically connected to it via magnetic contacts or mechanical clips. This layout cleverly utilizes the lever principle: the weight of the detachable temple battery module 8 acts as a counterweight, effectively offsetting the gravitational torque generated by the eye-tracking module 1 located at the nose pad of the frame. This shifts the center of gravity of the entire device to above the ear, preventing the front of the glasses from pressing too heavily on the bridge of the nose or slipping off, thus improving the comfort and stability of workers during extended wear. Meanwhile, the detachable design supports rapid battery replacement, and with the miniature backup capacitor integrated on the mainboard inside the temple, the battery can be replaced without stopping the machine, ensuring uninterrupted monitoring throughout the entire shift underground.

[0035] Each electronic functional module is arranged in the internal cavity of the frame and temple via a flexible circuit board (FPC) or embedded wires, and all sensor signal lines are connected to the embedded main control unit 3.

[0036] The eye-tracking module 1 is located inside the nose pad cavity of the glasses or at the lower edge of the frame. The eye-tracking module 1 integrates a miniature infrared CMOS image sensor and a near-infrared supplemental light. The near-infrared supplemental light emits near-infrared light with wavelengths of 850nm or 940nm, which is invisible to the human eye and will not interfere with the operator's vision or cause unnatural pupil contraction in the low-light environment underground. The eye-tracking module 1 employs an upward-angle optical path design, with the lens optical axis of the CMOS image sensor tilted upwards relative to the horizontal line of sight, avoiding the operator's direct line of sight and directly capturing dynamic images of the pupil and eyelid opening and closing data from below. The CMOS image sensor is configured in global exposure mode to eliminate the rolling shutter effect caused by rapid eye movements, ensuring clear and accurate capture of high-speed eye movement features.

[0037] A multi-point skin-contact physiological sensor array is distributed on the inner side of the temple assembly 2, specifically corresponding to the temples or mastoid region behind the ears of the wearer. This location was chosen because these areas are rich in subcutaneous arteries and have relatively thin skin, while being less affected by facial muscle movements, thus enabling the acquisition of high-quality physiological signals. The sensor array includes a reflective PPG sensor 4 and body temperature and skin conductance sensors 5. The reflective PPG sensor 4 is fitted to the skin and uses the photoplethysmography principle to emit a light beam into the subcutaneous tissue and receive the reflected light intensity signal that changes with the heartbeat. The embedded main control unit 3 calculates the heart rate and blood oxygen saturation (SpO2) values ​​based on this signal. The body temperature and skin conductance sensors 5 include a contact metal temperature probe and two skin conductance electrodes, used to monitor changes in body surface temperature and skin conductivity in real time, helping to determine whether the wearer is in a state of tension or stress.

[0038] An inertial sensor is integrated inside the temple assembly 2. The inertial sensor includes a six-axis or nine-axis inertial measurement unit (IMU), specifically comprising a three-axis accelerometer and a three-axis gyroscope. The inertial sensor's mounting coordinate system is aligned with the anatomical coordinate system of the human head, continuously acquiring three-axis acceleration and angular velocity data of the wearer's head. The smart protective glasses also feature an environmental sensing interface for acquiring data on the concentration of toxic and harmful gases in the environment. In one embodiment, a miniature gas sensor is directly integrated into the frame. In another embodiment, an embedded main control unit 3 establishes a data connection with a portable gas detector carried by the wearer via Bluetooth or ZigBee wireless communication protocol, receiving real-time readings of ambient carbon monoxide or methane concentrations.

[0039] The embedded main control unit 3 is built into the thickened area of ​​the frame or temple assembly 2. The embedded main control unit 3 includes a low-power processing chip and a power management module. The low-power processing chip has edge computing capabilities and is configured to perform local data fusion, feature extraction, and logical reasoning operations. The power management module is connected to a miniature lithium battery to power the entire system.

[0040] The active noise cancellation and alarm module includes a microphone 6 and a speaker 7. The microphone 6 is located on the outer surface of the temple assembly 2, facing the external environment, and is configured to collect ambient noise spectrum data. The speaker 7 is located on the temple assembly 2 near the wearer's ear canal, and employs a bone conduction vibrator or air conduction speaker structure. The ambient noise signal collected by the microphone 6 is transmitted to the embedded main control unit 3, processed, and then played by the speaker 7 with an inverted sound wave to cancel out steady-state low-frequency noise. Simultaneously, the speaker 7 is also configured to play voice alarm information generated by the embedded main control unit 3. When the system determines that a safety risk exists, the speaker 7 issues an audible alert to the wearer, achieving a dual function of hearing protection and safety warning.

[0041] See attached document Figure 2 This invention provides a downhole early warning method based on eye movement and multimodal perception. This method is executed by the embedded main control unit 3 in the aforementioned downhole protective goggles, and may include the following steps: Step S1: Real-time acquisition and preprocessing of multi-source heterogeneous data. The embedded main control unit 3 receives real-time data streams from various sensor modules in parallel through hardware interfaces. Specifically, the embedded main control unit 3 acquires high-frame-rate near-infrared image data containing the eye features of the worker from the eye-tracking sensing module 1, acquires photoplethysmography (PPG) signals and electrodermal response signals from the multi-point skin-contact physiological sensor array distributed on the inner side of the temples, acquires triaxial acceleration and angular velocity data reflecting head movement posture from the inertial sensors inside the temples, and acquires carbon monoxide and methane concentration data from the external environment through the environmental communication interface. The embedded main control unit 3 preprocesses the above raw signals, including removing instantaneous noise in the eye-tracking images, removing motion artifacts in the physiological signals using filtering algorithms, and performing drift correction on the inertial sensor data.

[0042] Step S2: In-depth extraction of eye and physiological features. The embedded main control unit 3 performs feature decomposition on the preprocessed data to obtain quantified physiological and behavioral indicators. For eye image data, the embedded main control unit 3 uses image recognition algorithms to locate the pupil center and eyelid edge, calculates the percentage of time the eyelid is closed for more than 80% of the time (PERCLOS value), pupil diameter, pupillary light reaction speed, and eye saccade speed. For photoplethysmography (PPG) signals, the embedded main control unit 3 calculates blood oxygen saturation values ​​and heart rate variability indicators. For skin conductance signals, the embedded main control unit 3 extracts the trend of skin conductance changes to quantify the stress level of individuals.

[0043] Step S3: Construct a three-dimensional state discrimination model based on eye movement features, blood oxygen data, and environmental data, and perform logical judgment. The embedded main control unit 3 inputs the feature parameters extracted in step S2 into the preset logical discrimination model to classify and judge the current state of the worker. The discrimination model first judges whether the ambient gas concentration exceeds the standard. If the ambient gas concentration is normal and the PERCLOS value is higher than the preset fatigue threshold, it is judged as a simple physiological fatigue state. If the ambient gas concentration is abnormally high, and at the same time, a slow pupillary light reflex or abnormal pupil dilation is detected, it is judged as a hypoxia or overt poisoning state. If the ambient gas sensor detects a trace amount of carbon monoxide and the blood oxygen saturation reading remains within the normal range, but at the same time, eye movement data shows a sluggish pupillary light reflex or nystagmus, the embedded main control unit 3 judges this state as a hidden carbon monoxide poisoning state, that is, it identifies the logical contradiction between the falsely normal physiological blood oxygen reading and the damage to the nervous system.

[0044] Step S4: Fall detection and false alarm rejection based on posture and eye movement coordination. The embedded main control unit 3 continuously monitors the three-axis acceleration data from the inertial sensor and calculates the resultant acceleration value. When the resultant acceleration value exhibits a sequence of characteristics—first a sharp drop approaching zero gravity, then a high-amplitude impact peak, and finally remaining stationary or slightly moving—the embedded main control unit 3 initially determines that a fall event has occurred. Based on this, the embedded main control unit 3 immediately calls upon synchronized eye movement data for secondary confirmation. If, within a preset time window after detecting the fall characteristics, the eye movement data shows violent eye movements or a high blinking frequency, the embedded main control unit 3 determines it to be an accidental slip by a conscious person and does not trigger an emergency call. If the eye movement data shows the eyes are stationary, closed, or the eye movement characteristics disappear, the embedded main control unit 3 determines it to be a fall event involving unconsciousness or serious injury.

[0045] Step S5 involves implementing active hearing protection, tiered early warning, and data reporting. The embedded main control unit 3 executes corresponding response actions based on the judgment results of steps S3 and S4. For simple physiological fatigue, it controls the speaker 7 to issue a voice prompt or controls the vibration motor to wake the user. For poisoning warnings or coma / fall judgments, the embedded main control unit 3 triggers the highest level alarm, controls the speaker 7 to emit a high-frequency alarm, and automatically sends a distress data packet containing the current location coordinates, alarm time, and a summary of physiological and environmental sensor data from the time period preceding the alarm to the ground command center via the wireless communication module. Simultaneously, the embedded main control unit 3 uses a microphone to collect ambient noise and performs spectrum analysis, generating an anti-phase sound wave that is played through the speaker 7 to cancel out steady-state low-frequency noise, thus achieving active hearing protection.

[0046] See attached document Figure 3 This invention provides a downhole early warning method based on eye tracking and multimodal sensing. Step S1 of this method involves the real-time acquisition and preprocessing of multi-source heterogeneous data, as detailed below: In step S1, the embedded main control unit 3 first establishes a unified system clock reference and sends synchronization trigger signals to the eye-tracking perception module 1, the multi-point skin-attached physiological sensor array, the inertial sensor, and the environmental perception interface to ensure that the data sampling frequency and timestamp of each sensor module are strictly aligned. The embedded main control unit 3 receives the raw data streams from the above modules in parallel through I2C, SPI, or MIPI interfaces, forming a multi-source heterogeneous dataset containing images, one-dimensional timing signals, and environmental values.

[0047] For the acquisition and preprocessing of eye-tracking image data, the embedded main control unit 3 controls the near-infrared supplementary light in the eye-tracking sensing module 1 to emit light in a pulsed manner, using a miniature infrared CMOS image sensor to capture near-infrared reflection images of the worker's eyes in global exposure mode. The embedded main control unit 3 performs image enhancement preprocessing on the acquired raw image frames, using a Gaussian filtering algorithm to remove high-frequency thermal noise from the images, and using histogram equalization to enhance the contrast between the pupil and iris regions. The embedded main control unit 3 also uses an edge detection algorithm to identify light spot interference in the images, eliminating invalid image frames that are locally overexposed due to direct external strong light, ensuring that the images input to subsequent algorithms have clear pupil edge features.

[0048] For the acquisition and preprocessing of physiological signals, the embedded main control unit 3 drives the reflective PPG sensor 4 located on the inner side of the endoscope temple to emit a light beam of a specific wavelength and receive the light intensity signal reflected by the skin tissue. Simultaneously, the body temperature and skin conductivity sensors 5 acquire the temperature and conductivity signals of the skin surface. Given the high-frequency vibrations and large limb movements present in the downhole working environment, the embedded main control unit 3 performs motion artifact elimination processing based on adaptive filtering. The embedded main control unit 3 uses synchronously acquired inertial sensor data as a reference signal, employs a least mean square adaptive filtering algorithm to estimate and subtract noise components related to head movement from the original PPG signal, thereby retaining the volume wave change component solely caused by heartbeats. Furthermore, the embedded main control unit 3 performs bandpass filtering on the physiological signals to filter out power frequency interference and respiratory baseline drift.

[0049] For the acquisition and preprocessing of motion posture and environmental data, the embedded main control unit 3 reads the raw values ​​of triaxial acceleration and triaxial angular velocity from the inertial sensor at a preset high sampling rate. The embedded main control unit 3 performs low-pass filtering on the inertial data to filter out steady-state high-frequency vibration noise generated by the mining machinery, retaining the low-frequency posture components reflecting the actual movement of the human head. Simultaneously, the embedded main control unit 3 reads the carbon monoxide and methane concentration values ​​output by the environmental sensor through a digital interface, and uses a moving average filtering algorithm to smooth the gas concentration data, eliminating instantaneous reading fluctuations caused by airflow disturbances and obtaining stable environmental gas concentration values. Finally, the embedded main control unit 3 packages all the preprocessed data in chronological order to construct a time-synchronized multi-dimensional feature input vector, which is then transmitted to subsequent steps for deep feature extraction.

[0050] See attached document Figure 4 This invention provides a downhole early warning method based on eye movement and multimodal perception. Step S2 of this method involves the deep extraction of eye features and physiological features, as detailed below: In step S2, the embedded main control unit 3 performs parallel computation on the preprocessed data stream output from step S1, extracting eye-tracking indicators reflecting the worker's visual attention and physiological indicators reflecting their physical function. For the eye-tracking indicator calculation, the embedded main control unit 3 first calculates the PERCLOS value based on eye image data, which is the percentage of time the eyelids are closed for more than 80% of the time per unit time. Specifically, the embedded main control unit 3 uses a facial key point detection algorithm to locate the coordinates of the upper and lower edges of the eyelids and calculates the eyelid opening amplitude in real time. The system sets a preset time window and continuously monitors the eyelid opening amplitude within this window. When the eyelid opening amplitude is less than 20% of the fully open amplitude, it is determined to be a closed-eye frame. The embedded main control unit 3 accumulates the duration of all closed-eye frames within this time window and divides it by the total duration of the time window to obtain a quantified PERCLOS value, which directly reflects the worker's level of drowsiness.

[0051] Meanwhile, the embedded main control unit 3 performs in-depth analysis of the pupil's dynamic characteristics, calculating the pupil diameter and its light response speed. The embedded main control unit 3 uses an edge fitting algorithm to accurately outline the pupil in the infrared image and converts the image pixel coordinates into actual physical dimensions based on camera calibration parameters, thus outputting the pupil diameter in millimeters in real time. The embedded main control unit 3 continuously records the pupil diameter change curve over time. When a step change in ambient light intensity is detected or a stimulus signal from an active supplemental light is received, it calculates the reaction latency required for the pupil diameter to contract from its initial state to its minimum state, as well as the average speed during the contraction process, using this as an indicator of the pupil's light response speed. Furthermore, the embedded main control unit 3 also calculates the eye's saccade speed by tracking the displacement of the pupil center coordinates in consecutive image frames. The system uses the pupil center displacement vector divided by the inter-frame time interval to obtain the instantaneous angular velocity and calculates the average saccade speed and saccade amplitude per unit time to evaluate the excitability and control of the visual nerves.

[0052] For the calculation of physiological indicators, the embedded main control unit 3 performs waveform analysis on the photoplethysmography (PPG) signal to extract the blood oxygen saturation value. The embedded main control unit 3 separates the DC and AC components of the red and infrared signals acquired by the reflective PPG sensor 4. The DC component represents the light absorption intensity of non-pulsatile tissues such as skin, bones, and venous blood, while the AC component represents the light intensity change caused by arterial blood pulsation. According to Beer-Lambert's law, the embedded main control unit 3 calculates the AC / DC ratio of the red light signal and the AC / DC ratio of the infrared light signal, and divides these two ratios to obtain the dual-wavelength absorbance ratio. The embedded main control unit 3 then substitutes this absorbance ratio into a preset calibration curve equation to calculate the current blood oxygen saturation percentage reading. This reading is used to subsequently determine whether hypoxia or abnormal blood oxygenation is present.

[0053] Furthermore, the embedded main control unit 3 also extracts heart rate variability (HRV) indicators based on photoplethysmography (PPG) signals. The embedded main control unit 3 uses a peak detection algorithm to identify the systolic peak point of each cardiac cycle in the PPG signal and calculates the time interval between two adjacent peak points, i.e., the successive heartbeat interval. The embedded main control unit 3 collects a series of successive heartbeat interval data within a preset sampling time and performs time-domain analysis on them. Specifically, this includes calculating the standard deviation of all successive heartbeat intervals and the root mean square of the difference between adjacent heartbeat intervals, thereby obtaining the HRV indicator. This indicator reflects the autonomic nervous system's ability to regulate cardiac activity. The embedded main control unit 3 uses it as a key parameter for assessing the mental load and stress state of workers, and together with eye-tracking indicators, it constitutes the multidimensional input vector of the subsequent state discrimination model.

[0054] See attached document Figure 5 This invention provides a downhole early warning method based on eye movement and multimodal sensing. Step S3 of this method involves constructing a three-dimensional state discrimination model based on eye movement parameters, blood oxygen parameters, and environmental parameters. The specific details are as follows: In step S3, the embedded main control unit 3 receives the environmental data collected in step S1 and the eye movement features and physiological features extracted in step S2, and maps these data to a preset three-dimensional state space for logical discrimination. The embedded main control unit 3 first executes the judgment logic for a simple physiological fatigue state, i.e., scenario A. The embedded main control unit 3 reads the gas concentration data and carbon monoxide concentration data uploaded by the environmental sensors and compares them with preset safety thresholds. When the ambient gas concentration values ​​are all below the safety threshold, and the blood oxygen saturation value from the reflective PPG sensor 4 is within the preset normal blood oxygen range, while the PERCLOS value in the eye movement features is higher than the preset fatigue threshold, the embedded main control unit 3 determines that the worker is in a simple physiological fatigue state. In response to this judgment result, the embedded main control unit 3 generates a level one alarm command, drives the linear motor to perform intermittent vibration, and plays a voice message prompting rest through the bone conduction speaker 7.

[0055] In step S3, the embedded main control unit 3 then executes the judgment logic for hypoxia or overt poisoning, i.e., scenario B. The embedded main control unit 3 continuously monitors environmental and blood oxygen data. When the concentration of toxic and harmful gases in the environment exceeds a safe threshold, or the blood oxygen saturation value is lower than a preset hypoxia threshold, the system enters a high-risk monitoring mode. In this mode, the embedded main control unit 3 simultaneously detects pupillary light reaction speed and pupil diameter data. If the pupillary light reaction speed is lower than a preset slow reaction threshold, or the pupil diameter is greater than a preset dilation threshold, the embedded main control unit 3 determines that the worker has early signs of poisoning or relatively severe hypoxia symptoms. In response to this judgment result, the embedded main control unit 3 generates a secondary alarm command, drives the speaker 7 to emit a high-frequency emergency alarm sound, and uploads the current abnormal data packet to the underground command center via the wireless communication module.

[0056] In step S3, the embedded main control unit 3 further executes the judgment logic for the concealed carbon monoxide poisoning state, i.e., scenario C. This judgment logic is based on the difference between the pathological optical characteristics and neurotoxic characteristics of carbon monoxide poisoning. Because carboxyhemoglobin's light absorption characteristics in the red and infrared light bands are highly similar to those of oxyhemoglobin, conventional photoplethysmography (PPG) sensors may still output normal blood oxygen saturation readings when workers suffer from carbon monoxide poisoning, presenting a false normality. However, carbon monoxide poisoning inhibits the central nervous system, leading to a significant decrease in eye saccade speed. The embedded main control unit 3 utilizes the above-mentioned characteristic differences to construct a logical paradox recognition algorithm.

[0057] The embedded main control unit 3 specifically uses the following logic formula to determine covert poisoning: when the condition is met... and and At that time, it was determined to be covert carbon monoxide poisoning. Among them, This represents the current measured blood oxygen saturation value. This represents the preset lower limit of normal blood oxygenation. This represents the current average eye saccade speed. This represents the preset threshold for neural response slowing. This represents the carbon monoxide concentration value collected by the current environmental sensors. This represents the minimum detection limit or baseline background concentration value for a carbon monoxide sensor.

[0058] When the aforementioned logical formula holds true, the embedded main control unit 3 detects a logical contradiction between the normal physiological blood oxygen reading and the suppressed neurological response, thus determining that the worker has a high probability of suffering from concealed carbon monoxide poisoning. In response to this determination, the embedded main control unit 3 generates a three-level alarm command, with the highest priority. The embedded main control unit 3 immediately plays a mandatory voice command for wearing a self-rescue device via the bone conduction speaker 7 and automatically sends a distress signal containing the current location coordinates and vital signs data to the ground command center to trigger the emergency rescue process.

[0059] See attached document Figure 6 This invention provides a downhole early warning method based on eye movement and multimodal perception. Step S4 of this method involves a fall detection algorithm based on posture and eye movement coordination, the details of which are as follows: In step S4, the embedded main control unit 3 uses an inertial measurement unit integrated inside the temple to collect real-time head motion posture data of the wearer. The inertial measurement unit continuously outputs triaxial acceleration data and triaxial angular velocity data at a preset high sampling rate. The embedded main control unit 3 first performs low-pass filtering on the raw acceleration data to eliminate high-frequency noise interference, and then calculates the resultant acceleration value SVM. The embedded main control unit 3 calculates the resultant acceleration using the following formula. : ; in, These represent the real-time component acceleration values ​​of the wearer along the X, Y, and Z axes in a coordinate system with the head as the origin. This resultant acceleration value (SVM) reflects the superposition effect of the total external force and gravitational acceleration experienced by the wearer, and is a fundamental indicator for judging the intensity of human movement.

[0060] In step S4, the embedded main control unit 3 performs preliminary identification of fall behavior based on the temporal variation characteristics of the sum of accelerations (SVM). The embedded main control unit 3 slides along the time axis to monitor the SVM value and identifies feature segments that conform to specific temporal logics of weightlessness, impact, and stillness. Specifically, when the embedded main control unit 3 detects that the SVM value drops sharply in a short period of time and falls below a preset weightlessness threshold (close to 0g), it determines that it is in the free fall or weightlessness stage; then, if it detects that the SVM value has a momentary spike within a preset time window after the weightlessness stage and exceeds a preset impact threshold, it determines that it is in the human body impact stage; subsequently, the embedded main control unit 3 continues to monitor the SVM value after the impact. If the SVM value remains within a narrow fluctuation range of gravitational acceleration of 1g for a period of time and the angular velocity data approaches zero, it is determined that it is in a still state after the impact, i.e., unable to stand up on its own.

[0061] In step S4, after completing the initial recognition of the fall behavior, the embedded main control unit 3 immediately calls the eye movement feature data extracted in real time in step S2 to perform a false alarm elimination and consciousness confirmation mechanism. This mechanism aims to distinguish between accidental slips while conscious and critical situations where a fall leads to unconsciousness. The embedded main control unit 3 reads the eye saccade velocity, blink frequency, and PERCLOS value during the static phase of the fall. If the eye movement data shows that the wearer's eye saccade velocity is greater than a preset active threshold, or the blink frequency is within the normal or high-frequency range, the embedded main control unit 3 determines that although the wearer has fallen, they are conscious. The system identifies this as an accidental slip and only inquires about the wearer's condition locally via voice, without triggering a remote alarm.

[0062] Conversely, if the embedded main control unit 3 detects that the wearer meets the characteristics of a fall, and the eye movement data at the same time shows that the eye saccade speed is close to zero, or the PERCLOS value indicates that the eyelids have been closed for a long time, or the pupillary light reflex is detected to be absent, the embedded main control unit 3 determines that the wearer is in a critical state of fall and unconsciousness / serious injury. In response to this determination, the embedded main control unit 3 immediately generates a distress data packet containing the highest priority. The data packet includes: the time of the fall calculated by the inertial sensor, the location coordinates at the time of the fall, a snapshot of physiological parameters (heart rate, blood oxygen) within a preset time period before the fall, and a snapshot of the current eye state. The embedded main control unit 3 automatically sends this data packet to the downhole dispatch center through the wireless communication module, and at the same time drives the speaker 7 on the glasses to emit a continuous audio-visual distress signal to guide nearby rescuers.

[0063] See attached document Figure 7 This invention provides a downhole early warning method based on eye movement and multimodal perception. Step S5 of this method involves active hearing protection and adaptive noise reduction, as detailed below: In step S5, the embedded main control unit 3 activates the microphone 6 located on the outside of the temple to collect ambient sound signals in real time. The microphone 6 converts the collected analog sound wave signals into a digital audio stream and transmits it to the embedded main control unit 3. The embedded main control unit 3 performs a Fast Fourier Transform on the received digital audio stream, converting the sound signal from the time domain to the frequency domain, thereby obtaining the spectral distribution data of the current ambient sound. The embedded main control unit 3 continuously monitors the energy distribution in the spectral data and divides the spectrum into low-frequency and mid-to-high-frequency bands to distinguish the characteristics of different types of sound sources.

[0064] In step S5, the embedded main control unit 3 executes selective noise reduction logic based on the spectrum analysis results. The embedded main control unit 3 identifies spectral components in the spectrum whose frequencies are below a preset low-frequency threshold and whose amplitudes remain stable over multiple consecutive sampling periods, classifying them as steady-state low-frequency noise generated by downhole machinery. For this steady-state low-frequency noise component, the embedded main control unit 3 uses an adaptive filtering algorithm to calculate an anti-phase acoustic signal with the same frequency and amplitude but opposite phase. This anti-phase acoustic signal aims to cancel out the continuous mechanical roaring noise from the external environment through the principle of acoustic wave interference cancellation.

[0065] In step S5, the embedded main control unit 3 simultaneously identifies spectral components in the spectrum whose frequencies fall within the preset human voice communication frequency band or high-frequency alarm frequency band. When non-steady-state signals with abrupt changes in energy amplitude are detected in these frequency bands, the embedded main control unit 3 determines them to be the voice communication of workers or the alarm sound emitted by equipment. For such signals, the embedded main control unit 3 does not generate an anti-phase cancellation signal, but instead performs pass-through processing or gain enhancement processing. The embedded main control unit 3 digitally synthesizes the generated anti-phase sound wave signal for low-frequency noise with the pass-through processed human voice and alarm signals, and plays it through the speaker 7 located at the ear position on the temple of the glasses, thereby reducing environmental background noise interference while ensuring that the wearer can clearly hear the shouts of colleagues and safety alarm sounds.

Claims

1. Eye-tracking and multimodal sensing downhole protective glasses, including a frame and temple assembly (2), characterized in that, The frame and temple assembly (2) is equipped with an embedded main control unit (3). The input end of the embedded main control unit (3) is connected to an eye-tracking sensor module (1), an inertial sensor, a reflective PPG sensor (4), a body temperature and skin conductance sensor (5), and a microphone (6). The output end of the embedded main control unit (3) is connected to a speaker (7) and a wireless communication module. The eye-tracking sensing module (1) is installed in the nose pad area or lower edge of the frame and temple assembly (2) and is configured to collect image data of the eye using an upward-angle optical path. The inertial sensor is integrated inside the frame and temple assembly (2) and is configured to collect head movement posture data; The reflective PPG sensor (4) and the body temperature and skin conductance sensor (5) are disposed on the inner sidewall of the frame and temple assembly (2) and are positioned corresponding to the wearer’s temples or the area behind the ears, and are configured to collect physiological signals. The microphone (6) and the speaker (7) are respectively located on the outside of the frame and temple assembly (2) and at the corresponding ear canal position; The embedded main control unit (3) is configured to perform logical judgments on physiological fatigue, environmental poisoning and fall accidents based on the collected data and control the speaker (7) to sound an alarm.

2. The downhole protective glasses for eye movement and multimodal perception according to claim 1, characterized in that, The eye-tracking sensing module (1) includes a miniature infrared CMOS image sensor and a near-infrared fill light; the near-infrared fill light is configured to emit near-infrared light with a wavelength range of 750 nm to 1000 nm; the miniature infrared CMOS image sensor is configured to operate in global exposure mode, and the optical axis of the miniature infrared CMOS image sensor is tilted upward relative to the horizontal line of sight to avoid the wearer's direct line of sight area.

3. The downhole protective glasses for eye movement and multimodal perception according to claim 1, characterized in that, The embedded main control unit (3) is also connected to an environmental sensing interface, which is configured to acquire data on carbon monoxide concentration and methane concentration in the environment. The environmental sensing interface is connected to a miniature gas sensor integrated in the frame and temple assembly (2), or to a portable gas detector carried by the wearer.

4. The downhole protective glasses for eye movement and multimodal perception according to claim 1, characterized in that, The end of the frame and temple assembly (2) is provided with a detachable temple battery module (8). The detachable temple battery module (8) is connected by magnetic contacts or a snap-fit ​​structure and is positioned behind the wearer's ear to form a gravity balance weight on the eye movement sensing module (1) at the front. The embedded main control unit (3) is located on a circuit board with an integrated backup power supply, configured to maintain system operation to support hot-swapping when the removable temple battery module (8) is disconnected.

5. A downhole early warning method based on eye tracking and multimodal perception, characterized in that, The application of downhole protective eyewear for eye movement and multimodal sensing as described in any one of claims 1 to 4 includes the following steps: S1. Real-time acquisition and preprocessing of multi-source heterogeneous data: The embedded main control unit (3) synchronously acquires eye image data from the eye-tracking sensing module (1), photoplethysmography pulse wave signal from the reflective PPG sensor (4), skin conductance response signal from the body temperature and skin conductance sensor (5), head motion posture data from the inertial sensor, and environmental gas concentration data, and performs filtering and noise reduction preprocessing on the acquired data. S2. Deep extraction of eye features and physiological features: The embedded main control unit (3) performs feature calculation on the preprocessed data and extracts PERCLOS value, pupil diameter, pupil light reaction speed, eye saccade speed, blood oxygen saturation value, heart rate variability index and skin conductance change trend. S3. Construct a three-dimensional state discrimination model of eye movement features, blood oxygen data and environmental data and make logical judgments: The embedded main control unit (3) inputs the feature parameters extracted in step S2 into the preset logical discrimination model to determine whether the worker is in a state of simple physiological fatigue, hypoxia or overt poisoning, or covert carbon monoxide poisoning. S4. Fall detection and false alarm elimination based on posture and eye movement coordination: The embedded main control unit (3) calculates the combined acceleration value based on the head movement posture data and identifies the fall timing characteristics. After determining that a fall event has occurred, it uses synchronized eye movement feature data to determine the wearer's consciousness state, thereby distinguishing between accidental slips and unconscious falls. S5. Perform active hearing protection, graded early warning and data reporting: The embedded main control unit (3) controls the speaker (7) to issue alarm prompts of the corresponding level according to the judgment results of the S3 and S4 steps, reports distress data packets through the wireless communication module, and uses the data collected by the pickup microphone (6) to generate anti-phase sound waves for active noise reduction.

6. The downhole early warning method based on eye movement and multimodal perception according to claim 5, characterized in that, In step S3, the logic for determining the state of concealed carbon monoxide poisoning is as follows: When the blood oxygen saturation value is greater than or equal to the preset lower limit threshold of normal blood oxygen, and the saccade speed of the eyeball is less than the preset threshold of slowed neural response, and the carbon monoxide concentration value in the environmental gas concentration data is greater than the minimum detection limit of the sensor, the embedded main control unit (3) identifies the logical contradiction between the normal physiological blood oxygen reading and the suppressed neural response state, and determines it to be a state of concealed carbon monoxide poisoning.

7. The downhole early warning method based on eye movement and multimodal perception according to claim 5, characterized in that, In step S4, the false alarm rejection and consciousness state confirmation mechanism includes: When a timing change in the combined acceleration that matches the characteristics of a fall is detected, the embedded main control unit (3) reads the eye movement characteristic data of the static phase of the fall. If eye movement data shows that the eye saccade speed is greater than the preset active threshold or the blinking frequency is within the normal range, it is determined to be an accidental slip with full consciousness and will not trigger an emergency call for help. If eye movement data shows that the eye saccade speed is close to zero, the eyelids are closed for a long time, or the pupillary light reflex is absent, it is determined to be a coma or a serious injury from a fall, triggering an emergency call for help.

8. The downhole early warning method based on eye movement and multimodal perception according to claim 5, characterized in that, In step S1, the preprocessing of the photoplethysmography (PPG) signal includes: The embedded main control unit (3) uses the synchronously acquired head motion posture data as a reference signal, and uses the least mean square adaptive filtering algorithm to estimate and subtract the noise component related to head motion from the original photoplethysmography pulse wave signal.

9. The downhole early warning method based on eye movement and multimodal perception according to claim 5, characterized in that, In step S5, the specific process of active hearing protection is as follows: The embedded main control unit (3) performs spectrum analysis on the sound signal collected by the microphone (6); Spectral components with frequencies below a preset low-frequency threshold and stable amplitudes are identified as steady-state low-frequency noise. Calculate the antiphase acoustic signal with the same frequency and amplitude as the steady-state low-frequency noise but opposite phase; The anti-phase sound wave signal is digitally synthesized with the human voice and alarm signal that have undergone pass-through processing and then played through the speaker (7).

10. The downhole early warning method based on eye movement and multimodal perception according to claim 5, characterized in that, In step S2, the PERCLOS value is calculated as follows: The facial landmark detection algorithm is used to locate the coordinates of the upper and lower edges of the eyelids and calculate the eyelid opening amplitude; Within a preset time window, the cumulative duration of closed frames where the eyelid opening is less than 20% of the full opening is counted. The PERCLOS value is obtained by dividing the cumulative duration by the total duration of the preset time window.

Images