Driver state monitoring method and device based on AR glasses and electronic equipment

Abstract

The invention discloses a driver state monitoring method and device based on AR glasses and electronic equipment, and relates to the field of auxiliary driving, and the method comprises the steps: obtaining the physiological reference data of a driver within a preset time threshold value, and synchronizing the initial state data of a vehicle; the physiological reference data comprises a PERCLOS value, a skin conductance baseline and a basal body temperature, and the vehicle initial state data comprises an air conditioner temperature, a fragrance concentration and an ADAS working state; physiological signals, motion state signals and environment data of a driver are collected through a sensor assembly integrated on the AR glasses; performing noise reduction and standardization processing on the physiological signal, the motion state signal and the environmental data of the driver; inputting the processed data into a preset model for analysis, and obtaining a driver state quantification result; according to the driver state quantification result, a third-level safety response instruction and a comfort level optimization instruction are generated; and outputting the third-level safety response instruction through AR glasses, and outputting a comfort level optimization instruction through a vehicle control interface.

Description

Technical Field

[0001] This application relates to the field of driver assistance systems, and more particularly to a driver state monitoring method based on AR glasses, a driver state monitoring device based on AR glasses, an electronic device, and a storage medium. Background Technology

[0002] The continuous evolution of intelligent vehicle technology has propelled intelligent cockpit systems to become a core platform for enhancing driving safety and optimizing the driving experience. Against this backdrop, Driver Monitoring Systems (DMS), as a key component of intelligent cockpits, significantly enhance active safety capabilities through real-time perception, analysis, and early warning feedback of driver behavior and physiological state. Currently, mainstream DMS solutions generally use infrared cameras as core sensors to collect key indicators such as eye movement trajectory, head orientation, blink frequency, and gaze direction. They also rely on artificial intelligence algorithms to identify potential risk states such as driver inattention, fatigue signs, or cognitive decline. However, current mainstream DMS solutions still have many limitations. They primarily rely on monocular / dual-lens cameras, which can only capture planar facial movements (such as closing eyes or turning the head), making it difficult to accurately pinpoint the focus of the gaze or detect early signs of fatigue. For example, Tesla owners commonly complain that the system frequently misinterprets normal operation of the control screen (such as adjusting navigation) as "distraction," and many users with smaller eyes are often misinterpreted as having their eyes closed, often forcing them to disable this function. Furthermore, the facial data collected by these cameras often causes user anxiety due to privacy concerns. More commonly, when users wear sunglasses, masks, or obstruct the camera, the DMS system struggles to make accurate judgments effectively. It's worth noting that while Advanced Driver Assistance Systems (ADAS) have significantly improved driving convenience and active safety, their widespread application has also brought new safety hazards—over-reliance on automated systems by drivers. NASA laboratories in the United States have conducted in-depth research on human-machine cooperative driving scenarios, comparing and analyzing the cognitive load and reaction capabilities of drivers in semi-autonomous driving modes and traditional manual driving modes. The research results show that during prolonged monitoring of autonomous vehicle operation, drivers are more prone to cognitive fatigue and inattention, and their emergency response speed to sudden traffic events is significantly slower, further highlighting the necessity of strengthening driver status monitoring and intervention mechanisms. Summary of the Invention

[0003] In view of this, the purpose of the present invention is to provide a driver state monitoring method based on AR glasses, a driver state monitoring device based on AR glasses, an electronic device and a storage medium, which aims to directly and without delay overlay warning information onto the driver's key field of vision, which is more direct and effective than traditional dashboard alarms or sounds. By combining visual perception and biosensing, it can monitor the driver's physiological condition more comprehensively, provide timely and accurate warnings and interventions, and reduce the occurrence of accidents.

[0004] The present invention provides the following solution: According to one aspect of the present invention, a driver status monitoring method based on AR glasses is provided, comprising the following steps:

[0005] Acquire the driver's physiological baseline data within a preset time threshold and synchronize the vehicle's initial state data;

[0006] Physiological baseline data include: PERCLOS value, skin conductance baseline, and basal body temperature; vehicle initial state data include: air conditioning temperature, fragrance concentration, and ADAS working status.

[0007] By integrating sensor components into AR glasses, the driver's physiological signals, motion state signals, and environmental data are collected;

[0008] Noise reduction and standardization processing are performed on driver physiological signals, motion state signals, and environmental data;

[0009] The processed data is input into a preset model for analysis to obtain quantitative results of the driver's state.

[0010] Based on the quantitative results of the driver's condition, three levels of safety response instructions and comfort optimization instructions are generated;

[0011] The AR glasses output the Level 3 safety response command, and the vehicle control interface outputs the comfort optimization command.

[0012] Furthermore, including:

[0013] The sensor components include: an eye-tracking sensor, a head IMU sensor, a PPG biosensor, a skin conductivity sensor, a temperature sensor, and an ambient light detector;

[0014] Physiological signals include: heart rate, blood oxygen, skin conductivity, and body temperature;

[0015] Motion state signals include: head yaw angle, gaze trajectory, and blink frequency;

[0016] Environmental data includes: ambient light intensity.

[0017] Furthermore, including:

[0018] AR glasses acquire real-time vehicle operation data via Wi-Fi / Bluetooth protocols, including vehicle speed and steering angle.

[0019] Furthermore, including:

[0020] The preset models include: fatigue risk model, distraction risk model, and comfort assessment model;

[0021] The fatigue risk model obtains a fatigue index by pre-setting weights on physiological baseline data;

[0022] The distraction risk model generates a distraction score by multiplying the weighted maximum value of the duration of gaze deviation and the head yaw angle by a stress coefficient.

[0023] The comfort assessment model quantifies comfort values ​​by using skin temperature fluctuations and EDA entropy.

[0024] Furthermore, including:

[0025] Stress coefficients include:

[0026] When the baseline skin conductance is within a preset threshold range, the stress coefficient = 1.1;

[0027] When the baseline of skin electrical conductance exceeds the preset threshold range, the stress coefficient is 1.2.

[0028] Furthermore, including:

[0029] The specific instructions for a Level 3 security response are as follows:

[0030] The low-risk response command projects a yellow circular warning icon only at the edge of the driver's field of vision via AR glasses;

[0031] The medium-risk response command is displayed as an orange animated icon projected onto the AR glasses, while a gentle prompt sound plays.

[0032] The high-risk response command projects a strong intervention red shaking icon onto the AR glasses, plays a high-frequency warning sound, and initiates continuous vibration for a preset time threshold. If no touch response from the driver is detected within the preset time threshold, a command is sent to the vehicle to switch to emergency assistance mode and control the vehicle to decelerate and stop at the nearest parking location.

[0033] Furthermore, including:

[0034] The specific comfort optimization instructions are as follows:

[0035] When fatigue-related discomfort is detected: control the release of strong menthol fragrance from the in-car fragrance system, and simultaneously switch the in-car audio system to natural white noise;

[0036] When the discomfort is determined to be stress-related: control the concentration of lavender essential oil fragrance released by the car fragrance system, and at the same time switch the car audio to piano music with a BPM of <60;

[0037] When environmental discomfort is detected: when the basal body temperature is higher than the preset threshold, the vehicle's air conditioning is controlled to cool down; when the vehicle detects that the CO2 concentration is higher than the preset threshold, the air conditioning external circulation mode is activated.

[0038] According to a second aspect of the present invention, a driver status monitoring device based on AR glasses is provided, comprising:

[0039] The system includes a data acquisition and synchronization unit, a real-time data acquisition unit, a data preprocessing unit, a quantization result output unit, an instruction generation unit, and an instruction execution unit.

[0040] The data acquisition and synchronization unit is used to acquire the driver's physiological baseline data within a preset time threshold and synchronize the vehicle's initial state data.

[0041] Physiological baseline data include: PERCLOS value, skin conductance baseline, and basal body temperature; vehicle initial state data include: air conditioning temperature, fragrance concentration, and ADAS working status.

[0042] The real-time data acquisition unit is used to collect driver physiological data, motion status data, and environmental data through sensor components integrated on the AR glasses;

[0043] The data preprocessing unit is used to perform noise reduction and standardization processing on the driver's physiological signals, motion state signals and environmental data;

[0044] The quantification result output unit is used to input the processed data into a preset model for analysis and obtain the driver's state quantification result;

[0045] The instruction generation unit is used to generate three-level safety response instructions and comfort optimization instructions based on the driver's state quantification results.

[0046] The instruction execution unit is used to output level 3 safety response instructions through AR glasses and comfort optimization instructions through the vehicle control interface.

[0047] According to three aspects of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0048] The memory stores a computer program that, when executed by a processor, causes the processor to perform steps of a driver state monitoring method based on AR glasses.

[0049] According to four aspects of the present invention, a computer-readable storage medium is provided that stores a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a driver state monitoring method based on AR glasses.

[0050] Compared with the prior art, the present invention has the following advantages:

[0051] This application utilizes the structural design and sensor layout scheme of AR glasses. An eye-tracking sensor is located inside the lens, functioning to locate pupils, monitor blinks, and track gaze. A head IMU sensor is located in the center of the frame, confirming head posture. A camera on the left side of the frame enables environmental perception and gesture recognition. A PPG sensor is embedded in the nose pad for all-day heart rate / blood oxygen tracking. A skin conductivity sensor is also located on the nose pad; when the sympathetic nervous system is excited, sweat gland secretion increases, and skin conductivity rises, allowing for better detection of user emotional changes. In situations prone to anxiety and anger, such as traffic jams or being cut off, the sensor can be linked with the vehicle's infotainment system to alleviate user tension through music, air conditioning, and fragrance, preventing impulsive and irrational decisions. A temperature sensor can better monitor the driver's body temperature, identifying physiological abnormalities such as heatstroke and fever that affect driving safety. It can also be integrated with air conditioning and window controllers to automatically optimize the in-car environment and improve driver comfort.

[0052] This application uses eyeglasses as the core hub, integrating multi-source sensors (including a visual camera, a nose pad biosensor array, an IMU motion module, and an ambient light detector) into the glasses to collect the driver's physiological signals (EDA / body temperature / heart rate) and ambient light data in real time. This data is combined with vehicle speed, steering, and ADAS status information transmitted from the vehicle for multimodal data fusion analysis. The decision engine dynamically generates three levels of safety warning instructions based on the fusion results—projecting AR warning images through an optical waveguide display system, transmitting information to the driver in conjunction with bone conduction audio and tactile feedback from the temples, and simultaneously outputting environmental adjustment instructions to the vehicle control system via Wi-Fi / Bluetooth protocols. The PID algorithm precisely controls the air conditioning temperature and fragrance concentration, forming a closed-loop control circuit of "physiological signal acquisition → AR warning feedback → vehicle environmental regulation," ultimately achieving a synergistic improvement in driving safety and passenger comfort.

[0053] This application achieves synchronous output through four channels—optical waveguide display, audio system, vehicle environmental control, and tactile feedback—through a full-scene feedback execution layer, forming a closed-loop control from biosignal perception to environmental adjustment, which significantly improves warning efficiency and reduces driver cognitive load. Attached Figure Description

[0054] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is a flowchart of a driver status monitoring method based on AR glasses provided by one or more embodiments of the present invention.

[0056] Figure 2 This is a structural diagram of a driver status monitoring device based on AR glasses provided in one or more embodiments of the present invention.

[0057] Figure 3 This is a schematic diagram of the structural design and sensor layout of AR glasses according to a specific embodiment of the present invention.

[0058] Figure 4 This is a schematic diagram of a driver status monitoring method based on AR glasses, according to a specific embodiment of the present invention.

[0059] Figure 5 This is a flowchart illustrating the control logic design of a driver status monitoring method based on AR glasses, according to a specific embodiment of the present invention.

[0060] Figure 6 This is a block diagram of an electronic device for a driver status monitoring method based on AR glasses, provided by one or more embodiments of the present invention. Detailed Implementation

[0061] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0062] Figure 1 This is a flowchart of a driver status monitoring method based on AR glasses provided by one or more embodiments of the present invention.

[0063] like Figure 1 As shown, it includes the following steps:

[0064] Step S1: Obtain the driver's physiological baseline data within a preset time threshold and synchronize the vehicle's initial state data.

[0065] Physiological baseline data include: PERCLOS value, skin conductance baseline, and basal body temperature; vehicle initial state data include: air conditioning temperature, fragrance concentration, and ADAS working status.

[0066] Specifically, personalized baseline data is collected during a 5-minute rest period (when the driver has not started driving and is in a stable state). This allows the system to determine the driver's normal physiological thresholds and the initial environmental state of the vehicle. Subsequent monitoring only requires comparing this baseline to accurately determine whether fatigue, distraction, or abnormal comfort has occurred.

[0067] Synchronizing the air conditioning temperature, fragrance concentration, and ADAS operating status (such as whether ACC is on) serves two purposes: first, it provides an initial basis for subsequent "comfort optimization" (e.g., if the air conditioning needs to be adjusted later, it can be precisely adjusted based on the initial temperature); second, it lets the system know the initial ADAS mode, avoiding interference with the driver's judgment due to changes in ADAS status.

[0068] Step S2: Collect the driver's physiological signals, motion state signals and environmental data through sensor components integrated on the AR glasses;

[0069] Specifically, the driver's physiological characteristics and environmental data are collected in real time through a variety of sensors integrated into the glasses (including eye-tracking sensors, head IMU, biosensor group (EDA / temperature / PPG), and environmental perception module).

[0070] Step S3: Noise reduction and standardization processing are performed on the driver's physiological signals, motion state signals and environmental data;

[0071] Specifically, the acquired raw signals are processed by a preprocessing layer, and the processed data is input into a multi-dimensional fusion analysis layer. The fatigue risk model calculates the fatigue index by integrating the PERCLOS visual index (weight 0.6), EDA baseline level (weight 0.3), and skin temperature change (weight 0.1).

[0072] Step S4: Input the processed data into a preset model for analysis to obtain the quantitative results of the driver's state;

[0073] Specifically, the distraction risk model generates a distraction score by multiplying the weighted maximum value of the duration of gaze deviation and the head yaw angle by a stress coefficient, while the comfort assessment module quantifies the driver's comfort state through skin temperature fluctuations and EDA entropy values.

[0074] Step S5: Based on the driver status quantification results, generate three-level safety response instructions and comfort optimization instructions;

[0075] Specifically, the fusion analysis results drive the dynamic decision engine to generate a three-level response strategy: when the safety risk level is Level 1, a yellow AR prompt icon is generated at the edge of the field of vision to remind the user of their status; at Level 2, a clear orange icon appears within the field of vision, and the glasses play a voice prompt, indicating that the current situation is somewhat dangerous; at Level 3, a strong intervention red shaking warning is activated, the glasses play a voice prompt and vibrate to stimulate the user to wake up from a tired or drowsy state. If the user does not take any response measures within 5 seconds in this state, the glasses send a signal to the vehicle system to automatically switch the driving mode and park at the nearest parking location.

[0076] Step S6: Output the Level 3 safety response command through the AR glasses and output the comfort optimization command through the vehicle control interface.

[0077] Specifically, in addition to detecting fatigue, the glasses can also independently optimize the environment based on comfort levels. When fatigued, they release a strong menthol fragrance and play natural white noise; when stressed, they release lavender essential oil and switch to piano music with a BPM of <60; when skin temperature is detected to be >37℃, they control the air conditioning to cool down by 2℃ and activate the external circulation mode when CO2 is >1000ppm. Finally, the full-scene feedback execution layer achieves synchronous output through four channels: optical waveguide display, audio system, vehicle environmental control, and tactile feedback, forming a closed-loop control from biosignal perception to environmental adjustment, significantly improving warning efficiency and reducing driver cognitive load.

[0078] Furthermore, including:

[0079] The sensor components include: an eye-tracking sensor, a head IMU sensor, a PPG biosensor, a skin conductivity sensor, a temperature sensor, and an ambient light detector;

[0080] Physiological signals include: heart rate, blood oxygen, skin conductivity, and body temperature;

[0081] Motion state signals include: head yaw angle, gaze trajectory, and blink frequency;

[0082] Environmental data includes: ambient light intensity.

[0083] Furthermore, including:

[0084] AR glasses acquire real-time vehicle operation data via Wi-Fi / Bluetooth protocols, including vehicle speed and steering angle.

[0085] Furthermore, including:

[0086] The preset models include: fatigue risk model, distraction risk model, and comfort assessment model;

[0087] The fatigue risk model obtains a fatigue index by pre-setting weights on physiological baseline data;

[0088] The distraction risk model generates a distraction score by multiplying the weighted maximum value of the duration of gaze deviation and the head yaw angle by a stress coefficient.

[0089] The comfort assessment model quantifies comfort values ​​by using skin temperature fluctuations and EDA entropy.

[0090] Furthermore, including:

[0091] Stress coefficients include:

[0092] When the baseline skin conductance is within a preset threshold range, the stress coefficient = 1.1;

[0093] When the baseline of skin electrical conductance exceeds the preset threshold range, the stress coefficient is 1.2.

[0094] Furthermore, including:

[0095] The specific instructions for a Level 3 security response are as follows:

[0096] The low-risk response command projects a yellow circular warning icon only at the edge of the driver's field of vision via AR glasses;

[0097] The medium-risk response command is displayed as an orange animated icon projected onto the AR glasses, while a gentle prompt sound plays.

[0098] The high-risk response command projects a strong intervention red shaking icon onto the AR glasses, plays a high-frequency warning sound, and initiates continuous vibration for a preset time threshold. If no touch response from the driver is detected within the preset time threshold, a command is sent to the vehicle to switch to emergency assistance mode and control the vehicle to decelerate and stop at the nearest parking location.

[0099] Furthermore, including:

[0100] The specific comfort optimization instructions are as follows:

[0101] When fatigue-related discomfort is detected: control the release of strong menthol fragrance from the in-car fragrance system, and simultaneously switch the in-car audio system to natural white noise;

[0102] When the discomfort is determined to be stress-related: control the concentration of lavender essential oil fragrance released by the car fragrance system, and at the same time switch the car audio to piano music with a BPM of <60;

[0103] When environmental discomfort is detected: when the basal body temperature is higher than the preset threshold, the vehicle's air conditioning is controlled to cool down; when the vehicle detects that the CO2 concentration is higher than the preset threshold, the air conditioning external circulation mode is activated.

[0104] Figure 2 This is a structural diagram of a driver status monitoring device based on AR glasses provided in one or more embodiments of the present invention.

[0105] like Figure 2 As shown, it includes:

[0106] The system includes a data acquisition and synchronization unit, a real-time data acquisition unit, a data preprocessing unit, a quantization result output unit, an instruction generation unit, and an instruction execution unit.

[0107] The data acquisition and synchronization unit is used to acquire the driver's physiological baseline data within a preset time threshold and synchronize the vehicle's initial state data.

[0108] Physiological baseline data include: PERCLOS value, skin conductance baseline, and basal body temperature; vehicle initial state data include: air conditioning temperature, fragrance concentration, and ADAS working status.

[0109] The real-time data acquisition unit is used to collect driver physiological data, motion status data, and environmental data through sensor components integrated on the AR glasses;

[0110] The data preprocessing unit is used to perform noise reduction and standardization processing on the driver's physiological signals, motion state signals and environmental data;

[0111] The quantification result output unit is used to input the processed data into a preset model for analysis and obtain the driver's state quantification result;

[0112] The instruction generation unit is used to generate three-level safety response instructions and comfort optimization instructions based on the driver's state quantification results.

[0113] The instruction execution unit is used to output level 3 safety response instructions through AR glasses and comfort optimization instructions through the vehicle control interface.

[0114] It is worth noting that although only some basic functional modules are disclosed in this embodiment, it does not mean that the composition of this system is limited to the above-mentioned basic functional modules. On the contrary, what this embodiment intends to express is that, based on the above-mentioned basic functional modules, those skilled in the art can arbitrarily add one or more functional modules in combination with existing technology to form an infinite number of embodiments or technical solutions. That is to say, this system is open rather than closed. The fact that this embodiment only discloses a few basic functional modules does not mean that the scope of protection of the claims of this invention is limited to the disclosed basic functional modules. At the same time, for the convenience of description, the above device is described separately according to its functions as various units and modules. Of course, in implementing this invention, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0115] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0116] Figure 3 This is a schematic diagram of the structural design and sensor layout of AR glasses according to a specific embodiment of the present invention.

[0117] like Figure 3 The diagram shows the structural design and sensor layout of the waveguide AR glasses. An eye-tracking sensor is located inside the lens, functioning to locate pupils, monitor blinks, and track gaze. A head IMU sensor is located in the center of the frame, confirming head posture. A camera on the left side of the frame enables environmental perception and gesture recognition. A PPG sensor is embedded in the nose pad for all-day heart rate / blood oxygen tracking. A skin conductivity sensor is also located on the nose pad; when the sympathetic nervous system is excited, sweat gland secretion increases, raising skin conductivity and allowing for better detection of user emotional changes. In situations prone to anxiety and anger, such as traffic jams or being cut off, the sensor can be linked to the vehicle's infotainment system to alleviate user tension through music, air conditioning, and fragrance, preventing impulsive and irrational decisions. A temperature sensor can better monitor the driver's body temperature, identifying physiological abnormalities such as heatstroke and fever that affect driving safety. It can also be integrated with air conditioning and window controls to automatically optimize the in-car environment and improve driver comfort.

[0118] Figure 4 This is a schematic diagram of a driver status monitoring method based on AR glasses, according to a specific embodiment of the present invention.

[0119] like Figure 4As shown, this embodiment uses eyeglasses as the core hub. Multi-source sensors integrated into the glasses (including a visual camera, a nose pad biosensor array, an IMU motion module, and an ambient light detector) collect real-time physiological signals (EDA / body temperature / heart rate) and ambient light data from the driver. This data is then combined with vehicle speed, steering, and ADAS status information transmitted from the vehicle for multimodal data fusion analysis. The decision engine dynamically generates three levels of safety warning instructions based on the fusion results—projecting AR warning images through a waveguide display system, transmitting information to the driver via bone conduction audio and tactile feedback from the temples, and simultaneously outputting environmental adjustment instructions to the vehicle control system via Wi-Fi / Bluetooth protocols. A PID algorithm precisely controls the air conditioning temperature and fragrance concentration, forming a closed-loop control circuit of "physiological signal acquisition → AR warning feedback → vehicle environmental regulation," ultimately achieving a synergistic improvement in driving safety and passenger comfort.

[0120] Figure 5 This is a flowchart illustrating the control logic design of a driver status monitoring method based on AR glasses, according to a specific embodiment of the present invention.

[0121] like Figure 5 As shown, this embodiment provides a driver status monitoring method based on vehicle-mounted optical waveguide AR glasses. The workflow is as follows: Figure 3As shown, the system first collects driver physiological characteristics and environmental data in real time through multiple sensors integrated into the glasses (including eye-tracking sensors, head IMU, biosensor group (EDA / temperature / PPG), and environmental perception module). The collected raw signals are processed by the preprocessing layer, and the processed data is input into the multi-dimensional fusion analysis layer. The fatigue risk model calculates the fatigue index by combining the PERCLOS visual index (weight 0.6), EDA baseline level (weight 0.3), and skin temperature change (weight 0.1). The distraction risk model generates a distraction score by multiplying the weighted maximum value of the duration of gaze deviation and head yaw angle by the stress coefficient. At the same time, the comfort assessment module quantifies the driver's comfort state through skin temperature fluctuation and EDA entropy value. The fusion analysis results drive the dynamic decision engine to generate a three-level response strategy: when the safety risk level is Level 1, a yellow AR prompt icon is generated at the edge of the field of vision to remind the user of their status; at Level 2, a clear orange icon appears within the field of vision, and the glasses themselves play a voice prompt indicating that the current situation is somewhat dangerous; Level 3... At 3 o'clock, a strong intervention red vibration warning is activated, and the glasses play a voice prompt while vibrating to stimulate the user to wake up from a state of fatigue or drowsiness. If the user does not take any action within 5 seconds in this state, the glasses send a signal to the vehicle's system to automatically switch driving modes and park at the nearest parking location. In addition to detecting fatigue, the glasses can also independently perform environmental optimization decisions based on comfort levels. In a fatigued state, a strong menthol fragrance is released and natural white noise is played; in a stressful state, lavender essential oil is released and piano music with a BPM of <60 is played; when a skin temperature >37℃ is detected, the air conditioning is lowered by 2℃ and the external circulation mode is activated when CO2 >1000ppm. Finally, the full-scene feedback execution layer achieves synchronous output through four channels: optical waveguide display, audio system, vehicle environmental control, and tactile feedback, forming a closed-loop control from biosignal perception to environmental adjustment, significantly improving warning efficiency and reducing driver cognitive load.

[0122] Figure 6 This is a block diagram of an electronic device for a driver status monitoring method based on AR glasses, provided by one or more embodiments of the present invention.

[0123] like Figure 6 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0124] The memory stores a computer program that, when executed by the processor, causes the processor to perform steps of a driver state monitoring method based on AR glasses.

[0125] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of a driver state monitoring method based on AR glasses.

[0126] This application relates to existing technology:

[0127] Optical waveguide technology:

[0128] Optical waveguide technology utilizes the principle of total internal reflection for imaging. It is further divided into geometric waveguides and diffractive waveguides. Geometric waveguides, also known as arrayed waveguides or reflective waveguides, are based on traditional geometric optics and use an array of semi-transparent and semi-reflective mirrors to achieve light transmission and coupling within the waveguide. Light emerges from a microdisplay, enters the waveguide through a mirror in the coupling region, and propagates within the waveguide via total internal reflection. In the coupling region, the exit pupil is expanded and coupled out through an array of semi-transparent and semi-reflective mirrors. Diffractive waveguide technology uses the diffraction properties of gratings to design the optical path, coupling the image from the microdisplay into the waveguide for propagation via total internal reflection, and finally entering the human eye through the coupling device for imaging. Diffractive waveguides are divided into two types: surface relief gratings and volume holographic gratings. Surface relief gratings are based on mature nanoimprinting or etching processes to etch periodic micro / nano structures onto the material surface, while volume holographic gratings use volume holographic grating elements instead of relief gratings.

[0129] Eye-tracking technology:

[0130] Eye tracking is a technology that captures and analyzes eye movement information to measure gaze direction, fixation point position, and eye movement patterns in real time. Its core lies in converting the physiological characteristics of the human eye into calculable data, enabling intelligent control through "gaze-based interaction." By utilizing sensors, such as infrared cameras, to record eye / pupil position, pupil changes, iris angle, gaze trajectory, and corneal reflection points, algorithms analyze gaze changes to obtain the distribution of visual attention. The technology requires a combination of hardware and software. Eye tracking solutions include pupil-corneal reflection methods, retinal image localization, structured light tracking, and corneal reflective light intensity. The most widely used method is the traditional corneal reflection method. Its principle involves shining low-power active near-infrared light into the eye. This light, invisible to the human eye, is reflected on the cornea. A sensor array then captures the near-infrared image, and after near-infrared signal processing, eye features are obtained, pupil position is identified, and the gaze direction and fixation position are estimated. This method offers advantages such as high accuracy, non-contact operation, minimal interference, and non-invasiveness. At its first Innovation Conference, Goertek launched high-precision eye-tracking AR smart glasses with an algorithm response latency of less than 3ms. The glasses feature an ultra-thin carbon fiber frame and ultra-light titanium alloy hinges, weighing only 36 grams, making them the lightest AR glasses in the industry. The dual-camera multi-light source solution, combined with advanced eye-tracking algorithms, enables full-field eye tracking with an average accuracy of 0.5°.

[0131] PPG Sensing:

[0132] Most wearable devices use photoplethysmography (PPG) to measure heart rate and other biometrics. PPG is a method that shines light into the skin and measures the light scattering caused by blood flow. It works by the predictable scattering of light entering the body when hemodynamics change, such as changes in pulse rate (heart rate) or blood volume (cardiac output). Meta glasses are the world's first to embed a PPG sensor in the nose pads of glasses, enabling all-day heart rate / blood oxygen tracking with a resting state error controlled within ±3 bpm, a medical-grade level.

[0133] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0134] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An AR glasses-based driver state monitoring method, characterized by, The method comprises the following steps: acquiring physiological baseline data of the driver within a preset time threshold, and synchronizing initial state data of the vehicle; the physiological baseline data comprises PERCLOS value, skin conductance baseline and basal body temperature, and the initial state data of the vehicle comprises air conditioning temperature, fragrance concentration and ADAS working state; acquiring driver physiological signals, motion state signals and environmental data through a sensor assembly integrated on AR glasses; performing noise reduction and standardization processing on the driver physiological signals, motion state signals and environmental data; inputting the processed data into a preset model for analysis to obtain driver state quantization results; generating three-level safety response instructions and comfort optimization instructions according to the driver state quantization results; outputting the three-level safety response instructions through the AR glasses and outputting the comfort optimization instructions through a vehicle control interface.

2. The AR glasses based driver state monitoring method of claim 1, wherein, the sensor assembly comprises eye tracking sensors, head IMU sensors, PPG biological sensors, skin conductance sensors, temperature sensors and ambient light detectors; the physiological signals comprise heart rate, blood oxygen, skin conductance value and body temperature; the motion state signals comprise head yaw angle, line of sight trajectory and blink frequency; the environmental data comprises ambient light intensity.

3. The AR glasses based driver state monitoring method of claim 1, wherein, the AR glasses acquire real-time running data of the vehicle through Wifi / Bluetooth protocol, and the real-time running data of the vehicle comprises vehicle speed and steering angle.

4. The AR glasses based driver state monitoring method of claim 1, wherein, the preset model comprises a fatigue risk model, a distraction risk model and a comfort evaluation model; the fatigue risk model acquires a fatigue index through a preset weight of the physiological baseline data; the distraction risk model generates a distraction score by multiplying a weighted maximum value of the head yaw angle and the line of sight deviation duration by a stress coefficient; the comfort evaluation model quantifies a comfort value by the skin temperature fluctuation and the EDA entropy value.

5. The AR glasses based driver state monitoring method of claim 4, wherein, the stress coefficient comprises: when the skin conductance baseline is within a preset threshold range interval from the baseline, the stress coefficient = 1.1; when the skin conductance baseline exceeds the preset threshold range interval from the baseline, the stress coefficient = 1.

2.

6. The AR glasses based driver state monitoring method of claim 1, wherein, the three-level safety response instructions specifically comprise: a low-risk response instruction, which only projects a yellow circular prompt icon on the edge of the driver's field of view through the AR glasses; a medium-risk response instruction, which projects an orange dynamic icon on the AR glasses and simultaneously plays a soft prompt sound; a high-risk response instruction, which projects a strong intervention red shaking icon on the AR glasses, plays a high-frequency warning sound and starts vibration for a preset time threshold; if no touch response of the driver is detected within the preset time threshold, an instruction is sent to the vehicle to switch to an emergency assistance mode and control the vehicle to decelerate and stop at the nearest parking location.

7. The AR glasses based driver state monitoring method of claim 1, wherein, the comfort optimization instructions specifically comprise: when fatigue-associated discomfort is determined, a car fragrance system is controlled to release a strong menthol fragrance, and a car audio is switched to natural white noise; when stress-associated discomfort is determined, a car fragrance system is controlled to release a strong lavender essential oil fragrance, and a car audio is switched to a piano music with BPM < 60. When it is determined that the environment is not suitable: when the basal body temperature is greater than a preset threshold, control the vehicle air conditioner to cool down; when the vehicle detects that the CO2 concentration is greater than a preset threshold, activate the air conditioner external circulation mode.

8. An AR glasses-based driver state monitoring apparatus, characterized by comprising: The method comprises the following steps: The data acquisition and synchronization unit, the real-time data acquisition unit, the data preprocessing unit, the quantitative result output unit, the instruction generation unit, and the instruction execution unit; The data acquisition and synchronization unit is configured to acquire physiological baseline data of a driver within a preset time threshold and synchronize vehicle initial state data; The physiological baseline data comprises a PERCLOS value, a skin conductance baseline, and a basal body temperature, and the vehicle initial state data comprises an air conditioner temperature, a fragrance concentration, and an ADAS working state; The real-time data acquisition unit is configured to acquire driver physiological data, motion state data, and environmental data through a sensor assembly integrated on AR glasses; The data preprocessing unit is configured to perform noise reduction and standardization processing on the driver physiological signals, motion state signals, and environmental data; The quantitative result output unit is configured to input the processed data into a preset model for analysis to obtain driver state quantitative results; The instruction generation unit is configured to generate three-level safety response instructions and comfort optimization instructions according to the driver state quantitative results; The instruction execution unit is configured to output the three-level safety response instructions through AR glasses and output the comfort optimization instructions through a vehicle control interface.

9. An electronic device, comprising: The method comprises the following steps: The processor, the communication interface, the memory, and the communication bus are in communication with each other through the communication bus; The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the driver state monitoring method based on AR glasses according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The memory stores a computer program executable by the electronic device, and when the computer program runs on the electronic device, the electronic device executes the steps of the driver state monitoring method based on AR glasses according to any one of claims 1-7.

Images