Driving fatigue state detection and reminding method, system and device and medium

Abstract

The invention belongs to the technical field of intelligent driving and vehicle active safety, and provides a driving fatigue state detection and reminding method, system and device and a medium, and the method comprises the steps: obtaining the multi-feature-dimension driving state data of a current vehicle driver, the multi-feature-dimension driving state data contains driver physiological behavior data, vehicle operation alarm data and driving duration data; based on the multi-feature-dimension driving state data, determining the fatigue level of the driver through a fatigue grading model fusing objective data driving weight and subjective experience weight; and according to the fatigue level, executing a safety response strategy including multi-mode reminding and vehicle active disposal. The fatigue detection accuracy and the driving safety are remarkably improved through subjective and objective collaborative modeling and hierarchical closed-loop intervention.

Description

Technical Field

[0001] This invention belongs to the field of intelligent driving and vehicle active safety technology, and relates to a method, system, device and medium for detecting and reminding drivers of fatigue state based on multi-source information fusion and subjective and objective weight coordination. Background Technology

[0002] Fatigue driving refers to a state in which a driver, after driving continuously for a long time, experiences a disruption in physiological and psychological functions, resulting in decreased attention, slow reaction time, and reduced operational ability. If a driver becomes fatigued while driving, it may manifest as slow operation and unstable vehicle control in mild cases; in severe cases, the driver may fall asleep instantly, completely losing control of the vehicle, which can easily lead to serious traffic accidents and is a major hidden danger to road traffic safety.

[0003] Traditional methods for detecting driver fatigue primarily rely on contact sensors (such as steering wheel grip force sensors and ECG / EEG monitoring devices) to collect the driver's physiological signals and issue warnings via a single sound or vibration. However, these methods have significant drawbacks: firstly, contact devices are inconvenient to wear and uncomfortable, making them difficult to apply on a large scale in ordinary passenger vehicles; secondly, they rely solely on single-modal data (such as heart rate variability alone), resulting in a limited detection dimension and susceptibility to individual differences or environmental interference, leading to high false alarm and false negative rates.

[0004] To improve detection reliability, non-contact detection solutions based on multi-source information fusion have emerged in recent years. For example, published patent CN118823959A proposes to classify fatigue levels by fusing driver facial features (such as blinking and yawning) with vehicle driving status (such as lane departure, sudden braking, and continuous driving duration). Published patent CN121033812A proposes a method, system, device, and medium for dynamic assessment of driver fatigue risk, which also employs multi-dimensional data input. Furthermore, published patent CN117789181A further proposes to dynamically adjust reminder strategies based on fatigue levels (such as low-level voice reminders and high-level voice + vibration combinations).

[0005] While the aforementioned technologies have improved detection coverage and alert adaptability to some extent, the following key bottlenecks still exist: 1. The weight allocation in fatigue grading models lacks scientific basis: While existing schemes mention "fusion models," they do not clearly define how to determine the weights of each indicator (such as "frequency of yawning" and "number of lane departures"). In practice, equal-weighted averaging or end-to-end neural networks are often used. The former ignores the differences in importance between indicators, while the latter is a "black box" model, lacking interpretability and tunability. Especially for physiological signals that are "strongly correlated but low-frequency" (such as yawning), they are easily underestimated due to data sparsity, seriously affecting the accuracy of grading.

[0006] 2. Limited dimensions of reminder strategies and insufficient wake-up effect: Existing reminder methods mainly focus on hearing (voice) and touch (vibration), lacking the ability to respond to bodily sensations (such as temperature changes). This makes it difficult to effectively wake up drivers in a state of moderate to severe fatigue, and may even be ignored or startled due to the single stimulus, thus increasing the risk.

[0007] 3. Lack of proactive safety closed-loop mechanism: Current systems generally stop at the "detection-alert" stage. When the driver is unable to respond to the alarm due to deep fatigue, the system cannot proactively intervene in the vehicle's status (such as speed limit, activating hazard lights, or guiding the vehicle to a stop), failing to achieve complete safety protection from risk identification to proactive avoidance.

[0008] Therefore, there is an urgent need for a comprehensive fatigue driving prevention and control solution that can scientifically integrate subjective and objective weights to accurately determine fatigue levels, implement multi-channel collaborative wake-up, and possess the vehicle's proactive handling capabilities, so as to break through the limitations of existing technologies and effectively improve driving safety in intelligent driving environments. Summary of the Invention

[0009] To address the technical problems in existing fatigue driving detection technologies, such as the lack of scientific basis for indicator weight allocation, insufficient multimodal reminder effects, and the lack of vehicle active safety response mechanisms, this invention discloses a method for detecting and reminding drivers of driver fatigue, aiming to accurately determine the driver's fatigue level, efficiently awaken the driver from fatigue, and construct an integrated active safety closed loop of detection, reminder, and intervention.

[0010] Specifically, the method includes the following steps: S101. Obtain multi-feature dimension driving status data of the current vehicle driver, the multi-feature dimension driving status data including driver physiological behavior data, vehicle operation alarm data and driving duration data; S102. Based on the multi-feature dimension driving state data, determine the driver's fatigue level by using a fatigue grading model that integrates objective data-driven weights and subjective experience weights. S103. Based on the fatigue level, implement a safety response strategy that includes multimodal alerts and proactive vehicle handling.

[0011] Further, in step S101, multi-feature dimension driving state data of the current vehicle driver is obtained, including: S201. By combining the camera of the driver monitoring system (DMS) with computer vision algorithms, driver physiological behavior data is obtained. The driver physiological behavior data includes the following basic indicators: blink count information, yawn count information, and head state change count information. S202. Vehicle operation alarm data is collected through the sensors and vehicle network communication bus of the Advanced Driver Assistance System (ADAS). The vehicle operation alarm data includes the following basic indicators: lane departure alarm count, emergency braking alarm count, and sharp turn alarm count. S203. Obtain driving time data through the vehicle-mounted timing module and extract continuous driving time information.

[0012] Further, in step S102, determining the driver's fatigue level includes: S301. Standardize the base class indicators in each feature dimension to obtain standardized values; S302. Calculate the objective weight of each base class indicator and the subjective weight of the feature dimension to which the base class indicator belongs, and calculate the product of the objective weight and the corresponding subjective weight to obtain the comprehensive weight of each base class indicator. S303. Based on all standardized values ​​and their corresponding comprehensive weights, a weighted sum is performed to obtain the comprehensive fatigue score; S304. Determine the driver's fatigue level based on the preset threshold range where the comprehensive fatigue score is located.

[0013] Furthermore, the objective weights are based on the CRITIC method, obtained according to the standard deviation of each base class indicator and its correlation with other base class indicators. The subjective weights are based on the Analytic Hierarchy Process (AHP), obtained by constructing importance judgment matrices for three feature dimensions: driver physiological behavior characteristics, vehicle dynamic characteristics, and driving duration characteristics, and solving for the eigenvectors.

[0014] Furthermore, the fatigue levels include no fatigue, mild fatigue, moderate fatigue, and severe fatigue.

[0015] Further, in step S103, based on the fatigue level, a safety response strategy including multimodal alerts and proactive vehicle handling is executed, including: S501. Based on the fatigue level, activate the corresponding multimodal alert strategy to remind the driver of fatigue, including: For mild fatigue, use a combination of at least two of the following methods to provide a reminder: voice, image, and light. For moderate fatigue, use a combination of voice and image reminders, along with light or temperature adjustments; For severe fatigue, a combination of voice, light, temperature adjustment, and image reminders will be used. S502. If no valid response is detected within a preset time, vehicle status handling is performed, including limiting vehicle speed, turning on hazard warning lights, or guiding the vehicle to a safe stop.

[0016] Furthermore, temperature adjustment reminders lower the interior temperature or increase airflow through the vehicle's air conditioning system to provide a sensory wake-up stimulus to the driver. Visual reminders display prominent fatigue warning icons or dynamic animated prompts via a head-up display (HUD) or the vehicle's instrument panel.

[0017] This invention also provides a system for detecting and alerting drivers to fatigue, including a data acquisition module, a fatigue grading module, and a safety response module.

[0018] The data acquisition module is used to acquire multi-feature-dimensional driving status data of the current vehicle driver. The multi-feature-dimensional driving status data includes driver physiological behavior data, vehicle operation alarm data, and driving duration data. The fatigue grading module is used to determine the driver's fatigue level based on the multi-feature dimension driving state data by fusing objective data-driven weights and subjective experience weights into a fatigue grading model. The safety response module is used to execute safety response strategies, including multimodal alerts and proactive vehicle handling, based on the fatigue level.

[0019] This invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned method for detecting and reminding drivers of any of the driving fatigue states, thereby solving the technical problems in existing fatigue driving detection technologies, such as the lack of scientific basis for the allocation of indicator weights, insufficient multimodal reminder effects, and the lack of active safety handling mechanisms for vehicles.

[0020] This invention also provides a computer-readable storage medium storing a computer program that executes any of the above-described methods for detecting and alerting driver fatigue, in order to solve the technical problems existing in current driver fatigue detection technologies, such as the lack of scientific basis for indicator weight allocation, insufficient multimodal alerting effect, and lack of vehicle active safety handling mechanisms.

[0021] The method of this invention first integrates driver physiological behavior characteristics (such as blinking and yawning) with vehicle dynamic information (such as lane departure, sudden braking, and continuous driving duration) to construct a unique correlation record; secondly, it calculates the objective weight of indicators based on the CRITIC method, determines the subjective weight of features based on the AHP method, and calculates the comprehensive weight, then weighted sums to obtain a comprehensive fatigue score to accurately determine the fatigue level; finally, it triggers multimodal reminders such as voice, lights, temperature adjustment, and images according to the level, and performs active vehicle actions such as speed limit, hazard lights, or automatic parking when necessary, forming an integrated active safety closed loop of "perception-assessment-response" to reduce accident risk, reduce road risk level, and ensure driving and driver safety.

[0022] This method and system reduce detection errors caused by single-dimensional data by collecting data from multiple feature dimensions. It can adapt to complex driving environments, improve the universality of fatigue driving detection, and provide differentiated alerts based on fatigue levels. This can prevent single alerts from being ignored and reduce the risk of accidents. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the method for detecting and alerting driver fatigue according to the present invention; Figure 2 This is the execution flow of the method for detecting and alerting driver fatigue status according to the present invention; Figure 3 Flowchart for acquiring multi-feature dimension driving status data of the current vehicle driver; Figure 4 The process for determining a driver's fatigue level; Figure 5 To determine the feature dimensions and corresponding base class indicators required to determine a driver's fatigue level; Figure 6 The process for implementing safety response strategies that include multimodal alerts and proactive vehicle handling; Figure 7 Architecture diagram of the driver fatigue state detection and reminder system of this invention; Among them, 701 is the data acquisition module; 702 is the fatigue grading module; and 703 is the safety response module. Detailed Implementation

[0025] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0026] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features of the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] This invention discloses a method for detecting and alerting drivers to fatigue. See [link to relevant documentation]. Figure 1 and Figure 2 As shown, the method includes the following steps: S101. Obtain multi-feature dimension driving status data of the current vehicle driver, the multi-feature dimension driving status data including driver physiological behavior data, vehicle operation alarm data and driving duration data; S102. Based on the multi-feature dimension driving state data, determine the driver's fatigue level by using a fatigue grading model that integrates objective data-driven weights and subjective experience weights. S103. Based on the fatigue level, implement a safety response strategy that includes multimodal alerts and proactive vehicle handling.

[0028] The method of this invention collects dynamic information about vehicle operation and physiological behavior data of the driver, using these two as uniquely correlated multi-feature-dimensional driving state data to provide a data foundation for subsequent fatigue driving detection. Based on this uniquely correlated multi-feature-dimensional driving state data and a preset fatigue grading model, the driver's fatigue level is determined, achieving multi-modal data complementarity, reducing single-modal errors, adapting to complex driving environments, improving the accuracy of fatigue driving detection, and reducing the fatigue false positive rate.

[0029] Furthermore, by employing different strategies for fatigue alerts based on the driver's fatigue level, it's possible to avoid ignoring a single alert, reduce accident risk, and ensure driving safety. Even better, if no effective response is detected after a fatigue alert, vehicle intervention can be implemented to further reduce road risk and ensure driver safety.

[0030] In one embodiment of step S101, see Figure 3 As shown, the system obtains multi-dimensional driving status data of the current vehicle driver, including: S201. By combining the camera of the Driver Monitoring System (DMS) with computer vision algorithms, driver physiological behavior data is obtained. The driver physiological behavior data includes the following basic indicators: blink count, yawn count, and head state change count.

[0031] Specifically, the driver in this invention refers to the person sitting in the driver's seat and operating the vehicle. The driver's physiological behavior data mainly includes the driver's facial expressions and head posture changes. The driver's driving status data is captured by the in-vehicle camera in front of the driver's seat or the driver monitoring system of intelligent assisted driving.

[0032] The blink count information can be extracted by calculating the driver's eye opening and closing degree using an algorithm. The yawn count information (i.e., mouth state) can also be determined by calculating changes in the driver's mouth size using an algorithm. Head state information includes the number of times the head is tilted back, tilted down, or looking straight ahead; these changes can be identified using a camera to capture facial images and computer vision algorithms. It should be noted that the calculation of the above information can be performed using known algorithms, and this invention will not elaborate on the calculation methods.

[0033] S202. Vehicle operation alarm data is collected through the sensors and vehicle network communication bus of the Advanced Driver Assistance System (ADAS). The vehicle operation alarm data includes the following basic indicators: lane departure alarm count, emergency braking alarm count, and sharp turn alarm count.

[0034] S203. Obtain driving time data through the vehicle-mounted timing module and extract continuous driving time information.

[0035] Specifically, the aforementioned vehicle operation alarm data and continuous driving time information can both be categorized as dynamic information of vehicle operation. This dynamic information refers to data generated by any slight change in the vehicle's state during driving. It can be obtained by using onboard sensors to collect driving data such as alarms for abnormal lane departure, braking alarms for sudden speed changes, changes in speed, and rapid body rotation.

[0036] The driver's physiological behavior data and vehicle dynamic information (including vehicle operation alarm data and continuous driving duration information) obtained in steps S201-S203 are used to construct a unique associated record, providing a rich data foundation for subsequent fatigue classification.

[0037] In one embodiment of step S102, see Figure 4 As shown, determining a driver's fatigue level includes: S301. Standardize the base class indicators in each feature dimension to obtain standardized values; S302. Calculate the objective weight of each base class indicator and the subjective weight of the feature dimension to which the base class indicator belongs, and calculate the product of the objective weight and the corresponding subjective weight to obtain the comprehensive weight of each base class indicator. S303. Based on all standardized values ​​and their corresponding comprehensive weights, a weighted sum is performed to obtain the comprehensive fatigue score; S304. Determine the driver's fatigue level based on the preset threshold range where the comprehensive fatigue score is located.

[0038] Furthermore, the objective weights are based on the CRITIC method, obtained according to the standard deviation of each base class indicator and its correlation with other base class indicators. The subjective weights are based on the Analytic Hierarchy Process (AHP), obtained by constructing importance judgment matrices for three feature dimensions: driver physiological behavior characteristics, vehicle dynamic characteristics, and driving duration characteristics, and solving for the eigenvectors.

[0039] Furthermore, fatigued driving is generally categorized into two states: no fatigue driving and fatigued driving. No fatigue driving is defined as driving without fatigue. Fatigued driving can be further classified into three levels based on the degree of fatigue: mild fatigue, moderate fatigue, and severe fatigue. This level is determined based on collected multi-dimensional driving status data, a fatigue driving judgment model, and a pre-set fatigue threshold range.

[0040] More specifically, a Driver Monitoring System (DMS) can be used to analyze anomalies in the driver's physiological behavioral characteristics, such as eye, mouth, and head posture. This can be further used to determine the dynamic characteristics of the vehicle, including the number of lane departure warnings, emergency braking warnings, sharp turn warnings, and vehicle driving time. By comprehensively utilizing the driver's physiological behavioral characteristics, vehicle dynamic characteristics, and driving time characteristics, the driver's driving state can be determined. Figure 5 As shown, this allows for improved fatigue detection accuracy through multi-dimensional data assessment. By judging the driver's driving state, the fatigue level can be determined as mild, moderate, or severe.

[0041] Figure 4 It provides feature dimensions for determining fatigue levels, with each feature dimension further divided into certain base class indicators. During the processing of each detection data point, the standardized values ​​of all base class indicators from the multi-feature-dimensional driving state data, along with their corresponding comprehensive weights, are weighted and fused to calculate the comprehensive fatigue score. The calculation formula is as follows: ; in, It is the comprehensive fatigue score, where n is the number of feature dimensions; It is the weight of the i-th feature dimension. It is the first Fatigue scores for each feature dimension.

[0042] The weights of each feature dimension are obtained by multiplying the objective weights by the corresponding subjective weights. The objective weights are obtained using the CRITIC method, and the subjective weights are obtained using the Analytic Hierarchy Process (AHP).

[0043] Each fatigue level, such as no fatigue, mild fatigue, moderate fatigue, and severe fatigue, has a comprehensive fatigue score. Each vehicle has its own fatigue score range. The fatigue score range is generated by a machine learning model trained on historical data. This model establishes a nonlinear mapping relationship between fatigue level and physiological behavioral characteristics, vehicle dynamic characteristics, and driving duration characteristics by analyzing a large amount of historical sample data, and finally outputs a quantitative range with statistical confidence.

[0044] like Figure 5 As shown, the fatigue level is determined by a fusion of three characteristic categories: physiological behavioral characteristics, vehicle dynamic characteristics, and driving duration characteristics. That is, in the formula above... , , The fatigue scores represent physiological behavioral characteristics, vehicle dynamic characteristics, and driving duration characteristics, respectively. When the driver's overall fatigue score... The result was obtained by weighting the three fatigue scores mentioned above.

[0045] Based on comprehensive fatigue score The score range of the fatigue level can be used to determine which fatigue level the driver belongs to. For example: when... The fatigue level can be considered as no fatigue. At that time, the fatigue level was determined to be mild fatigue; At that time, the fatigue level was determined to be moderate fatigue; At that time, the fatigue level was determined to be severe fatigue. It should be noted that the above division of intervals is not unique and can be adjusted or redefined based on historical data or specific needs.

[0046] More specifically, if the overall fatigue score is detected... If the overall fatigue score is low, the driver is deemed to be not fatigued and no warning is required; if the overall fatigue score is low, the driver is deemed to be not fatigued and no warning is required. If the overall fatigue score is 5, the driver is determined to be in a state of mild fatigue; if the overall fatigue score is... A score of 7 indicates the driver is in a state of moderate fatigue; if the overall fatigue score is... If the value is 9, the driver is determined to be in a state of severe fatigue. This avoids misjudgments caused by fluctuations in single-modal data and facilitates differentiated alerts for fatigued drivers in the future.

[0047] In addition, fatigue scores of physiological and behavioral characteristics Fatigue score of vehicle dynamic characteristics Fatigue score based on driving duration characteristics The processing also follows the principle of fusing features from each base class: ; in, It is the i-th feature dimension. The combined weight of each base category indicator. Corresponding to the i-th feature dimension The original values ​​of each base class indicator. It is the i-th feature dimension. Standardized values ​​of each base class indicator, and This will be described and processed later when handling the weights; it will be ignored here. For example... Figure 5 As shown, These represent the standardized values ​​of blinking frequency, yawning frequency, and head-down and head-up frequencies in physiological behavioral characteristics; the standardized values ​​of emergency braking warning frequency, sharp turn warning frequency, and lane departure warning frequency in vehicle dynamic characteristics; and the standardized values ​​of driver's driving behavior.

[0048] The weight calculation of the base class indicators and the weight calculation of the feature dimensions adopt objective weight and subjective weight calculation methods, respectively. This invention only describes in detail the objective weight calculation method of the base class indicators.

[0049] Specifically, using historical data as training data, we need to obtain the frequency of blinking, yawning, head-down and head-up movements, sudden braking, sharp turning, lane departure, and driving duration. These base class indicators serve as positive indicators for fatigue driving assessment, and the influence of dimensions needs to be eliminated. Each driver will correspond to a set of base class indicators under a specific feature dimension, which will then undergo Min-Max normalization standardization. ; in, This represents the standardized value of the j-th base class index in the i-th feature dimension of the k-th driver. This represents the value of the j-th base class index in the i-th feature dimension of the k-th driver. and This represents the minimum and maximum values ​​of the base class index in the i-th feature dimension across all training data.

[0050] Next, we need to calculate the mean and standard deviation of the base class metrics, where T represents the number of training datasets: ; in, This represents the average value of the base class indicators. This represents the standard deviation of the base class index.

[0051] Based on the above standardization, mean, and standard deviation processing, we can obtain a standardized list of base class indicators based on drivers, as well as a list of the mean and standard deviation of each base class indicator.

[0052] Based on the above standardized list, mean, and standard deviation list, the correlation coefficient can be obtained. , which represents the correlation coefficient between the m-th base class indicator and the n-th base class indicator in the i-th feature dimension, and is used to describe the degree of correlation between the two base class indicators in the i-th feature dimension.

[0053] After obtaining the pairwise correlation coefficients for each feature, a list is generated where the number of rows and columns are both the number of base class indicators for each feature dimension. Finally, the information content of the j-th base class indicator in the i-th feature dimension is determined. Thus, its corresponding weight is obtained. ,in Let be the capacity of the base class indicators in the i-th feature dimension.

[0054] In one embodiment of step S103, see Figure 6 As shown, based on the fatigue level, a safety response strategy including multimodal alerts and proactive vehicle handling is implemented, including: S501. Based on the fatigue level, activate the corresponding multimodal alert strategy to remind the driver of fatigue, including: For mild fatigue, use a combination of at least two of the following methods to provide a reminder: voice, image, and light. For moderate fatigue, use a combination of voice and image reminders, along with light or temperature adjustments; For severe fatigue, a combination of voice, light, temperature adjustment, and image reminders will be used. S502. If no valid response is detected within a preset time, vehicle status handling is performed, including limiting vehicle speed, turning on hazard warning lights, or guiding the vehicle to a safe stop.

[0055] It should be noted that the vehicle condition handling of S502 is mainly aimed at moderate and severe fatigue.

[0056] Furthermore, temperature adjustment reminders lower the interior temperature or increase airflow through the vehicle's air conditioning system to provide a sensory wake-up stimulus to the driver. Visual reminders display prominent fatigue warning icons or dynamic animated prompts via a head-up display (HUD) or the vehicle's instrument panel.

[0057] Voice prompts not only include audible and silent prompts, but also varying volume levels for different fatigue levels, and gradually increasing loudness of repeated prompts to prevent drivers from making mistakes due to disorientation when transitioning from quiet to noisy environments. Similarly, light prompts go beyond simply indicating brightness levels; they include varying the color and speed of color changes at different fatigue levels to prevent drivers from neglecting safety due to the dimness of a single light source.

[0058] In addition, other reminder methods can be set, such as using the fragrance enhancement of the in-car fragrance system or the vibration and massage of the seat massage system to assist in reminding you.

[0059] Furthermore, if no response is received within a reasonable time after being alerted, vehicle status adjustments can be made. For example, the vehicle itself will activate hazard warning lights and reduce speed. If the vehicle is equipped with autonomous driving assistance, this function can be activated to guide the vehicle to a safe location and stop.

[0060] The effective response referred to in step S502 can be understood as the driver's actions within a preset time window (e.g., 30 seconds) after receiving a fatigue warning, demonstrating a return to alertness, including but not limited to any of the following: (1) The steering wheel rotation angle exceeds the preset threshold (e.g., ±15°). (2) A significant change occurs in the force / frequency of pressing the accelerator or brake pedal; (3) The blinking frequency returns to the normal range (e.g., 15–30 times per minute); (4) The head posture returns from a drooping position to a normal position looking straight ahead.

[0061] If no valid response is detected within the preset time, the driver is determined to be in a state of incapacity or deep fatigue, and the system will automatically trigger the vehicle's active response measures.

[0062] This method and system reduce detection errors caused by single-dimensional data through multi-dimensional data collection, can adapt to complex driving environments, improve the universality of fatigue driving detection, and provide differentiated reminders and actions based on fatigue levels, which can prevent single reminders from being ignored and reduce the risk of accidents.

[0063] Based on the same inventive concept, this invention also provides a system for detecting and alerting driver fatigue, as described in the following embodiments. Since the principle underlying the problem-solving of the system for detecting and alerting driver fatigue is similar to that of the method for detecting and alerting driver fatigue, the implementation of the system can refer to the implementation of the method for detecting and alerting driver fatigue disclosed in the above embodiments, and will not be repeated. As used below, the terms "unit" or "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0064] Figure 7 This is a structural block diagram of a system for detecting and alerting driver fatigue, as disclosed in an embodiment of the present invention. Figure 7 As shown, the system includes a data acquisition module 701, a fatigue grading module 702, and a safety response module 703. The structure is described below.

[0065] The data acquisition module 701 is used to acquire multi-feature dimension driving status data of the current vehicle driver, including: driver monitoring module and vehicle operation monitoring module. The driver monitoring module is used to acquire driver physiological behavior data, and the vehicle operation monitoring module is used to acquire vehicle operation alarm data and driving time data.

[0066] The fatigue grading module 702 is used to determine the driver's fatigue level based on the multi-feature dimension driving state data by fusing objective data driving weights and subjective experience weights into a fatigue grading model. The safety response module 703 is used to execute a safety response strategy, including multimodal alerts and proactive vehicle handling, based on the fatigue level.

[0067] The embodiments of this invention significantly improve the accuracy, robustness, and safety of fatigue driving detection through multi-feature dimension data fusion and subjective-objective collaborative modeling, specifically manifested in the following technical effects: 1. By complementing heterogeneous data from multiple sources, detection robustness can be significantly improved: The method of this invention constructs a unique association record between the driver and the vehicle by simultaneously acquiring the driver's physiological behavioral characteristics (including the number of blinks, yawns, and head posture changes) and the vehicle's dynamic operation information (including the number of lane departure warnings, emergency braking / sharp turning warnings, and continuous driving duration). This multimodal data fusion mechanism effectively overcomes the problems of misjudgment caused by occlusion, individual differences, or environmental interference from single sensors or single-type indicators, significantly reducing the false alarm rate and the missed alarm rate, and adapting to complex and ever-changing real-world driving scenarios.

[0068] 2. Construct a scientific weighting model to improve the accuracy of fatigue classification: The objective weights of each indicator are calculated using the CRITIC method, reflecting the inherent dispersion and conflict of the data. Subjective weights for three categories of characteristics—physiological, vehicle-related, and duration—are determined using the AHP method, incorporating expert judgment. A comprehensive weight is generated through this fusion, and a comprehensive fatigue score is then calculated. This collaborative subjective and objective weighting mechanism effectively addresses the industry challenge of underestimating "strongly correlated but low-frequency signals" (such as yawning), making fatigue level assessment more scientific and interpretable.

[0069] 3. Enhance wake-up effectiveness through tiered multimodal reminders: Based on the fatigue level, a multi-channel reminder method is dynamically combined, including voice, light, temperature adjustment (such as air conditioning to cool down or increase fan speed), and HUD / instrument images. This avoids the problem that traditional single sound or vibration reminders are easily ignored or cause fright, and significantly improves the wake-up efficiency for mild to severe fatigue states.

[0070] 4. Proactive safety closed-loop system to enhance risk prevention and control capabilities: When driver fatigue is moderate to severe and warnings are ineffective, the system automatically implements vehicle status control measures, including limiting speed, activating hazard lights, and even guiding the vehicle to a safe stop. This constructs a complete proactive safety loop of "detection-assessment-warning-intervention," reducing the risk of major traffic accidents caused by driver incapacity at the source.

[0071] In summary, the method of this invention not only improves the universality and accuracy of fatigue driving detection, but also achieves a technological leap from passive warning to active protection through differentiated response strategies and active vehicle control, demonstrating outstanding substantive features and significant progress.

[0072] In this embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-described methods for detecting and reminding drivers of fatigue states.

[0073] Specifically, the computer device can be a computer terminal, a server, or a similar computing device.

[0074] In this embodiment, a computer-readable storage medium is provided, which stores a computer program that performs any of the above-described methods for detecting and alerting driver fatigue.

[0075] Specifically, computer-readable storage media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable storage media does not include transient media, such as modulated data signals and carrier waves.

[0076] Obviously, those skilled in the art should understand that the modules or steps of the above-described embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0077] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations can be made to the embodiments of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method of driving fatigue state detection and reminding, characterized in that, The method comprises the following steps: acquiring multi-feature dimension driving state data of a current vehicle driver, the multi-feature dimension driving state data comprising driver physiological behavior data, vehicle operation warning data and driving duration data; determining a fatigue level of the driver based on the multi-feature dimension driving state data by using a fatigue grading model that fuses objective data driving weights and subjective experience weights; executing a safety response strategy comprising multi-modal reminders and vehicle active handling according to the fatigue level.

2. The method of claim 1, wherein The method of acquiring multi-feature dimension driving state data of a current vehicle driver comprises the following steps: acquiring driver physiological behavior data by using a camera of a driver monitoring system (DMS) and a computer vision algorithm, the driver physiological behavior data comprising the following base indicators: blink frequency information, yawn frequency information and head state change frequency information; collecting vehicle operation warning data by using sensors of an advanced driver assistance system (ADAS) and a vehicle network communication bus, the vehicle operation warning data comprising the following base indicators: lane departure warning frequency information, sudden braking frequency warning information and sudden turning frequency warning information; acquiring driving duration data by using a vehicle timing module and extracting continuous driving duration information.

3. The method of claim 1, wherein the step of detecting the driver's fatigue state comprises the steps of: detecting the driver's fatigue state by using the driver's eye state and the driver's head state. The method of determining a fatigue level of the driver comprises the following steps: standardizing each base indicator in each feature dimension to obtain standardized numerical values; calculating objective weights of each base indicator and subjective weights of the feature dimension to which the base indicator belongs, calculating the product of the objective weight and the corresponding subjective weight to obtain the comprehensive weight of each base indicator; performing weighted summation based on all standardized numerical values and their corresponding comprehensive weights to obtain a comprehensive fatigue score; determining the fatigue level of the driver according to the preset threshold interval in which the comprehensive fatigue score falls.

4. The method of claim 3, wherein, The objective weights are obtained based on the CRITIC method according to the standard deviation of each base indicator and its correlation with other base indicators; The subjective weights are obtained by constructing an importance judgment matrix of the three feature dimensions of driver physiological behavior characteristics, vehicle dynamic characteristics and driving duration characteristics and solving the characteristic vector based on the analytic hierarchy process (AHP).

5. The method of claim 1 to 4, wherein The fatigue level comprises no fatigue, mild fatigue, moderate fatigue and severe fatigue.

6. The method of claim 5, wherein, The method of executing a safety response strategy comprising multi-modal reminders and vehicle active handling according to the fatigue level comprises the following steps: starting a corresponding multi-modal reminder strategy to remind the driver of fatigue according to the fatigue level, which comprises the following steps: if the fatigue level is mild fatigue, at least two of voice, image and light are combined for reminding; if the fatigue level is moderate fatigue, voice and image are combined for reminding with light or temperature adjustment; if the fatigue level is severe fatigue, voice, light, temperature adjustment and image are combined for reminding; when no valid response is detected within a preset time, executing vehicle state handling, which comprises limiting vehicle speed, turning on a danger warning light or guiding the vehicle to safely park.

7. The method of claim 6, wherein, Temperature adjustment reminders provide a sense of awakening stimulation to the driver by reducing the temperature inside the vehicle or enhancing the air outlet wind power through a vehicle air conditioning system; Image reminders display a prominent fatigue warning icon or dynamic prompt animation on a head-up display (HUD) or a vehicle instrument panel.

8. A system for detecting and alerting drivers to fatigue, characterized in that, The method comprises the following steps: A data collection module is configured to acquire multi-feature dimension driving state data of a current vehicle driver, the multi-feature dimension driving state data including driver physiological behavior data, vehicle operation warning data, and driving duration data; A fatigue grading module is configured to determine a fatigue level of the driver based on the multi-feature dimension driving state data by using a fatigue grading model that fuses objective data-driven weights and subjective experience weights. A safety response module is configured to execute a safety response strategy including multi-modal reminders and vehicle active handling according to the fatigue level.

9. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method for detecting and reminding driving fatigue according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program for executing the method for detecting and reminding driving fatigue according to any one of claims 1 to 7.

Images