Hierarchical driving fatigue intervention method based on biological feature recognition

Abstract

The invention discloses a hierarchical driving fatigue intervention method based on biological feature recognition, and relates to the technical field of intelligent driving safety monitoring, and the method comprises the steps: collecting a facial image of a driver through a vehicle-mounted multispectral camera system, extracting facial feature data, and carrying out the preprocessing of the facial feature data; constructing a fatigue recognition model based on the preprocessed facial feature data, respectively obtaining a visual fatigue score and a behavior fatigue score, and fusing the visual fatigue score and the behavior fatigue score to generate a comprehensive fatigue index; implementing a hierarchical intervention mechanism according to the comprehensive fatigue index, and continuously monitoring and updating the comprehensive fatigue index to determine whether to trigger a safety protection mechanism. According to the invention, accurate response and risk prevention and control of the fatigue state are realized through hierarchical intervention and a safety protection mechanism, and the driving safety guarantee capability and the user experience are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving safety monitoring technology, and in particular to a hierarchical driving fatigue intervention method based on biometric recognition. Background Art

[0002] With the development of intelligent driving assistance systems, driver state monitoring has garnered widespread attention as a key research area for ensuring road traffic safety. In recent years, fatigue driving has been widely recognized by traffic management departments and research institutions as a key factor in traffic accidents. To promptly identify driver fatigue, researchers have conducted extensive research in visual perception, physiological signal acquisition, and multimodal fusion. Among these, image processing-based visual fatigue detection methods, such as those based on features such as eyelid opening and closing frequency, eye closure duration (PERCLOS), and eye movements, have proven effective. Furthermore, behavioral characteristics such as head posture changes are often used to characterize driver distraction. However, most current fatigue recognition systems still focus on single-dimensional signal recognition and lack multi-source fusion mechanisms, resulting in limitations in fatigue recognition accuracy and real-time performance. Furthermore, in terms of intervention mechanism design, many systems only provide a single prompt upon fatigue recognition, such as an audible alarm or instrument panel prompt. These systems lack the ability to differentiate interventions based on fatigue severity, making them difficult to meet the precise intervention requirements in complex driving scenarios.

[0003] Existing technologies also have obvious deficiencies in fatigue state intervention. On the one hand, traditional fatigue detection methods rely on a single visual or physiological parameter, which is easily interfered with by the external environment, leading to misjudgment or missed judgment; on the other hand, most systems fail to achieve closed-loop feedback and are unable to dynamically adjust intervention measures during continuous monitoring. In addition, current intervention methods are mostly limited to passive warnings and lack intervention logic based on fatigue level classification, resulting in insignificant stimulation effects and even affecting the safety of driving operations in some cases. How to establish a more accurate and robust fatigue assessment system and implement graded intervention based on the assessment results is a key difficulty in the current technological development in this field. Summary of the Invention

[0004] The present invention is proposed in view of the problems existing in the existing hierarchical driver fatigue intervention method based on biometric recognition. Therefore, the problem to be solved by the present invention is how to provide a hierarchical driver fatigue intervention method based on biometric recognition.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In a first aspect, the present invention provides a hierarchical driving fatigue intervention method based on biometric recognition, comprising:

[0007] The driver's facial image is collected through the vehicle-mounted multispectral camera system, facial feature data is extracted, and the facial feature data is preprocessed;

[0008] A fatigue recognition model is constructed based on the preprocessed facial feature data to obtain visual fatigue scores and behavioral fatigue scores respectively. The visual fatigue scores and behavioral fatigue scores are then integrated to generate a comprehensive fatigue index, which is expressed as:

[0009] F s =ω1V s +ω2B s

[0010] ω1+ω2=1

[0011] Among them: F s is the comprehensive fatigue index, V s is the visual fatigue score, B s is the behavioral fatigue score, ω1 is the visual fatigue score weight, and ω2 is the behavioral fatigue score weight;

[0012] The fatigue recognition model includes a visual fatigue scoring model and a behavioral fatigue scoring model; the visual fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer arranged in sequence; the behavioral fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer arranged in sequence;

[0013] Implement a graded intervention mechanism based on the comprehensive fatigue index, continuously monitor and update the comprehensive fatigue index to determine whether to trigger the safety protection mechanism;

[0014] The hierarchical intervention mechanism based on the comprehensive fatigue index includes:

[0015] When the comprehensive fatigue index is greater than the first level limit and less than the second level limit, the driver's fatigue state is judged to be mild fatigue, the in-vehicle intelligent aromatherapy system is activated to release a scent that stimulates the sense of smell, the air conditioning system in the car is adjusted simultaneously, the temperature in the car is lowered, and a mild fatigue reminder message is issued;

[0016] When the comprehensive fatigue index is greater than the second level limit and less than the third level limit, the driver's fatigue state is judged to be moderate fatigue, and the vehicle audio system is activated to issue a moderate fatigue prompt message;

[0017] When the comprehensive fatigue index is greater than the third level limit and the comprehensive fatigue index is less than the fourth level limit, the driver's fatigue state is judged to be severe fatigue, the micro directional air vents on the top of the cockpit are activated to blow cold air to the forehead area for stimulation, the central control screen flashing warning light is activated, and a severe fatigue warning message is issued through voice broadcast, and the safety monitoring state is entered at the same time.

[0018] As a preferred embodiment of the hierarchical driving fatigue intervention method based on biometric recognition described in the present invention, the fatigue recognition model includes the following contents:

[0019] Divide the acquired facial feature data into training set, validation set and test set;

[0020] Conduct model architecture design and training. The visual fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are set in sequence.

[0021] The input dimension of the input layer is 3, and the input is the eyelid opening and closing frequency, pupil diameter change amplitude, and eyeball rotation speed; the number of nodes in the first fully connected layer is 32, and the activation function is ReLU; the output of the first fully connected layer is connected to the second fully connected layer, the number of nodes in the second fully connected layer is 16, and the activation function is ReLU; the output of the second fully connected layer is connected to the output layer, the number of nodes in the output layer is 1, and a linear activation function is used to output the visual fatigue score;

[0022] The behavioral fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are arranged in sequence;

[0023] The input dimension of the input layer is 1, and the head tilt angle is input; the number of nodes in the first fully connected layer is 16, and the activation function is ReLU; the output of the first fully connected layer is connected to the second fully connected layer, the number of nodes in the second fully connected layer is 8, and the activation function is ReLU; the output of the second fully connected layer is connected to the output layer, the number of nodes in the output layer is 1, and a linear activation function is used to output the behavioral fatigue score;

[0024] The visual fatigue scoring model and the behavioral fatigue scoring model were trained, and the training goal was to stop when the mean square error (MSE) of the validation set dropped to ≤0.005.

[0025] As a preferred embodiment of the hierarchical driving fatigue intervention method based on biometric recognition described in the present invention, the multispectral camera system includes a visible light camera and a near-infrared camera; the facial feature data includes eyelid opening and closing frequency, head tilt angle, pupil diameter change amplitude, and eyeball rotation speed;

[0026] The pre-processing of facial feature data comprises:

[0027] The facial feature data extracted from consecutive frames are smoothed using the sliding average method;

[0028] All extracted facial feature data are normalized.

[0029] As a preferred solution of the hierarchical driving fatigue intervention method based on biometric recognition described in the present invention, in which: in the safety monitoring state, the comprehensive fatigue index calculation is repeated at predetermined intervals, and the most recent fatigue level is recorded. If the comprehensive fatigue index is greater than the fourth level limit in three consecutive detection results, the safety protection mechanism is triggered and vehicle speed control intervention is performed. At the same time, based on the positioning information of the on-board navigation system, the route to the nearest service area or parking area is automatically planned, and the driver is prompted through an in-vehicle alarm.

[0030] In a second aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein when the processor executes the computer program, it implements the steps of a hierarchical driving fatigue intervention method based on biometric recognition.

[0031] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, the steps of a hierarchical driving fatigue intervention method based on biometric recognition are implemented.

[0032] The beneficial effects of this invention include: This method enhances the system's ability to perceive fatigue states in multiple dimensions; It achieves precise response and risk control to fatigue states through graded intervention and safety protection mechanisms, significantly improving driving safety and user experience. This method is practical and feasible, and is applicable to a variety of scenarios, including smart cockpits, commercial transport vehicles, and unmanned driving monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0034] Figure 1 This is a flowchart of a hierarchical driving fatigue intervention method based on biometric recognition. DETAILED DESCRIPTION

[0035] To make the above-mentioned objects, features, and advantages of the present invention more easily understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in this field without creative work should fall within the scope of protection of the present invention.

[0036] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0037] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it constitute a separate or selective embodiment that is mutually exclusive with other embodiments.

[0038] Reference Figure 1 , which is the first embodiment of the present invention, provides a hierarchical driving fatigue intervention method based on biometric recognition, comprising:

[0039] S1: Capture the driver's facial image through the vehicle-mounted multispectral camera system, extract facial feature data, and preprocess the facial feature data;

[0040] Specifically, the driver's facial feature data is collected through a vehicle-mounted multispectral camera system, which includes a visible light camera and a near-infrared camera;

[0041] The driver's facial image is collected synchronously by visible light and near-infrared cameras, and the image frames are processed synchronously using timestamps to ensure that the data collected by the two cameras correspond to each other in time.

[0042] A face detection algorithm is used to identify the face region and further locate the eyes, pupils, and head position. The eyelid region is used to determine the eyelid opening and closing status, the eyeball region is used to analyze pupil and eye movement, and the head position is used to determine the tilt angle.

[0043] The collected facial images are cropped and standardized, and the key areas of the eyes and head are cropped out from the original images and uniformly adjusted to a fixed size (e.g., 64×64 pixels) for subsequent processing.

[0044] Extracting the driver's facial feature data from the fixed-size image, including eyelid opening and closing frequency, head tilt angle, pupil diameter change amplitude, and eyeball rotation speed;

[0045] The facial feature data extracted from continuous frames (such as degree of opening and closing, pupil diameter, head angle, etc.) are smoothed using the sliding average method to reduce jitter and noise interference.

[0046] All extracted facial feature data (such as eyelid opening and closing frequency, eyeball rotation speed, pupil size, and head angle) are normalized and uniformly converted into the [0,1] interval to ensure consistency of different feature scales for subsequent model processing.

[0047] S2: A fatigue recognition model is constructed based on the preprocessed facial feature data to obtain visual fatigue scores and behavioral fatigue scores respectively, and then the visual fatigue scores and behavioral fatigue scores are integrated to generate a comprehensive fatigue index;

[0048] Specifically, the facial feature data obtained in each time series window is divided into a training set, a validation set, and a test set according to a ratio of 70% / 15% / 15%.

[0049] Conduct model architecture design and training. The visual fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are set in sequence.

[0050] The input dimension of the input layer is 3, and the three visual feature parameters of the current window are input, namely eyelid opening and closing frequency, pupil diameter change amplitude, and eyeball rotation speed;

[0051] The first fully connected layer has 32 nodes and uses ReLU as the activation function. The output of the first fully connected layer is connected to the second fully connected layer, which has 16 nodes and uses ReLU as the activation function.

[0052] The output of the second fully connected layer is connected to the output layer. The number of nodes in the output layer is 1, and a linear activation function is used to regress and output the visual fatigue score.

[0053] The behavioral fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are arranged in sequence;

[0054] The input dimension of the input layer is 1, and the single behavioral feature parameter of the current window is input, that is, the head tilt angle;

[0055] The first fully connected layer has 16 nodes and uses ReLU as the activation function. The output of the first fully connected layer is connected to the second fully connected layer, which has 8 nodes and uses ReLU as the activation function.

[0056] The output of the second fully connected layer is connected to the output layer. The number of nodes in the output layer is 1, and a linear activation function is used to regress the output behavior fatigue score.

[0057] Training strategy: Adam optimizer, learning rate 0.001, batch size 32, maximum epoch 100;

[0058] Early stopping condition: If the validation set loss does not decrease for 10 consecutive rounds, the system terminates and saves the best weight.

[0059] Training goal: Stop when the mean square error (MSE) of the validation set drops to ≤0.005.

[0060] Verify on the test set: Confirm that MSE ≤ 0.005 and the residuals are unbiased.

[0061] The obtained visual fatigue score and behavioral fatigue score are used to calculate the comprehensive fatigue index. The two scores are linearly weighted according to the weight to form a comprehensive fatigue index, which is expressed as:

[0062] F s =ω1F s +ω2B s

[0063] ω1+ω2=1

[0064] Among them: F s is the comprehensive fatigue index, V s is the visual fatigue score, B s is the behavioral fatigue score, ω1 is the visual fatigue score weight, and ω2 is the behavioral fatigue score weight. The initial default settings are ω1 = 0.6 and ω2 = 0.4 to emphasize the visual fatigue score weight.

[0065] S3: Implement a graded intervention mechanism based on the comprehensive fatigue index, continuously monitor and update the comprehensive fatigue index to determine whether to trigger the safety protection mechanism.

[0066] Specifically, the system divides the driver's fatigue status into four levels based on the comprehensive fatigue index calculated in real time, and configures corresponding intervention strategies for each level. The level division and intervention measures are as follows:

[0067] When the comprehensive fatigue index is greater than the first level limit and the comprehensive fatigue index is less than the second level limit, the driver's fatigue state is judged to be mild fatigue, and the in-vehicle intelligent aromatherapy system is activated to release odors that stimulate the sense of smell to stimulate the sense of smell and enhance concentration; the air-conditioning system in the car is adjusted simultaneously to lower the temperature in the car to create a slightly cool environment to enhance the level of nervous excitement; and a mild fatigue prompt message is issued: "Mild fatigue is detected, it is recommended to stay focused."

[0068] When the comprehensive fatigue index is greater than the second level limit and less than the third level limit, the driver's fatigue state is judged to be moderate fatigue, and the vehicle audio system is activated at the same time; a moderate fatigue prompt message is issued: "Attention: You are in a moderate fatigue state, please consider stopping and resting as appropriate."

[0069] When the comprehensive fatigue index is greater than the third level limit and the comprehensive fatigue index is less than the fourth level limit, the driver's fatigue state is judged to be severe fatigue, and the micro directional air vents on the top of the cockpit are activated to blow cold air to the forehead area for stimulation; the central control screen flashes the prompt light and sounds a warning; a severe fatigue prompt message is issued through voice broadcast: "You are in a state of severe fatigue. It is recommended to find a suitable place to rest as soon as possible." At the same time, the system enters the safety monitoring state.

[0070] In the safety monitoring state, the system repeats the comprehensive fatigue index calculation process at predetermined intervals and records the most recent fatigue level. If the following conditions are met, the mandatory safety protection mechanism is activated:

[0071] If the comprehensive fatigue index is greater than the fourth level limit in three consecutive test results, the safety protection mechanism will be triggered, and the vehicle speed control intervention will be carried out to gradually reduce the vehicle speed. At the same time, based on the positioning information of the on-board navigation system, the route to the nearest service area or parking area will be automatically planned, and the driver will be notified through an in-vehicle alarm.

[0072] This embodiment also provides a computer device, which is applicable to a hierarchical driving fatigue intervention method based on biometric recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement all or part of the steps of the method described in the embodiment of the present invention as proposed in the above embodiment.

[0073] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, performs the method of any optional implementation of the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0074] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. Technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0075] In summary, this method enhances the system's ability to perceive fatigue states in multiple dimensions. Through graded intervention and safety protection mechanisms, it achieves precise response and risk control to fatigue states, significantly improving driving safety and user experience. This approach is both practical and feasible, and is applicable to a variety of scenarios, including smart cockpits, commercial transport vehicles, and autonomous driving monitoring.

[0076] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A hierarchical driving fatigue intervention method based on biometric recognition, characterized by: include: The driver's facial image is collected through the vehicle-mounted multispectral camera system, facial feature data is extracted, and the facial feature data is preprocessed; A fatigue recognition model is constructed based on the preprocessed facial feature data to obtain visual fatigue scores and behavioral fatigue scores respectively. The visual fatigue scores and behavioral fatigue scores are then integrated to generate a comprehensive fatigue index, which is expressed as: F s =ω1V s +ω2B s ω1+ω2=1 Among them: F s is the comprehensive fatigue index, V s is the visual fatigue score, B s is the behavioral fatigue score, ω1 is the visual fatigue score weight, and ω2 is the behavioral fatigue score weight; The fatigue recognition model includes a visual fatigue scoring model and a behavioral fatigue scoring model; the visual fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer arranged in sequence; the behavioral fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer arranged in sequence; Implement a graded intervention mechanism based on the comprehensive fatigue index, continuously monitor and update the comprehensive fatigue index to determine whether to trigger the safety protection mechanism; The hierarchical intervention mechanism based on the comprehensive fatigue index includes: When the comprehensive fatigue index is greater than the first level limit and less than the second level limit, the driver's fatigue state is judged to be mild fatigue, the in-vehicle intelligent aromatherapy system is activated to release a scent that stimulates the sense of smell, the air conditioning system in the car is adjusted simultaneously, the temperature in the car is lowered, and a mild fatigue reminder message is issued; When the comprehensive fatigue index is greater than the second level limit and less than the third level limit, the driver's fatigue state is judged to be moderate fatigue, and the vehicle audio system is activated to issue a moderate fatigue prompt message; When the comprehensive fatigue index is greater than the third level limit and the comprehensive fatigue index is less than the fourth level limit, the driver's fatigue state is judged to be severe fatigue, the micro directional air vents on the top of the cockpit are activated to blow cold air to the forehead area for stimulation, the central control screen flashing warning light is activated, and a severe fatigue warning message is issued through voice broadcast, and the safety monitoring state is entered at the same time.

2. The hierarchical driving fatigue intervention method based on biometric recognition according to claim 1, characterized in that: The fatigue recognition model includes the following contents: Divide the acquired facial feature data into training set, validation set and test set; Conduct model architecture design and training. The visual fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are set in sequence. The input dimension of the input layer is 3, and the input is the eyelid opening and closing frequency, pupil diameter change amplitude, and eyeball rotation speed; the number of nodes in the first fully connected layer is 32, and the activation function is ReLU; the output of the first fully connected layer is connected to the second fully connected layer, the number of nodes in the second fully connected layer is 16, and the activation function is ReLU; the output of the second fully connected layer is connected to the output layer, the number of nodes in the output layer is 1, and a linear activation function is used to output the visual fatigue score; The behavioral fatigue scoring model includes an input layer, a first fully connected layer, a second fully connected layer, an activation function unit, and an output layer, which are arranged in sequence; The input dimension of the input layer is 1, and the head tilt angle is input; the number of nodes in the first fully connected layer is 16, and the activation function is ReLU; the output of the first fully connected layer is connected to the second fully connected layer, the number of nodes in the second fully connected layer is 8, and the activation function is ReLU; the output of the second fully connected layer is connected to the output layer, the number of nodes in the output layer is 1, and a linear activation function is used to output the behavioral fatigue score; The visual fatigue scoring model and the behavioral fatigue scoring model were trained, and the training goal was to stop when the mean square error (MSE) of the validation set dropped to ≤0.

005.

3. The hierarchical driving fatigue intervention method based on biometric recognition according to claim 2, characterized in that: The multispectral camera system includes a visible light camera and a near-infrared camera; the facial feature data includes eyelid opening and closing frequency, head tilt angle, pupil diameter change amplitude and eyeball rotation speed; The pre-processing of facial feature data comprises: The facial feature data extracted from consecutive frames are smoothed using the sliding average method; All extracted facial feature data are normalized.

4. The hierarchical driving fatigue intervention method based on biometric recognition according to claim 3, characterized in that: In the safety monitoring state, the comprehensive fatigue index calculation is repeated at predetermined intervals, and the most recent fatigue level is recorded. If the comprehensive fatigue index is greater than the fourth-level limit in three consecutive test results, the safety protection mechanism is triggered and vehicle speed control intervention is performed. At the same time, based on the positioning information of the on-board navigation system, the route to the nearest service area or parking area is automatically planned, and the driver is notified through an in-vehicle alarm.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the hierarchical driving fatigue intervention method based on biometric recognition according to any one of claims 1 to 4 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the hierarchical driving fatigue intervention method based on biometric recognition as described in any one of claims 1 to 4 are implemented.

Images