Personalized freight driver fatigue monitoring method

Abstract

The invention discloses a personalized freight driver fatigue monitoring method, which comprises the steps of 1, acquiring a driver face video stream in real time, extracting multi-dimensional face fatigue features and constructing a face reference vector; 2, setting a reference data acquisition window, and collecting personalized reference feature vectors; 3, establishing a personalized judgment threshold value of each fatigue behavior of the driver; 4, constructing a personalized fatigue risk function of the driver, and calculating a fatigue risk value; and 5, comparing the personalized driving risk value with a preset threshold value, and judging whether a risk early warning signal is sent to the driver or not. According to the invention, personalized driver fatigue monitoring and risk assessment can be realized, so that traffic accidents are prevented, and the road traffic safety level can be improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation and driving safety, specifically a personalized method for monitoring fatigue in freight drivers. Background Technology

[0002] In recent years, with the continuous development of road transportation systems, the number of freight vehicles on the roads has been increasing, and the risks and pressures faced by drivers during transportation gradually accumulate with driving time. In the monotonous driving environment of long hours and high loads, drivers are highly susceptible to fatigue, which not only reduces reaction speed and judgment but also seriously threatens road traffic safety. However, most existing visual fatigue monitoring technologies are based on general population physical characteristics and set fixed judgment thresholds, lacking the monitoring of individual driver fatigue characteristics.

[0003] Furthermore, traditional driving risk monitoring methods lack the ability to comprehensively monitor and dynamically assess multiple fatigue risks when dealing with multidimensional fatigue characteristics. Additionally, driver behavior exhibits significant time-varying characteristics at different fatigue stages, and fixed-weight assessment models cannot capture the most significant risk features in real time. Therefore, with the support of computer vision technology, designing a personalized freight driver fatigue monitoring method that can adaptively calibrate individual driver benchmarks and dynamically adjust evaluation weights based on real-time feature data fluctuations is a pressing technical problem that needs to be solved. Summary of the Invention

[0004] This invention aims to overcome the shortcomings of existing technologies by proposing a personalized method for monitoring driver fatigue in freight vehicles. This method enables personalized monitoring and risk assessment of driver fatigue, thereby preventing traffic accidents and improving road traffic safety.

[0005] To achieve the above-mentioned objectives, the present invention adopts the following technical solution: The present invention provides a personalized method for monitoring fatigue in freight drivers, characterized by the following steps: Step 1: Real-time acquisition and processing of the driver's facial video stream inside the truck cab to obtain the set of key facial feature points in the t-th frame image. and its facial features independent variables ; Step 2: Calculate the independent variables of facial features in each frame of the facial video stream within the acquisition window. Intra-basic eigenvectors ;in, Indicates the data acquisition window The average aspect ratio of the eyes within the eye, Indicates the data acquisition window The average aspect ratio of the mouth is within the mouth. Indicates the data acquisition window Average head pitch angle within, Indicates the data acquisition window The average facial center point coordinates within the area, Indicates the data acquisition window The average interocular distance within; Step 3, based on Establish thresholds for determining various fatigue driving states, including the threshold for determining the state of closed eyes. Threshold for determining yawning state Threshold for determining nodding state Threshold for determining abnormal body posture This allows for the construction of a fatigue judgment benchmark vector for fatigued driving conditions. and will Any one of the reference features is denoted as the criterion value for determining the j-th type of fatigue driving state. Among them, when hour, This represents the baseline value for determining the closed-eye state; when hour, This represents the baseline value for determining the yawning state; when hour, This represents the baseline value for determining the nodding state. hour, This represents the baseline value for judging abnormal body posture. Step 4: Construct the closed-eye state indicator for the t-th frame image using equation (6). Yawning status indicator Nodding status indicator Attitude abnormal status indicator This allows us to construct the driver fatigue state of the t-th frame image. Thus, the characteristic temporal state matrix is ​​obtained. T is the length of the time window; (6) Step 5, based on A driver fatigue risk function is constructed using the CRITIC weighting method. ; Step 6, if This indicates that the driver is in a safe driving state. This indicates that the driver is in a state of general fatigue. This indicates that the driver is in a highly fatigued and dangerous state. , This indicates two thresholds.

[0006] The personalized fatigue monitoring method for freight drivers described in this invention is characterized in that step 1 includes the following steps: Step 1.1: Use a facial landmark detection algorithm to extract the set of facial landmarks in the t-th frame of the driver's facial video stream. ;in, This represents the nth facial key feature point in the t-th frame image; N represents the total number of facial key feature points. Step 1.2, Extraction The driver's facial feature parameters include: the aspect ratio of the eyes in the t-th frame image. The aspect ratio of the mouth in the t-th frame image The head pitch angle of the t-th frame image The coordinates of the center point of the face in the t-th frame image The interpupillary distance of the t-th frame image This allows us to construct the facial feature independent variables for the t-th frame image. .

[0007] Furthermore, step 3 includes the following steps: Step 3.1, based on The interquartile range method was used to calculate the interquartile range of the j-th type of fatigue driving state. Therefore, the normalized deviation coefficient of the feature vector of the j-th type of fatigue driving state is calculated according to equation (1). ; (1) In equation (1), Let be the diffusion factor for the j-th type of fatigue driving state; Step 3.2: Calculate the threshold for determining the driver's closed-eye state according to formula (2). : (2) Step 3.3: Calculate the threshold for determining the driver's yawning state according to formula (3). : (3) Step 3.4: Calculate the threshold for determining the driver's nodding state according to formula (4). : (4) Step 3.5: Calculate the threshold for determining abnormal body posture of the driver according to formula (5). : (5).

[0008] Furthermore, step 5 includes the following steps: Step 5.1: Calculate the information content of the j-th type of fatigue driving state using equation (7). : (7) In equation (9), express The standard deviation of the j-th type of fatigue driving state in T-frame images. Let be the correlation coefficient between the j-th type of fatigue driving state and the k-th type of fatigue driving state; Step 5.2, using equation (8) to... After normalization, the weight coefficients of the j-th type of fatigue driving state are obtained. : (8) Step 5.3: Calculate the driver's personalized baseline fatigue risk function using equation (9). : (9) In equation (9), express Any j-th state indicator in the array.

[0009] The present invention provides an electronic device, including a memory and a processor, characterized in that the memory is used to store a program supporting the processor in performing the method described therein, and the processor is configured to execute the program stored in the memory.

[0010] The present invention provides a computer-readable storage medium on which a computer program is stored, characterized in that the computer program is executed by a processor to perform the steps of the method described thereon.

[0011] Compared with existing technologies, the beneficial technical effects of this invention are reflected in: 1. This invention takes into account the physiological and habitual differences among individual drivers, introduces the concept of individual baseline feature vector, and adaptively generates personalized fatigue judgment thresholds that conform to the current physiological characteristics of the driver, making the prediction accuracy of fatigued driving higher, providing a more scientific reference for future accident prevention and safety management of drivers, and helping managers to grasp the real-time safety status of vehicle operation.

[0012] 2. This invention takes into account the problem of feature data fluctuation caused by the driver's instantaneous non-fatigue behavior, and introduces a continuous analysis mechanism of frame validity determination and time window voting, which can effectively filter out false detections caused by algorithm instantaneous failure and single frame anomalies, thereby enhancing the robustness and anti-interference ability of the method.

[0013] 3. This invention takes into account the temporal variability of driver fatigue behavior and introduces a data-based dynamic evaluation model to accurately determine the risk level of the driver. This helps to overcome the limitations of fixed weights in adapting to changes in characteristics and achieves accurate quantification of driver fatigue status. Attached Figure Description

[0014] Figure 1 This is the overall flowchart of the present invention.

[0015] Figure 2 This is a flowchart of the driver fatigue state determination decision-making process of the present invention. Detailed Implementation

[0016] In this embodiment, a personalized method for monitoring driver fatigue in freight transport involves triggering driver behavior data, analyzing driver behavior data during the driving process, such as eye aspect ratio and mouth aspect ratio, collecting personalized parameters for the driver, establishing thresholds for various dangerous states, and ultimately displaying the driver's fatigue risk level to provide targeted warnings and ensure driver safety during driving. Specifically, for example... Figure 1 As shown, the method includes the following steps: Step 1: Real-time acquisition and processing of the driver's facial video stream inside the truck cab to obtain the set of key facial feature points in the t-th frame image. and its facial features independent variables .

[0017] Step 1.1: Use the face detection algorithm in the Dlib library to extract the set of feature points in the t-th frame of the driver's face video stream. ;in, This represents the nth facial key feature point in the t-th frame image; N represents the total number of facial key feature points. Step 1.2, Extraction The driver's facial feature parameters include: the aspect ratio of the eyes in the t-th frame image. The aspect ratio of the mouth in the t-th frame image The head pitch angle of the t-th frame image The coordinates of the center point of the face in the t-th frame image The interpupillary distance of the t-th frame image This allows us to construct the facial feature independent variables for the t-th frame image. .

[0018] Step 2: To prevent the driver's facial features from deviating from commonly accepted standards, during the initial stage after vehicle startup, the driver's natural facial features in a conscious state are captured, and the independent variables of facial features in each frame of the facial video stream are calculated within the capture window. Intra-basic eigenvectors ;in, Indicates the data acquisition window The average aspect ratio of the eyes within the eye, Indicates the data acquisition window The average aspect ratio of the mouth is within the mouth. Indicates the data acquisition window Average head pitch angle within, Indicates the data acquisition window The average facial center point coordinates within the area, Indicates the data acquisition window The average interocular distance within the eye.

[0019] Step 3, based on Establish thresholds for determining various fatigue driving states, including the threshold for determining the state of closed eyes. Threshold for determining yawning state Threshold for determining nodding state Threshold for determining abnormal body posture This allows for the construction of a fatigue judgment benchmark vector for fatigued driving conditions. and will Any one of the reference features is denoted as the criterion value for determining the j-th type of fatigue driving state. ;in, Let be the characteristic value of the j-th type of fatigue driving state, when hour, This represents the baseline value for determining the closed-eye state; when hour, This represents the baseline value for determining the yawning state; when hour, This represents the baseline value for determining the nodding state. hour, This represents the baseline value for judging abnormal body posture. Step 3.1, based on The interquartile range method was used to calculate the interquartile range of the j-th type of fatigue driving state. Therefore, the normalized deviation coefficient of the feature vector of the j-th type of fatigue driving state is calculated according to equation (1). ; (1) In equation (1), Let be the diffusion factor for the j-th type of fatigue driving state, whose value is obtained from statistical theory. In this example, take . .

[0020] Step 3.2: Calculate the threshold for determining the driver's closed-eye state according to formula (2). : (2) Step 3.3: Calculate the threshold for determining the driver's yawning state according to formula (3). : (3) Step 3.4: Calculate the threshold for determining the driver's nodding state according to formula (4). : (4) Step 3.5: Calculate the threshold for determining abnormal body posture of the driver according to formula (5). : (5) Step 4, as follows Figure 2 As shown, the closed-eye state indicator of the t-th frame image is constructed using equation (6). Yawning status indicator Nodding status indicator Attitude abnormal status indicator This allows us to construct the driver fatigue state of the t-th frame image. Determine whether the driver is in any fatigue state in frame t, thereby obtaining the feature temporal state matrix. T is the length of the time window; in this example, we take... .

[0021] (6) Step 5: Based on the multi-feature coupling enhancement model, construct the driver fatigue risk function using the CRITIC weighting method. ; Step 5.1: Calculate the information content of the j-th type of fatigue driving state using equation (7). : (7) In equation (9), express The standard deviation of the j-th type of fatigue driving state in T-frame images. Let be the correlation coefficient between the j-th type of fatigue driving state and the k-th type of fatigue driving state.

[0022] Step 5.3: To eliminate the impact of different dimensions of dangerous driving behavior, use formula (8) to... After normalization, the weight coefficients of the j-th type of fatigue driving state are obtained. : (8) Step 5.4: Calculate the driver's personalized baseline fatigue risk function using equation (9). : (9) In equation (9), express Any j-th state indicator in the array.

[0023] Step 6, as follows Figure 2 As shown, a tiered risk early warning mechanism is constructed based on a personalized baseline fatigue risk function. Determine the current driving status and divide it into three zones: safe zone, warning zone, and danger zone. This indicates that the driver is in a safe driving state. This indicates that the driver is in a state of general fatigue. A buzzer will vibrate intermittently at a slow rhythm, and a yellow indicator light will flash intermittently at a frequency of 1Hz. This indicates that the driver is in a highly fatigued and dangerous state. A buzzer will vibrate rapidly, and a red indicator light will flash rapidly at a frequency of 3Hz. , This represents the threshold.

[0024] In this embodiment, an electronic device includes a memory and a processor. The memory stores a program that supports the processor in executing the methods described above, and the processor is configured to execute the program stored in the memory.

[0025] In this embodiment, a computer-readable storage medium stores a computer program, which is executed by a processor to perform the steps of the above method.

[0026] In this example, a network driving video dataset is used as the research object, and a data acquisition window is set. Time window length Collect the individual baseline feature vector of the target driver, and calculate the fatigue judgment thresholds for closed eyes, nodding, yawning, and abnormal body posture using the interquartile range method. , , , The real-time risks calculated using equations (1) to (9) via the CRITIC weighting method are shown in Table 1. Table 1

[0027] Taking frame 1200 as an example, the specific feature indicators obtained within the sliding window are shown in Table 2: Table 2

[0028] Fatigue risk value of each driver is calculated based on a sliding time window. ,like If so, the driver is deemed to be in a safe driving state. If so, the driver is determined to be in a fatigue warning state. If the driver is in a high-risk driving state, for example, at frame 1200 of the driving video, the fatigue risk value is calculated based on the data within the sliding window. This is a safe driving condition.

Claims

1. A personalized method for monitoring fatigue in freight drivers, characterized in that, Includes the following steps: Step 1: Real-time acquisition and processing of the driver's facial video stream inside the truck cab to obtain the set of key facial feature points in the t-th frame image. and its facial features independent variables ; Step 2: Calculate the independent variables of facial features in each frame of the facial video stream within the acquisition window. Intra-basic eigenvectors ;in, Indicates the data acquisition window The average aspect ratio of the eyes within the eye area Indicates the data acquisition window The average aspect ratio of the mouth is within the mouth. Indicates the data acquisition window Average head pitch angle within, Indicates the data acquisition window The average facial center point coordinates within the area, Indicates the data acquisition window The average interocular distance within; Step 3, based on Establish thresholds for determining various fatigue driving states, including the threshold for determining the state of closed eyes. Threshold for determining yawning state Threshold for determining nodding state Threshold for determining abnormal body posture This allows for the construction of a fatigue judgment benchmark vector for fatigued driving conditions. and will Any one of the reference features is denoted as the criterion value for determining the j-th type of fatigue driving state. Among them, when hour, This represents the baseline value for determining the closed-eye state; when hour, This represents the baseline value for determining the yawning state; when hour, This represents the baseline value for determining the nodding state. hour, This represents the baseline value for judging abnormal body posture. Step 4: Construct the closed-eye state indicator for the t-th frame image using equation (6). Yawning status indicator Nodding status indicator Attitude abnormal status indicator This allows us to construct the driver fatigue state of the t-th frame image. Thus, the characteristic temporal state matrix is ​​obtained. T is the length of the time window; (6) Step 5, based on A driver fatigue risk function is constructed using the CRITIC weighting method. ; Step 6, if This indicates that the driver is in a safe driving state. This indicates that the driver is in a state of general fatigue. This indicates that the driver is in a highly fatigued and dangerous state. , This indicates two thresholds.

2. The personalized method for monitoring fatigue in freight drivers according to claim 1, characterized in that, Step 1 includes the following steps: Step 1.1: Use a facial landmark detection algorithm to extract the set of facial landmarks in the t-th frame of the driver's facial video stream. ;in, This represents the nth facial key feature point in the t-th frame image; N represents the total number of facial key feature points. Step 1.2, Extraction The driver's facial feature parameters include: the aspect ratio of the eyes in the t-th frame image. The aspect ratio of the mouth in the t-th frame image The head pitch angle of the t-th frame image The coordinates of the center point of the face in the t-th frame image The interpupillary distance of the t-th frame image This allows us to construct the facial feature independent variables for the t-th frame image. .

3. The personalized method for monitoring fatigue in freight drivers according to claim 2, characterized in that, Step 3 includes the following steps: Step 3.1, based on The interquartile range method was used to calculate the interquartile range of the j-th type of fatigue driving state. Therefore, the normalized deviation coefficient of the feature vector of the j-th type of fatigue driving state is calculated according to equation (1). ; (1) In equation (1), Let be the diffusion factor for the j-th type of fatigue driving state; Step 3.2: Calculate the threshold for determining the driver's closed-eye state according to formula (2). : (2) Step 3.3: Calculate the threshold for determining the driver's yawning state according to formula (3). : (3) Step 3.4: Calculate the threshold for determining the driver's nodding state according to formula (4). : (4) Step 3.5: Calculate the threshold for determining abnormal body posture of the driver according to formula (5). : (5)。 4. The personalized fatigue monitoring method for freight drivers according to claim 3, characterized in that, Step 5 includes the following steps: Step 5.1: Calculate the information content of the j-th type of fatigue driving state using equation (7). : (7) In equation (9), express The standard deviation of the j-th type of fatigue driving state in T-frame images. Let be the correlation coefficient between the j-th type of fatigue driving state and the k-th type of fatigue driving state; Step 5.2, using equation (8) to... After normalization, the weight coefficients of the j-th type of fatigue driving state are obtained. : (8) Step 5.3: Calculate the driver's personalized baseline fatigue risk function using equation (9). : (9) In equation (9), express Any j-th state indicator in the array.

5. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store a program that supports a processor in executing the method of any one of claims 1-4, the processor being configured to execute the program stored in the memory.

6. A computer-readable storage medium storing a computer program thereon, characterized in that, The computer program is executed by the processor to perform the steps of the method according to any one of claims 1-4.

Images