A Driver Fatigue Detection Method Based on Two-Stage Detection and Eye Feature Analysis

Abstract

This invention relates to the field of fatigue driving detection technology and discloses a driver fatigue driving detection method based on two-stage detection and eye feature analysis. The method includes: acquiring a real-time video stream containing the driver's face; performing face and key point detection, accurately locating the face bounding box and extracting three-dimensional face key points; performing posture-adaptive eye feature analysis, calculating and comparing the aspect ratio of both eyes to determine the direction of face deflection and selecting the eye with the clearest features as the optimal analysis object for single-eye image cropping; inputting the cropped single-eye image into an image classification model to determine the eye opening and closing state; and finally, cumulatively calculating fatigue metrics such as the percentage of eyelid closure and blinking frequency per unit time, comparing them with preset thresholds to determine the fatigue state and output a warning. This invention overcomes interference caused by driver head deflection or partial occlusion by adaptively selecting the optimal single-eye analysis, improving the robustness and accuracy of fatigue detection in complex real-world driving scenarios.

Description

Technical Field

[0001] This invention relates to the field of fatigue driving detection technology, specifically a driver fatigue driving detection method based on dual-stage detection and eye feature analysis. Background Technology

[0002] Currently, rail transit is a crucial artery for the transportation of people and goods, and its operational safety is of paramount importance. Rail transit drivers, working in a monotonous and high-pressure environment, are prone to fatigue driving due to visual fatigue and mental stress, posing a safety hazard. Therefore, developing efficient driver fatigue detection technology has become a necessary means to ensure transportation safety.

[0003] To address the aforementioned safety requirements, existing fatigue detection systems typically acquire facial videos of the driver using image acquisition devices. The workflow generally involves first locating the facial region in the image using a face detection algorithm. Then, an eye keypoint detection model is run within this region to extract coordinate data of feature points such as the upper and lower eyelids and corners of the eyes. Based on these coordinates, quantitative indicators such as eye aspect ratio (EAR) or eyelid closure (PERCLOS) are calculated. Finally, the system compares the real-time calculated indicator values ​​with a fixed threshold. When the indicator falls below the threshold for a certain period, the system determines that the driver has entered a state of fatigue.

[0004] However, existing technologies have limitations. First, conventional face detection models suffer from decreased localization accuracy when handling non-frontal poses such as sideways or tilted heads, often resulting in offset or even missed face bounding boxes. This leads to a loss of accurate initial ranges for subsequent keypoint extraction. Second, existing methods generally lack adaptability to pose changes. When a driver's head turns, the image of one eye may be compressed due to perspective or obscured by the bridge of the nose, yet the algorithm still indiscriminately uses the distortion data from both eyes for calculation, introducing judgment errors. Finally, relying on a single fixed threshold for judgment is too rigid. This threshold is difficult to universally apply to the differences in physiological characteristics of all individuals, nor can it adapt to the impact of different lighting environments on eye images. This results in frequent false alarms and false negatives in practical applications, leading to insufficient reliability.

[0005] Therefore, this invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis to address the shortcomings of existing technologies. Summary of the Invention

[0006] To address the shortcomings of existing technologies, this invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis. This method solves the problems of low detection accuracy and frequent false alarms and missed alarms in existing fatigue detection technologies when the driver is not in a frontal face posture, due to eye feature distortion, occlusion, and limitations of single threshold judgment.

[0007] To achieve the above objectives, the present invention provides the following technical solution: A driver fatigue detection method based on two-stage detection and eye feature analysis includes the following steps: Perform image data acquisition to obtain a real-time video stream containing the driver's face, the real-time video stream consisting of a sequence of consecutively arranged single-frame images; Perform face and key point detection, process the single frame images in the single frame image sequence to locate the face bounding box, and use the facial image region within the face bounding box as input to extract the three-dimensional face key points; Perform posture-adaptive eye feature analysis and extraction, calculate the aspect ratio of both eyes using the three-dimensional facial key points, compare the aspect ratios of both eyes to select the best analysis object for single-eye image cropping, and obtain single-eye cropped images. The system performs eye condition classification and fatigue comprehensive determination. The single-eye cropped image is input into the image classification model for feature classification. The fatigue measurement index is accumulated and compared with the preset fatigue threshold. The system determines that the driver is in a fatigued state and outputs a warning signal to trigger the alarm device.

[0008] By adopting the above technical solution, the system employs a two-stage detection mechanism that combines face boundary localization with 3D facial key point extraction. After key point detection, a horizontal comparison and selection logic based on the aspect ratio of both eyes is introduced. Therefore, in complex environments where the driver's head deflects at multiple angles or there is unilateral occlusion, the system can adaptively select the single-eye image with the most complete feature exposure for independent subsequent neural network classification calculation. This avoids the judgment interference caused by the distortion of features in both eyes in traditional methods and improves the overall robustness of the fatigue detection mechanism.

[0009] Preferably, the specific steps for performing face and keypoint detection are as follows: input the pixel matrix of a single-frame image into the backbone feature extraction network of the face detection model, extract and enhance the feature response values ​​of facial contour edges and facial feature edges to generate a multi-scale feature map set; perform region feature matching calculation on the generated multi-scale feature maps, and filter out candidate regions with an intersection-union ratio greater than a preset intersection-union ratio threshold; further perform bounding box regression calculation and classification confidence determination on the selected candidate regions, and apply the non-maximum suppression algorithm to remove redundant boxes and locate the final face bounding box; extract the local face image matrix based on the final face bounding box, adjust it to a preset fixed pixel size, and input it into the face keypoint detection model to extract the spatial topological structure feature data of facial features, and output a set of facial keypoints consisting of 468 three-dimensional spatial points as three-dimensional face keypoints.

[0010] By adopting the above technical solution, multi-scale feature maps are used in combination with cross-union ratio (CUI) thresholding to accurately locate the global facial region. Then, within this constraint, a dedicated keypoint network is called to extract 468 high-density three-dimensional spatial coordinates, which enhances the accuracy of facial feature topology localization.

[0011] Preferably, the specific steps for calculating the aspect ratio of both eyes and selecting the best analysis object are as follows: access the eye key point index mapping table, extract six feature points corresponding to the left eye boundary contour and six feature points corresponding to the right eye boundary contour from the three-dimensional face key points; obtain the coordinates of the above feature points on the two-dimensional image plane, calculate the Euclidean distance between each pair of key points among the six feature points of the corresponding eye boundary contour, and calculate the aspect ratio of the left eye and the right eye respectively; input the aspect ratio values ​​of the left eye and the right eye to the numerical comparator module to trigger the comparison operation process; set the comparison logic rules, determine the eye with the smaller aspect ratio value as the best analysis object, and execute the best analysis eye index calculation to determine the final retained eye data.

[0012] By adopting the above technical solution, scalar values ​​of eye opening and closing on both sides are constructed based on Euclidean distance calculation. Comparison rules are set based on the projection deformation law caused by head deflection. Eye data with severe distortion caused by posture is directly separated through logical comparison, thereby reducing the error of subsequent state classification.

[0013] Preferably, the specific steps for obtaining a single-eye cropped image by single-eye image cropping are as follows: based on the selected eye index result generated by the optimal analysis eye index calculation, retrieve the two-dimensional pixel coordinate data of the six feature points of the corresponding side eye; obtain the minimum and maximum coordinate values ​​in the coordinate data to form an initial boundary rectangle; perform region expansion calculation on the basis of the initial boundary rectangle to generate the final eye cropping boundary coordinate combination to define the eye cropping box; extract local images from the single-frame image matrix according to the defined pixel range and encapsulate them into a single-eye image matrix as the single-eye cropped image.

[0014] By adopting the above technical solution, the bounding box of a single eye is established by using extreme coordinates and an expansion mechanism, and redundant facial and background pixel matrix information is removed. While preserving the complete peri-eye texture features, the image resolution entering the classification model is effectively compressed, thereby improving the forward computation efficiency of the network.

[0015] Preferably, the specific steps for inputting a single-eye cropped image into an image classification model for feature classification are as follows: the single-eye cropped image containing pixel feature information of the eye region is converted into a multi-dimensional tensor data structure, input into the image classification model for forward propagation calculation, and deep high-dimensional features related to eyelid closure are extracted layer by layer, and the log probability values ​​corresponding to different classification states are output; at the output end, a normalized exponential function operation is introduced to perform exponential operation on the log probability values ​​of each category, and the probability values ​​of the single-eye image belonging to the closed-eye state category and the open-eye state category are calculated.

[0016] By adopting the above technical solution, the feature classification network is called to extract eyelid state representation parameters with high abstraction ability, and then the continuous values ​​are mapped to discrete probability values ​​of mutually exclusive classification through normalization exponent processing, so that the eye state detection results have mathematical determinism.

[0017] Preferably, the specific steps for cumulatively calculating fatigue measurement indicators are as follows: The instantaneous binarized eye state is determined based on the probability value of the state category. When the final classification is a closed eye state, the state logic value is recorded as a constant 1; when the eye state is an open eye state, the state logic value is recorded as a constant 0. A summation algebra operation is performed on all instantaneous state values ​​recorded as constant 1 within the time window to accumulate the total number of closed eye frames. This summation is then divided by the total number of image frames within the time window to obtain the percentage of eyelid closure per unit time. The instantaneous eye state change sequence within the time window is scanned, and the total number of complete blink events is counted and divided by the time interval to generate a blink frequency value, which together constitute the fatigue measurement indicators.

[0018] By adopting the above technical solution, the network classification results are binarized by setting constant 1 and constant 0, and the single-frame signal spikes caused by occasional missed detections or false detections are eliminated by combining the time window accumulation operation mechanism, thus completing the fatigue feature denoising and extraction from instantaneous image judgment to the time-series dimension.

[0019] Preferably, the specific steps for comparing with a preset fatigue threshold and outputting a warning signal to trigger the alarm device are as follows: The absolute duration of the continuous state with a logic value of constant 1 within the accumulated time window is used to obtain the numerical variable of the long eye-closing duration; the standard upper limit threshold of eyelid closure percentage, the standard upper and lower limit thresholds of blinking frequency, and the standard alarm threshold for continuous long eye-closing duration are used as preset fatigue thresholds; a multi-channel concurrent numerical comparison process is initiated, and the corresponding indicators generated in real time are compared with the standard preset thresholds respectively; when any value reaches or exceeds the standard preset threshold, the internal fatigue state judgment register flag is changed, and a hardware-level control command to trigger an audible and visual alarm is generated to activate the buzzer array and flashing warning light module.

[0020] By adopting the above technical solution, multi-channel concurrent numerical matching logic is used to conduct comprehensive cross-verification of dimensions such as eye-closing frequency, duration ratio, and extremely long abnormal closure, and directly connects to the underlying hardware status register to ensure that a strong warning level physical signal is immediately executed when any fatigue signs exceeding the safety limit are detected.

[0021] This invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis. It has the following beneficial effects: 1. This invention achieves high-precision face region localization by calculating the gradient change matrix of image pixels and enhancing the feature response values ​​of facial contours and facial feature edges during the face region localization stage, combined with cross-union ratio (CUI) thresholding on multi-scale feature maps. For non-frontal posture scenarios commonly encountered by drivers in actual driving, such as tilting their heads to the side, looking up, or looking down, this invention effectively avoids the problem of missing face bounding boxes or localization offsets caused by facial rotation, providing a stable and complete initial calculation area for subsequent 3D keypoint extraction.

[0022] 2. After extracting the key points of a 3D face, this invention introduces a strategy of calculating and comparing the aspect ratios of both eyes to determine the optimal analysis target. It utilizes the objective condition that the lateral projection distance between the two eyes changes asymmetrically due to perspective during facial rotation. By actively eliminating eyes whose features are distorted or obscured due to rotation through comparative logic, it accurately selects the single eye with the clearest exposed features for local cropping, overcoming the calculation bias caused by the blind selection of key point data from both eyes in traditional detection methods.

[0023] 3. This invention inputs the selected monocular cropped images into an image classification model for feature classification. By extracting deep, high-dimensional features related to the eyelids and calculating probability values, it replaces the traditional fixed numerical threshold judgment method. On the one hand, cropping local monocular regions effectively reduces the image resolution of the input model, lowering the system's computational overhead to adapt to edge computing devices. On the other hand, by utilizing the state probability output by the classification network, it eliminates the shortcomings of a single hard judgment standard in adapting to different driver physiological differences, varying lighting conditions, and complex background interference, thus improving the accuracy of eye state classification. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the driver fatigue detection system based on dual-stage detection and eye feature analysis of the present invention. Figure 2 The flowchart shows the driver fatigue detection method based on dual-stage detection and eye feature analysis of the present invention. Figure 3 This is a schematic diagram of the key facial features in the MediaPipe of the present invention; Figure 4This is a diagram showing the numbering of the eye parts in this invention; Figure 5 This is a graph showing the change in the aspect ratio of the eye over time according to the present invention. Figure 6 This is a curve comparing the accuracy of eye-closed detection under different facial deflection angles according to the present invention. Detailed Implementation

[0025] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] See attached document Figure 1 This invention provides a driver fatigue detection system based on dual-stage detection and eye feature analysis, which may include an image acquisition module, a dual-stage detection module, an eye feature analysis module, and a fatigue state determination module. This driver fatigue detection system based on dual-stage detection and eye feature analysis is applied in scenarios such as rail transit driver's cabs, car driver's cabins, and industrial control consoles to perform real-time monitoring of the operator's fatigue state.

[0027] The image acquisition module is used to acquire video streams or image sequences containing the driver's face in real time. In rail transit applications, the image acquisition device in the image acquisition module is installed at the driver's side-front position, so that the acquired video stream or image sequence contains facial images of the driver in non-frontal postures such as lateral deflection, head tilting up, and head tilting down. The image acquisition module transmits the real-time acquired video stream or image sequence to the two-stage detection module for subsequent processing.

[0028] The two-stage detection module is used to detect face regions and extract facial key points from acquired images. It utilizes the RetinaFace face detection model to process single-frame images from the image sequence transmitted by the image acquisition module, locating a complete bounding box containing the side-facing face, providing an initial range for eye feature extraction. After obtaining the bounding box, the two-stage detection module uses the MediaPipe facial key point detection model to output 3D facial key points. The number of 468 3D facial key points covers multiple feature points in the eye region. The two-stage detection module sends the coordinate data of the 468 extracted 3D facial key points, along with the single-frame image data, to the eye feature analysis module.

[0029] The eye feature analysis module analyzes the driver's head posture based on facial key points and adaptively selects the target eye region for cropping. From the received 468 3D facial key points, the module extracts six left-eye key points and six right-eye key points. It calculates the left-eye EAR value based on the coordinates of the left-eye key points and the right-eye EAR value based on the coordinates of the right-eye key points. The module compares the left-eye and right-eye EAR values. When the face is turned laterally, the lateral distance of one eye is lengthened, while the lateral distance of the other eye is shortened. The EAR value corresponding to the eye with the lengthened lateral distance is less than that corresponding to the eye with the shortened lateral distance. The module selects the eye with the smaller EAR value as the target analysis object to obtain an unobstructed eye image. Based on the key point coordinates corresponding to the target analysis object, the module crops a single-eye cropped image containing single-eye information from the received single-frame image data and transmits the single-eye cropped image to the fatigue state determination module.

[0030] The fatigue state determination module classifies the received monocular cropped images and combines this with temporal information to comprehensively determine the driver's fatigue state. The fatigue state determination module receives monocular cropped images transmitted from the eye feature analysis module and inputs these images into a pre-trained ResNet image classification model. The ResNet image classification model is trained on an eye dataset covering forward, left-leaning, right-leaning, tilting-up, and tilting-down angles. The ResNet image classification model performs feature classification on the input monocular cropped images and outputs corresponding open-eye or closed-eye state labels for each image. The specific process of pre-training the ResNet image classification model includes: constructing a multi-pose eye image dataset, which covers facial features of the target person from multiple angles such as forward, left, right, tilted head, and tilted head, and also includes test samples with different ambient lighting conditions and individual differences; extracting local eye regions from the collected facial features and labeling them with open and closed eye states; dividing the labeled dataset into training and validation sets and inputting them into the initial ResNet network, and updating the network weight parameters through iterative forward and backward propagation; stopping the iteration when the classification error of the network model on the validation set reaches the preset convergence condition, and generating and saving the pre-trained ResNet image classification model. The fatigue state determination module calculates fatigue metrics based on the open and closed eye state labels output within a continuous time window. Fatigue metrics include eye-closed duration and blinking frequency. The fatigue state determination module compares the calculated duration of eye closure and blinking frequency with a preset fatigue threshold. When the duration of eye closure or blinking frequency exceeds the preset fatigue threshold, the fatigue state determination module determines that the driver is in a fatigued state and outputs a warning signal to trigger the alarm device.

[0031] Please see Figure 2 This invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis. This method can be executed by the driver fatigue detection system based on dual-stage detection and eye feature analysis described above. By fusing image data, this method achieves real-time and accurate detection and early warning of driver fatigue. Specifically, it includes the following steps: Step S100: Perform image data acquisition. Start the image acquisition module to continuously acquire a real-time video stream containing the driver's face at a preset frame rate, such as 30 frames per second. This video stream consists of a series of single-frame images arranged in chronological order. Since the image acquisition device is installed to the driver's side and in front, the acquired single-frame images will contain facial information of the driver in various non-frontal postures, such as facing forward, turning sideways, tilting the head up, and tilting the head down.

[0032] Step S200: Perform face and keypoint detection. Each single-frame image acquired in step S100 is processed. First, the single-frame image is input to the RetinaFace face detection model. This model analyzes the image and outputs the coordinates of one or more face bounding boxes, which precisely delineate the image region containing the driver's face. Then, the facial image region within this bounding box is used as input to the MediaPipe face keypoint detection model. This model outputs the coordinates of 468 three-dimensional face keypoints covering the facial contours, eyes, eyebrows, nose, and lips.

[0033] Step S300: Perform pose-adaptive eye feature analysis and extraction. Using the coordinates of 468 3D facial key points output in step S200, calculate the aspect ratio of both eyes to determine head pose, and crop the target monocular image for analysis. This step first calculates the aspect ratio of the left and right eyes separately, then selects the monocular with less morphological distortion and clearer image due to head rotation by comparing the aspect ratios of both eyes. Finally, based on the key point coordinates of the selected monocular, crop the image region of that monocular from the corresponding single-frame image.

[0034] Step S400 involves performing eye state classification and fatigue comprehensive determination. The target monocular image cropped in step S300 is input into a pre-trained ResNet image classification model to determine the open or closed state of the monocular image. This method performs cumulative analysis on the state of continuous monocular images over time, calculating fatigue metrics such as the percentage of eyelid closure, blink frequency, and longest duration of eye closure per unit time. The calculated fatigue metrics are compared in real time with preset fatigue thresholds. When any metric value exceeds its corresponding threshold, the driver is determined to be in a fatigued state, and a warning signal is generated.

[0035] This invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis, which may include: Step S100: Perform image data acquisition. Start the image acquisition module and establish a data transmission connection with the image acquisition device. The image acquisition device is installed in the driver's seat inside the cab, at the side-front position. The image acquisition module acquires real-time video stream data containing the rail transit driver's face through this connection. The real-time video stream data consists of a sequence of single-frame images arranged sequentially in time.

[0036] The image acquisition module continuously reads single-frame images at a set fixed frame rate, such as 30fps. Because the image acquisition device is located at a side-front position, the acquired single-frame image sequence covers multi-angle visual feature information of the driver's face under different working conditions. The facial posture recorded in a single frame image includes frontal facial features, as well as non-frontal facial features such as turning left, turning right, tilting the head up, and tilting the head down. The single-frame image data contains the pixel distribution of the facial region under different ambient lighting conditions.

[0037] The acquired single-frame image sequence serves as the basic data input source for the detection process. The image acquisition module transmits the continuously generated single-frame image data to the corresponding embedded edge computing device for buffering and storage. The buffered single-frame image data is then used sequentially as the analysis object of the dual-stage detection module, for the face detection model and facial key point detection model in subsequent processes to perform calculations. The image data acquisition operation is continuously performed during the operation of the detection system, providing an unfiltered raw image sequence for continuous determination of fatigue state.

[0038] This invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis, which may include: Execute step S200 to perform face and key point detection.

[0039] Sub-step S201: Perform face region localization. Receive the single-frame image data obtained in step S100. Start the face detection program in the two-stage detection module. Call the RetinaFace face detection model. Input the pixel matrix of the single-frame image to the backbone feature extraction network of the RetinaFace face detection model. This backbone feature extraction network performs continuous convolution operations on the input pixel matrix to obtain the basic feature representation of the image.

[0040] For non-frontal poses such as lateral turning, tilting the head up, and tilting the head down, the RetinaFace face detection model performs edge feature extraction. By calculating the gradient transformation matrix of image pixels, it extracts and enhances the feature response values ​​of facial contour edges and facial feature edges. Based on the enhanced feature response values, it generates a multi-scale feature map set of different resolutions to adapt to changes in face scale at different distances and tilt angles.

[0041] Region feature matching calculations are performed on the generated multi-scale feature maps. Intersection over Union (IoU) ratios are calculated using preset baseline bounding boxes with different aspect ratios and high-response regions in the feature maps. Numerical criteria for the IoU are set, and candidate regions with IoU ratios greater than a preset threshold are selected. These candidate regions are used as input for subsequent processes to determine the initial location of the face.

[0042] The selected candidate regions are further processed using bounding box regression calculations and classification confidence assessments. The regression algorithm is used to adjust the boundary coordinates of the candidate regions to better match the true boundaries of the face. Then, a non-maximum suppression algorithm is applied to process bounding boxes with overlapping intersections, eliminating redundant boxes. The system outputs the final face bounding box with the highest classification confidence and definitive boundary coordinates.

[0043] The face bounding box is defined by the two-dimensional pixel coordinates of its four vertices. The two-dimensional pixel coordinate data of this face bounding box is recorded and output. Based on the coordinate data of this face bounding box, a complete image region containing a non-frontal face is extracted from the original single-frame image. This extracted complete image region serves as the initial input range for subsequent facial keypoint extraction. If no face bounding box is output at this stage, the system control flow returns to step S100 to obtain the next frame image.

[0044] Sub-step S202: Perform facial key point extraction. Receive the face bounding box coordinate data and the corresponding single-frame image region data output from sub-step S201. Based on the face bounding box coordinates, extract a local face image matrix from the single-frame image. Adjust the extracted local face image matrix to a preset fixed pixel size using a bilinear interpolation algorithm. The fixed-size pixel matrix meets the input dimension requirements of the subsequent key point extraction model.

[0045] The resized local face image matrix is ​​input into the MediaPipe facial landmark detection model. The MediaPipe facial landmark detection model contains a deep convolutional neural network structure. This network layer performs layer-by-layer convolution and pooling operations on the input local face image matrix. Through computation, spatial topological feature data of facial features are extracted from the local face image matrix.

[0046] The MediaPipe facial landmark detection model, after forward propagation calculations, outputs a set of 468 3D spatial points for facial landmark detection. (See attached image.) Figure 3 These 468 three-dimensional spatial points are arranged according to a pre-defined fixed topological index. Each index number uniquely corresponds to and maps to the specific anatomical location of the facial contours, eyes, eyebrows, nose, and mouth.

[0047] Each 3D keypoint in the facial keypoint set contains horizontal 2D pixel coordinates, vertical 2D pixel coordinates, and relative depth coordinates. During head rotation, these coordinates accurately reflect the spatial relative positions of various facial organs in a lateral posture. These 468 3D spatial points include multiple keypoints defining the eyelid boundaries and corners of the eyes, forming a feature point set covering the eye region.

[0048] The system stores the 3D coordinates of 468 facial key points and their corresponding topological index numbers in a memory register. After verifying the integrity of the coordinate data, the extracted facial key point coordinate dataset is transferred from the two-stage detection module to the eye feature analysis module. The transferred facial key point coordinate dataset serves as the basic input data for triggering the subsequent pose-adaptive eye feature analysis and extraction process.

[0049] See attached document Figure 4 The present invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis, which may include: Execute step S300 to perform pose-adaptive eye feature analysis and extraction.

[0050] Execute sub-step S301 to calculate the aspect ratio of both eyes. Receive the facial keypoint set data containing 468 three-dimensional keypoints output from the previous step. Since the 468 keypoints provided by the previous step cover all dimensional feature points of the entire face, the system needs to filter out the core point set specifically for eye morphology analysis.

[0051] The system accesses a pre-defined eye keypoint index mapping table stored in memory. Based on this mapping table, it extracts six feature points corresponding to the left eye's boundary contour from 468 keypoints. Similarly, the system extracts six feature points corresponding to the right eye's boundary contour based on the same mapping table. These twelve feature points will serve as the basic data input for subsequent calculations of the eyelid opening and closing degree.

[0052] For the six feature points of the left eye, the system obtains their horizontal and vertical coordinates on the two-dimensional image plane. The system then sorts and numbers the six feature points of the left eye according to a preset topological order. These six feature points accurately depict the relative positional relationship of the inner and outer corners of the left eye, as well as the upper and lower eyelids.

[0053] For the six feature points of the right eye, the system performs the same coordinate acquisition and sorting operations. The six feature points of the right eye are processed sequentially and mapped onto a two-dimensional image plane. These acquired two-dimensional coordinate data are fed into the processing unit for quantization calculation of the geometric morphological parameters of the eye.

[0054] The system's processing unit first calculates the Euclidean distance between each pair of key points among the six feature points corresponding to the left eye's boundary contour. The Euclidean distance is calculated based on the following formula: ; in, Indicates the first One key point, Indicates the first One key point, Indicates the first Two-dimensional pixel horizontal coordinates of key points Indicates the first Two-dimensional pixel vertical coordinates of key points Indicates the first Two-dimensional pixel horizontal coordinates of key points Indicates the first Two-dimensional pixel vertical coordinates of key points For the first The key point and the first The Euclidean distance between two key points.

[0055] Based on the calculated Euclidean distance between the two points, the system further calculates the aspect ratio of the left eye. The formula for calculating the aspect ratio of the left eye is as follows: ; in, This indicates the first key point for locating the left eye. This indicates the second key point for locating the left eye. This indicates the third key point for locating the left eye. This indicates the fourth key point for locating the left eye. This indicates the fifth key point for locating the left eye. This indicates the sixth key point for locating the left eye. The aspect ratio of the left eye.

[0056] Subsequently, the system uses the same computational logic to process the key point data of the right eye. The formula for calculating the aspect ratio of the right eye is as follows: ; in, This indicates the first key point for locating the right eye. This indicates the second key point for locating the right eye. This indicates the third key point for locating the right eye. This indicates the fourth key point for locating the right eye. This indicates the fifth key point for locating the right eye. This indicates the sixth key point for locating the right eye. The aspect ratio of the right eye is given. The calculated aspect ratios of the left and right eyes are simultaneously stored in a register.

[0057] Execute sub-step S302 to select the optimal analysis eye based on the eye aspect ratio. The system reads the left and right eye aspect ratio values ​​calculated in sub-step S301. The system inputs these two values ​​into the numerical comparator module, triggering the comparison operation process.

[0058] In real-world image acquisition scenarios, the faces of individuals are often in non-frontal postures, such as being turned to the side, looking up, or looking down. When an individual's head is turned to one side, due to optical perspective, the horizontal width of the eye facing the image acquisition device on the two-dimensional image plane will be elongated. Simultaneously, the horizontal width of the eye on the other side, which is facing away from the image acquisition device, will be correspondingly shortened on the two-dimensional image plane.

[0059] Because the denominator in the formula for calculating the aspect ratio of the eye includes the horizontal width of the corner of the eye, an increase in the horizontal width will reduce the calculated aspect ratio. Therefore, eyes that are in a perspective elongation state will have a relatively small aspect ratio. These eyes appear as larger pixels in 2D images, retain more image features, and are less likely to be obscured by other facial features.

[0060] The system directly determines the direction of head rotation of a target person by comparing the aspect ratios of the left and right eyes. The system uses a set comparison logic rule to identify the eye with the smaller aspect ratio as the best object for analysis in the current frame. This selection strategy adapts to complex facial poses and avoids using distorted or occluded eye areas for analysis.

[0061] The system performs optimal analysis eye index calculation to determine the final eye data to be retained. The formula for this calculation is as follows: ; in, The aspect ratio of the left eye. The aspect ratio of the right eye. For the selected eye index, The function returns the index of the variable that minimizes its argument value. The system then outputs this selected eye index as a control signal to the monocular image cropping process.

[0062] Execute sub-step S303, monocular image cropping. The system receives the selected eye index signal output from sub-step S302. The system parses the signal and, based on the index result, retrieves the two-dimensional pixel coordinate data of the six feature points of the corresponding side eye on the original single-frame image from the buffer.

[0063] The system iterates through the horizontal coordinate data of the six retrieved feature points and obtains the minimum and maximum horizontal coordinate values ​​through extreme value comparison. Similarly, the system iterates through the vertical coordinate data of these six feature points and obtains the minimum and maximum vertical coordinate values. These four extreme coordinates form an initial bounding rectangle that closely follows the corresponding eye contour.

[0064] To ensure that the subsequently extracted image region contains sufficient periorbital texture features and boundary transition features, the system performs region expansion calculations outward based on the initial bounding rectangle. The system calls the preset horizontal and vertical pixel fill margin values ​​in the parameter configuration table.

[0065] The system subtracts the horizontal pixel padding margin from the minimum horizontal coordinate value and adds the horizontal pixel padding margin to the maximum horizontal coordinate value. Similarly, the system subtracts the vertical pixel padding margin from the minimum vertical coordinate value and adds the vertical pixel padding margin to the maximum vertical coordinate value. After these expansion calculations, the system generates the final combination of eye cropping boundary coordinates.

[0066] The system uses the generated coordinate combinations to define the image cropping region, and the calculation formula is as follows: ; in, The minimum horizontal coordinate of the keypoint coordinates of the selected eye. The minimum vertical coordinate of the keypoint coordinates of the selected eye. The maximum horizontal coordinate of the keypoint coordinates of the selected eye. The maximum vertical coordinate of the keypoint coordinates of the selected eye. Fill the horizontal margins with pixels. Fill the margins for vertical pixels. Create a cropping frame for the eye area.

[0067] Based on the pixel range defined by the eye cropping box, the system extracts a local set of monocular image pixels from the original single-frame image matrix. The system then performs boundary alignment and format encapsulation on this monocular image pixel set. The encapsulated monocular image matrix is ​​transmitted to subsequent modules as the basic input source for determining the eye's open / closed state.

[0068] This invention provides a driver fatigue detection method based on dual-stage detection and eye feature analysis, which may include: Perform step S400: Classify eye condition and comprehensively determine fatigue.

[0069] Execute sub-step S401, eye state classification. After completing the monocular image cropping operation, the system obtains the corresponding local monocular image matrix. This monocular image matrix is ​​transmitted to the preset image classification module interface inside the computing unit. At this time, the local monocular image matrix contains complete pixel feature information of the eye region, constituting the basic input source for subsequent state determination.

[0070] The computation unit performs preprocessing operations on the received local monocular image matrix. This preprocessing includes adjusting the image pixel values ​​to the data range required by the deep learning model and standardizing the spatial resolution of the image matrix. After resizing and numerical normalization, the two-dimensional image pixel matrix is ​​converted into a multi-dimensional tensor data structure suitable for neural network computation.

[0071] Before performing forward propagation computation, the ResNet image classification model is trained offline. The training process is as follows: Facial image samples covering multiple poses (front, left, right, tilted, and tilted) and including multi-source lighting environments and individual differences are collected; pixel sets of the eye region in the samples are extracted to form a multi-pose eye dataset, and single-eye images in the dataset are labeled with open or closed eye status; the labeled dataset is input into the initial ResNet network structure for feature learning and iterative update of weight parameters until the classifier loss function converges, completing model training. The system inputs this multidimensional tensor data into the pre-trained ResNet image classification model. This classification model contains multiple cascaded residual operation blocks, convolutional layers, and global average pooling layers. The multidimensional tensor data undergoes forward propagation computation within the classification model, extracting deep, high-dimensional features related to eyelid closure in the image layer by layer. Finally, the fully connected layer at the end of the model outputs two log-probability values ​​corresponding to different classification states.

[0072] To convert the log-odds values ​​output by the fully connected layers into intuitive probability distribution data, the system introduces a normalized exponential function operation at the output of the ResNet image classification model. This operation performs an exponential operation on the log-odds value for each class, and then divides it by the sum of the exponential operations for all classes, thereby calculating the posterior probability value of the input image belonging to each target state class.

[0073] The formula for calculating the state classification probability is as follows: ; in, This represents the input image of a cropped portion of the eye. Indicates the first There are two preset categories: the first category corresponds to the closed-eye state, and the second category corresponds to the open-eye state. This indicates the total number of categories; in this case, the total number of categories is 2. This indicates that the ResNet image classification model targets categories. The generated log-odds output value, This indicates that the ResNet image classification model targets categories. The generated log-odds output value, This indicates that the local eye image belongs to the category. The posterior probability.

[0074] Based on the state classification probability calculation process, the system obtains the probability values ​​of whether the monocular image in the current video frame belongs to the closed-eye state category and the probability values ​​of whether it belongs to the open-eye state category. These two numerical probability parameters are synchronously stored in the memory buffer register, serving as the core input conditions for the next stage of calculating instantaneous eye features and continuous fatigue indicators.

[0075] Execute sub-step S402 to calculate fatigue measurement indicators. The system uses an internal clock to set a fixed analysis time window to accumulate and analyze continuous eye state sequences that change over time. The establishment of the time window allows the system to quantitatively assess the instantaneous characteristic changes of the target personnel during the work process from a continuous time dimension.

[0076] Based on the probability values ​​calculated in sub-step S401, the system determines the instantaneous binarized eye state at each time point within the analysis time window. The system compares the probability values ​​of the closed-eye state category and the open-eye state category, selecting the category label with the largest value as the deterministic state for the current time point. When the system determines the final classification as the closed-eye state, the corresponding state logic value is recorded as a constant 1; when the final classification as the open-eye state, the corresponding state logic value is recorded as a constant 0.

[0077] The formula for determining the instantaneous state of the eye is as follows: ; in, Indicates a specific time point in the captured video sequence. Indicates at a point in time Acquire and crop the generated monocular local image. Indicates the first Preset categories, Indicates at a point in time This image belongs to the category The posterior probability value, The operation represents retrieving the index of the category variable that maximizes the corresponding probability value. Indicates at a point in time Extracted binarized eye state logic values.

[0078] The system continuously iterates through and records the binary eye state logic value of each frame within the set analysis time window. The system performs a summation algebraic operation on all instantaneous state values ​​recorded as a constant 1 within this time window to obtain the total number of frames in which the target person is in a closed-eye state. The system further divides this total number of closed-eye frames by the total number of image frames included in the time window to obtain the percentage of eyelid closure per unit time.

[0079] The formula for calculating the percentage of eyelid closure per unit time is as follows: ; in, This indicates the start time recording point of the set analysis time window. This indicates the end time recording point of the analysis time window. Indicates a specific point in time. The binary eye state logic value, This indicates the total number of video frames acquired within the analysis time window. This represents the percentage of eyelid closure per unit time calculated within the analysis time window.

[0080] The system simultaneously scans the instantaneous eye state change sequence within the time window on the time axis, identifying the complete blinking physical action of the target person. When the system detects that the state logic value in a continuous frame sequence jumps from a constant 0 to a constant 1, and then returns to a constant 0 within a predetermined time threshold, the system records this continuous jump process as a single complete blink event. The system counts the total number of blink events occurring within the current time window and divides it by the duration interval of the time window to generate the corresponding blink frequency parameter.

[0081] The formula for calculating blink frequency is as follows: ; in, This indicates the start time recording point of the set analysis time window. This indicates the end time recording point of the analysis time window. This represents the total number of complete blink events detected and recorded by the system within this time window. This represents the blink frequency values ​​statistically obtained within a time period. After all measurement parameters are calculated, they are encapsulated and transmitted to the comprehensive judgment module.

[0082] Execute sub-step S403 for comprehensive judgment and early warning. The system receives the percentage of eyelid closure per unit time and the blink frequency value calculated in sub-step S402. While reading these parameters, the system accumulates the absolute duration of the continuous state with a logical value of constant 1 within the time window to obtain the long eye closure duration value variable.

[0083] The system accesses its internally configured non-volatile security configuration storage area and reads multiple preset fatigue threshold parameters. These extracted parameters include standard upper and lower limits for eyelid closure percentage, standard upper and lower limits for blink frequency, and standard alarm thresholds for continuous prolonged eye closure duration. These preset thresholds together form the absolute baseline for the system to determine the target person's level of alertness.

[0084] The system's internal logic operation unit initiates a multi-channel concurrent numerical comparison process. The logic operation unit extracts the real-time calculated percentage of eyelid closure per unit time and compares it with the extracted upper threshold of the standard eyelid closure percentage. Simultaneously, the logic operation unit extracts the real-time calculated blink frequency and compares it with the extracted upper and lower thresholds of the standard blink frequency for range comparison.

[0085] The logic unit continues to compare the real-time accumulated value of the prolonged eye-closing duration with the preset standard alarm threshold for continuous prolonged eye-closing duration. The system synthesizes the Boolean results from these three independent comparison channels using a logic OR gate structure. When any of the above real-time calculated values ​​reaches or exceeds its corresponding preset standard threshold, the system immediately changes the flag bit of the internal fatigue state determination register.

[0086] When the fatigue state determination register flag is toggled and activated, the system's main control chip generates a hardware-level control command to trigger an audible and visual alarm based on the configuration rules. This control command is transmitted to the connected external electrical warning device interface via the system's internal bus. Upon receiving the command level, the external electrical warning device powers on and activates the buzzer array and flashing warning light module, forcibly outputting a high-decibel acoustic signal and an optical strobe signal.

[0087] Specific application examples: See attached document Figure 5 The present invention provides a specific application example of a driver fatigue detection method based on dual-stage detection and eye feature analysis, as follows: The system processes a 10-second video stream with a fixed capture frame rate of 30fps, containing a total of 300 frames. Figure 5 Taking the 100th frame image of the target person with their head turned to the right as an example, the system extracted six key points describing the contour of the left eye. to And six key points describing the outline of the right eye to The two-dimensional pixel coordinates. Assume the coordinates of the left eye's keypoint at this point make the vertical Euclidean distance between its upper and lower eyelids equal. =8, =8, horizontal distance between the inner and outer corners of the eyes =40. The system substitutes these values ​​into the formula to calculate the aspect ratio of the left eye. : =0.20.

[0088] Similarly, using the same calculation logic, let's assume the relevant distance value extracted to the right eye is... =10.5, )=10.5, =35. Then, the aspect ratio of the right eye was calculated. : =0.30.

[0089] The system compares and calculates =0.20 and After reaching 0.30, the optimal analytical eye index calculation formula is used. Determine the index Point to the left eye. The system then uses the extreme coordinates of the left eye's key points and sets the horizontal and vertical pixel fill margins. =10, =10, calculate the eye cropping frame. To perform image cropping.

[0090] Combination Figure 5 The graph is presented in a frame-by-frame manner. The horizontal axis represents the progress of the image sequence, ranging from 0 to 300 frames; the vertical axis represents the calculated aspect ratio of the eyes, limited to between 0.05 and 0.4. The solid line marked with black dots details the trajectory of the left eye's aspect ratio, while the dashed line marked with gray squares depicts the trajectory of the right eye's aspect ratio. For ease of observation, data markers are evenly distributed on the two curves at 20-frame intervals. Furthermore, to reflect the true characteristics of the sensor data, small-amplitude fluctuation noise is superimposed on both the solid and dashed lines.

[0091] like Figure 5 As shown, between frames 50 and 150, due to the driver's head turning significantly to the right, the solid line (left eye) value generally decreased to between 0.20 and 0.25, while the dashed line (right eye) value remained at a higher level of around 0.28 to 0.32. Within this range, the system continuously made judgments. Less than . Figure 5 The image specifically uses gray background areas to visually identify the frame intervals where the system selects the left eye as the analysis target (e.g., frames 0 to 15, 50 to 150, 250 to 270, and segments after frame 280). Conversely, white background areas in the image (e.g., most intervals between frames 150 and 200, and between 200 and 250) represent... Less than or equal to Within these uncolored intervals, the system adaptively switches to selecting the right eye as the analysis object.

[0092] also, Figure 5 The system fully recorded the driver's blinking during this process, which is represented by a sharp dip in the curve. Specifically, around frames 80 and 280, the solid line (left eye) value drops sharply to a low point close to 0.1 (precisely marked by a black dot in the graph), indicating that the system captured the eye closing action; around frame 180, the dashed line (right eye) also shows a sharp drop to a low point of approximately 0.11 (marked by a gray square). The system scans these states and logic values... Accurately accumulate jumps within the time window And calculate the blink frequency per unit time. Fatigue indicators, etc.

[0093] Experimental verification and effect comparison section: See attached document Figure 6 The system conducted a comparative experiment using a dataset containing 10,000 labeled real-state facial images. The control method used a fixed binocular average aspect ratio threshold for judgment, while the method of this invention uses the aforementioned pose-adaptive monocular selection and ResNet image classification process to calculate the posterior probability. .

[0094] like Figure 6 As shown, the horizontal axis of the graph represents the facial deflection angle, ranging from 0 degrees to 60 degrees in 10-degree increments; the vertical axis represents the percentage accuracy of the detection, ranging from 30 to 100. The thick solid line marked with a large black dot represents the accuracy of the testing process of this invention, while the thick dashed line marked with a gray square represents the accuracy of the control calculation process. Each discrete marker is clearly labeled with its specific percentage accuracy value.

[0095] Combination Figure 6 The numerical labels clearly show that, in a frontal pose with a deflection angle of 0 degrees, the two methods perform similarly, with the accuracy of the test process of this invention reaching 98.5%, while the accuracy of the control calculation process is 95.2%. However, as the deflection angle increases to 10 degrees, 20 degrees, and 30 degrees, the accuracy of this test process stabilizes at a high level of 94.8%, 93.5%, and 92.6%, respectively, while the accuracy of the control calculation process shows a significant decrease, dropping to 91.0%, 83.5%, and 74.2%, respectively.

[0096] When the facial deflection angle further increased to 40 degrees, 50 degrees, and 60 degrees, the geometric distortion caused by the large-angle profile led to a sharp drop in the accuracy of the comparison calculation process to 62.4%, 51.5%, and finally only 42.0% at 60 degrees. In contrast, because the present invention adopts real-time feature calculation and comparison and target eye adaptive cropping strategy, it effectively eliminates the interference of occlusion and perspective distortion. Its test accuracy remains at 91.8%, 89.5%, and 88.2% at 40 degrees, 50 degrees, and 60 degrees, respectively. Figure 6 The significant difference between the solid and dashed lines as the angle increases, along with the specific label data, fully demonstrates that the robustness of the method of this invention under complex non-frontal poses is significantly better than that of the traditional fixed threshold method.

Claims

1. A driver fatigue detection method based on dual-stage detection and eye feature analysis, characterized in that, Includes the following steps: Perform image data acquisition to obtain a real-time video stream containing the driver's face, the real-time video stream consisting of a sequence of consecutively arranged single-frame images; Perform face and key point detection, process the single frame images in the single frame image sequence to locate the face bounding box, and use the facial image region within the face bounding box as input to extract the three-dimensional face key points; Perform posture-adaptive eye feature analysis and extraction, calculate the aspect ratio of both eyes using the three-dimensional facial key points, compare the aspect ratios of both eyes to select the best analysis object for single-eye image cropping, and obtain single-eye cropped images. The system performs eye condition classification and fatigue comprehensive determination. The single-eye cropped image is input into the image classification model for feature classification. The fatigue measurement index is accumulated and compared with the preset fatigue threshold. The system determines that the driver is in a fatigued state and outputs a warning signal to trigger the alarm device.

2. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 1, characterized in that, The specific steps for performing face and key point detection are as follows: To perform face region localization, the pixel matrix of the single frame image in the single frame image sequence is input into the face detection model for processing, and the final face bounding box with the highest classification confidence and determined boundary position coordinates is output, thereby locating the face bounding box; Facial key point extraction is performed. Based on the final face bounding box, a local face image matrix is ​​extracted as the face image region. The face image region is used as input to the facial key point detection model. After forward propagation calculation, a set of facial key points composed of multiple three-dimensional spatial points is output. The set of facial key points is used as the extracted three-dimensional facial key points.

3. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 2, characterized in that, The specific steps for performing face region localization are as follows: The pixel matrix of the single frame image is input to the backbone feature extraction network of the face detection model. The backbone feature extraction network extracts and enhances the feature response values ​​of the facial contour edges and facial feature edges by calculating the gradient change matrix of the image pixels, and generates a multi-scale feature map set based on the enhanced feature response values. Perform region feature matching calculations on the generated multi-scale feature map to filter out candidate regions with an intersection-union ratio greater than a preset intersection-union ratio threshold; The selected candidate regions are further subjected to bounding box regression calculation and classification confidence determination. The non-maximum suppression algorithm is applied to process bounding boxes with overlapping intersections, and redundant boxes are removed. Finally, the final face bounding box with the highest classification confidence and determined boundary position coordinates is output.

4. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 2, characterized in that, The specific steps for performing facial key point extraction are as follows: Based on the final face bounding box coordinates, a local face image matrix is ​​extracted from a single frame image. The extracted local face image matrix is ​​then adjusted to a preset fixed pixel size using a bilinear interpolation algorithm to form the facial image region. The resized local face image matrix is ​​used as the face image region and input into the face key point detection model. The face key point detection model performs layer-by-layer convolution and pooling operations on the input local face image matrix to extract the spatial topological structure feature data of the facial features. The facial key point detection model performs forward propagation calculations and outputs a set of facial key points consisting of 468 three-dimensional spatial points, which are then used as the three-dimensional facial key points.

5. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 1, characterized in that, The specific steps for calculating the aspect ratio of both eyes are as follows: Access the eye key point index mapping table, and extract six feature points corresponding to the left eye boundary contour and six feature points corresponding to the right eye boundary contour from the extracted three-dimensional face key points according to the mapping table. For the six feature points corresponding to the left eye boundary contour, the horizontal and vertical coordinates of the six feature points corresponding to the left eye boundary contour on the two-dimensional image plane are obtained respectively. The Euclidean distance between any two key points of the six feature points corresponding to the left eye boundary contour is calculated. The aspect ratio of the left eye is calculated based on the calculated Euclidean distance between the two points. For the six feature points corresponding to the boundary contour of the right eye, the same calculation logic is used to process the key point data of the right eye to calculate the aspect ratio of the right eye. The aspect ratio of the left eye and the aspect ratio of the right eye together constitute the calculated aspect ratio of both eyes.

6. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 5, characterized in that, The specific steps for selecting the best analysis object by comparing the aspect ratio of both eyes are as follows: Read the calculated aspect ratio values ​​of the left and right eyes, input the aspect ratio values ​​of the left and right eyes into the numerical comparator module, and trigger the comparison calculation process. By comparing the aspect ratios of the left and right eyes through the aforementioned comparison calculation process, the direction of head deflection of the target person can be directly determined. By setting comparison logic rules, the eye with the smaller aspect ratio is identified as the best analysis object in the current frame image, and the best analysis eye index is calculated for the best analysis object to determine the final retained eye data.

7. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 6, characterized in that, The specific steps for performing monocular image cropping and obtaining a monocular cropped image are as follows: Based on the selected eye index result generated by the best analytical eye index calculation, the two-dimensional pixel coordinate data of the six feature points of the corresponding side eye on the original single frame image are retrieved from the cache. Iterate through the horizontal and vertical coordinate data of the six retrieved feature points, obtain the minimum and maximum horizontal coordinate values ​​in the horizontal coordinate data and the minimum and maximum vertical coordinate values ​​in the vertical coordinate data, and construct the initial boundary rectangle. Based on the initial boundary rectangle, perform outward region expansion calculation to generate the final eye cropping boundary coordinate combination, define the eye cropping box, and complete the monocular image cropping process; Based on the pixel range defined by the eye cropping box, a local monocular image pixel set is extracted from the original single-frame image matrix. The local monocular image pixel set is then formatted and encapsulated into a monocular image matrix, which is used as the obtained monocular cropped image.

8. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 1, characterized in that, The specific steps for inputting the monocular cropped image into the image classification model for feature classification are as follows: Preprocessing is performed on the received monocular cropped image containing pixel feature information of the eye region to convert the two-dimensional image pixel matrix into a multi-dimensional tensor data structure suitable for neural network calculation; The multidimensional tensor data is used as input and fed into a pre-trained image classification model. Forward propagation calculation is performed within the image classification model to extract deep high-dimensional features related to eyelid closure in the image layer by layer. The feature classification is then performed, and finally, two log-probability values ​​corresponding to different classification states are output. A normalized exponential function operation is introduced at the output end to perform exponential operation on the logarithmic probability value of each category, and calculate the probability value of the monocular image in the current video frame belonging to the closed-eye state category and the probability value of belonging to the open-eye state category.

9. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 8, characterized in that, The specific steps for cumulatively calculating the fatigue measurement index are as follows: Based on the probability values ​​of the state categories, the instantaneous binarized state of the eyes at each time node within the analysis time window is determined. When the final classification is determined to be the closed eye state, the corresponding state logic value is recorded as a constant 1, and when the final classification is determined to be the open eye state, the corresponding state logic value is recorded as a constant 0. Perform a summation algebra operation on all instantaneous state values ​​recorded as a constant 1 within the time window to accumulate the total number of frames in which the target person is in a closed-eye state. Divide the total number of closed-eye frames by the total number of image frames contained in the time window to calculate the cumulative percentage of eyelid closure per unit time. The instantaneous eye state change sequence within the time window is scanned on the time axis. The total number of complete blink events occurring within the current time window is counted and divided by the duration interval of the time window to generate the corresponding blink frequency value. The fatigue measurement index is composed of the percentage of eyelid closure per unit time and the blink frequency value.

10. The driver fatigue detection method based on dual-stage detection and eye feature analysis according to claim 9, characterized in that, The specific steps for comparing the driver's condition with a preset fatigue threshold, determining that the driver is fatigued, and outputting a warning signal to trigger the alarm device are as follows: The absolute duration of the continuous state logic value being 1 within the accumulated time window is used to obtain the numerical variable of the long eye-closing duration. Read multiple preset fatigue threshold parameters, and use the extracted standard upper limit threshold for eyelid closure percentage, standard upper and lower limit thresholds for blinking frequency, and standard alarm threshold for continuous long eye closure duration as the preset fatigue thresholds; Initiate a multi-channel concurrent numerical comparison process, and compare the real-time calculated per-unit-time eyelid closure percentage, blink frequency, and real-time accumulated long eye closure duration variables with the corresponding standard preset thresholds that serve as the preset fatigue thresholds, and perform the comparison with the preset fatigue thresholds. When any of the above-mentioned real-time calculated values ​​reach or exceed the corresponding standard preset threshold, the internal fatigue state determination register flag is changed, thereby determining that the driver is in a fatigue state, and a hardware-level control command to trigger an audible and visual alarm is generated as the warning signal output. The control command powers on and activates the buzzer array and flashing warning light module to trigger the alarm device.

Images