A driver fatigue detection method, device and storage medium based on improved PIPNet network

Abstract

The present invention discloses an improved driver fatigue detection method, device and storage medium based on a PIPNet network. The method comprises: inputting a driver face image to be detected into a trained driver fatigue detection model, the model comprising a backbone network, a PIP regressor and a deformable decoder, wherein the backbone network is a ResNet backbone network that replaces the PIPNet network by embedding a CA attention module into a ScalableViT‑S network; extracting features from the driver face image through the backbone network to obtain a feature map; then, using the PIP regressor, predicting the coordinates of each key point and the 10 nearest key point neighbors of each key point to obtain initial key point coordinates; sampling the last three layers of the feature map through the deformable decoder, and fine-tuning the initial key point coordinates to obtain 68 facial key point coordinates; and judging whether the driver is in a fatigued driving state based on the 68 facial key point coordinates.

Description

Technical Field

[0001] The invention relates to an improved driver fatigue detection method based on a PIPNet network, and belongs to the field of key point detection. Background Art

[0002] As the number of cars continues to grow, most traffic accidents are caused by driver errors in cognition and operation, resulting in numerous casualties and economic losses, becoming a major hidden danger in people's lives. Promptly reminding drivers of fatigue driving behavior can reduce the occurrence of a large number of traffic accidents.

[0003] Detection based on the driver's physiological signals requires wearing medical equipment to monitor the driver's physiological signals, which is not flexible and affects the driver's normal driving.

[0004] Driver fatigue detection based on facial keypoint technology requires only in-vehicle cameras and related software. It's simple to deploy, convenient to use, and protects the driver's privacy. This method offers high real-time accuracy and prompts drivers to take safety measures, thus preventing accidents caused by fatigued driving.

[0005] Traditional key point detection tasks require higher resolution and are accompanied by a large amount of computation. Summary of the Invention

[0006] The ScalableViT-S network incorporates scalable self-attention and window-based interactive self-attention, fusing local and global attention mechanisms to extract hierarchical features from images. Scalable self-attention incorporates variable factors into the spatial and channel dimensions of self-attention, reducing computational complexity. Window-based interactive self-attention divides the image into several windows and interleaves information between independent windows through local interaction modules, effectively improving the model's expressiveness.

[0007] The attention mechanism is a lightweight self-attention module that can learn the spatial positions in the input feature map and the relationship between the features at these positions, which helps the network locate the target more accurately.

[0008] The PIPNet network proposes a nested regression head, which performs coordinate regression based on heatmap regression from low-resolution feature maps, saving computation while maintaining high accuracy. To enhance robustness to extreme facial poses, the network simultaneously predicts the offset of each keypoint and its neighbors, imposing certain shape constraints on the coordinate regression.

[0009] The multi-scale deformable decoder includes self-attention and cross-attention. Traditional self-attention uses global information to calculate attention weights. To address this issue, the deformable attention module, inspired by deformable convolution, samples features at different scales based on the normalized reference point coordinates. By focusing on four sampling points around the reference point, facial keypoints are predicted as relative offsets from the reference point, reducing the difficulty of predicting facial keypoints. Using facial keypoint detection technology can promptly detect fatigued driving and remind drivers to rest, thereby reducing the occurrence of traffic accidents.

[0010] Objective: To overcome the deficiencies in the prior art, the present invention provides an improved driver fatigue detection method, device and storage medium based on the PIPNet network.

[0011] Technical solution: To solve the above technical problems, the technical solution adopted by the present invention is:

[0012] In a first aspect, the present invention provides an improved driver fatigue detection method based on a PIPNet network, comprising:

[0013] Obtain the face image of the driver to be detected;

[0014] Inputting the driver face image to be detected into a trained driver fatigue detection model, wherein the driver fatigue detection model includes a backbone network, a PIP regressor, and a deformable decoder, wherein the backbone network is a ResNet backbone network that replaces the PIPNet network by embedding a CA attention module into a ScalableViT-S network;

[0015] The driver's face image to be detected is extracted through the backbone network to obtain a feature map;

[0016] The feature map is passed through the PIP regressor to predict the coordinates of each key point and the 10 nearest key point neighbors of each key point to obtain the initial key point coordinates;

[0017] Based on the initial key point coordinates, the deformable decoder samples the last three layers of feature maps and fine-tunes the initial key point coordinates to obtain 68 facial key point coordinates.

[0018] Based on the coordinates of the 68 facial key points and the preset threshold, it is determined whether the driver is in a fatigue driving state and a driver fatigue detection result is obtained.

[0019] In some embodiments, the training method of the driver fatigue detection model includes:

[0020] Step 1: Obtain a dataset with 68 facial key points, generate the key point coordinates of the average face based on the dataset, and label the dataset;

[0021] Step 2: Build a driver fatigue detection model;

[0022] Step 3: Use the labeled dataset and the key point coordinates of the average face to train the constructed driver fatigue detection model to obtain a trained driver fatigue detection model.

[0023] Furthermore, in some embodiments, obtaining a dataset having 68 facial key points also includes preprocessing the dataset having 68 facial key points, wherein the preprocessing includes: expanding the driver's face picture in the dataset by 10% according to a preset bounding box, cropping the face image, and then resizing it to 256×256.

[0024] Furthermore, in some embodiments, during the training process of the constructed driver fatigue detection model using a labeled data set and the key point coordinates of the average face, the coordinates of the 10 nearest key point neighbors of each key point are calculated based on the key point coordinates of the average face generated by the training set, which are used to assist the coordinate regression of the neighbor key points.

[0025] In some embodiments, the facial image of the driver to be detected is subjected to feature extraction by the backbone network to obtain a feature map, including: the CA attention module in the ScalableViT-S network uses a self-attention mechanism for feature extraction instead of the traditional convolutional network using convolution kernels for feature extraction; the CA attention module integrates channel attention and spatial attention to learn the relationship between channels and the location information of key points.

[0026] In some embodiments, the feature map is passed through a PIP regressor to predict the coordinates of each key point and each key point's 10 nearest key point neighbors to obtain the initial key point coordinates, including:

[0027] The PIP regressor uses heatmap regression at low resolution to predict the grid where each keypoint is located, the offset of the keypoint relative to the upper left corner of the grid on the x-axis and y-axis, and the offset of the 10 nearest neighbor keypoints of each keypoint relative to the upper left corner of the grid on the x-axis and y-axis;

[0028] The coordinates of each key point directly predicted and the coordinates of the key point predicted by the neighboring key points are averaged to obtain the initial key point coordinates;

[0029] Specifically include:

[0030]

[0031] where x i、y i Indicates the x-axis and y-axis values ​​of the i-th initial key point. x and y are the x-axis and y-axis values ​​of each key point directly predicted. K means that each key point has K neighboring key points to assist in finding the coordinate value. j 、y j Indicates the x-axis and y-axis values ​​of the i-th key point predicted by the K-th neighbor key point.

[0032] In some embodiments, based on the initial key point coordinates, the deformable decoder samples the last three layers of feature maps and fine-tunes the initial key point coordinates to obtain 68 facial key point coordinates, including:

[0033] The deformable decoder includes three layers of deformable decoders, each layer of which is connected in series with a self-attention module and a multi-scale deformable attention module;

[0034] The deformable decoder uses the initial key points as reference points, downsamples the driver's face image to feature maps of 8, 16, and 32, and samples them. It outputs the offset of each key point on the x-axis and y-axis and adds it to the initial key point coordinates to obtain the final 68 facial key point coordinates.

[0035] In some embodiments, based on the coordinates of the 68 facial key points and a preset threshold, determining whether the driver is in a fatigue driving state and obtaining a driver fatigue detection result includes:

[0036] The eye and mouth key points are obtained based on the 68 facial key points to calculate the eye aspect ratio EAR, mouth aspect ratio MAR, and eye closure time percentage PERCLOS;

[0037] In response to EAR being less than 0.20 or MAR being greater than 0.6 or PERCLOS being greater than 0.8, it is determined that the driver is in a fatigue driving state.

[0038] In some embodiments, obtaining a facial image of a driver to be detected includes: extracting the facial image of the driver from a video using an MTCNN network.

[0039] In a second aspect, the present invention provides an improved driver fatigue detection device based on a PIPNet network, comprising a processor and a storage medium;

[0040] The storage medium is used to store instructions;

[0041] The processor is configured to operate according to the instructions to perform the method according to the first aspect.

[0042] In a third aspect, the present invention provides a device comprising:

[0043] Memory;

[0044] processor;

[0045] as well as

[0046] computer programs;

[0047] The computer program is stored in the memory and is configured to be executed by the processor to implement the method described in the first aspect above.

[0048] In a fourth aspect, the present invention provides a storage medium having a computer program stored thereon, wherein the computer program implements the method described in the first aspect when executed by a processor.

[0049] Beneficial effects: The improved driver fatigue detection method, device, and storage medium based on the PIPNet network provided by the present invention have the following advantages:

[0050] The MTCNN network is used to extract the driver's face from the video. The improved PIPNet network is then used to extract facial features and generate 68 facial landmarks. Based on the eye and mouth landmarks, the eye aspect ratio (EAR), percentage of time closed (PERCLOS), and mouth aspect ratio (MAR) are calculated to assess the driver's fatigue status. In the feature extraction phase, an attention mechanism module is integrated with the ScalableViT-S network, replacing the original PIPNet backbone. 68 facial landmarks are generated on a feature map downsampled by a ratio of 32 relative to the input face image. Finally, a cascaded deformable decoder fine-tunes the generated landmarks. Embedding the attention mechanism module into the ScalableViT-S network, replacing the original ResNet50 backbone, improves global field of view perception and positional awareness while maintaining the same model parameter count and computational complexity, enabling the model to more accurately locate the initial facial landmarks. The deformable decoder fine-tunes the positions of the initial landmarks, improving the model's detection accuracy.

[0051] The present invention proposes a driver fatigue detection method based on an improved PIPNet network. Through key point detection technology, the coordinates of key points on the driver's face and eyes can be detected, the driver's eye opening and closing and yawning behaviors in the video can be judged, and the driver's fatigue state can be monitored in real time. The method can be applied to drivers of different ages, genders, nationalities and driving experience, does not require physical contact with the driver and manual intervention, and is convenient and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 is a flow chart of a method provided by an embodiment of the present invention;

[0053] Figure 2are 68 facial key points according to one embodiment of the present invention;

[0054] Figure 3 is a CA attention module embedded in the improved PIPNet according to one embodiment of the present invention;

[0055] Figure 4 1 is a diagram showing the improved PIPNet network backbone structure according to an embodiment of the present invention;

[0056] Figure 5 FIG. 4 is a structural diagram of a deformable decoder according to an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0058] The present invention will be further described below in conjunction with the accompanying drawings and examples. The following examples are only used to more clearly illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention.

[0059] In the description of the present invention, "several" means more than one, "plurality" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0060] In the description of the present invention, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the exemplary expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0061] Example 1

[0062] In a first aspect, this embodiment provides an improved driver fatigue detection method based on the PIPNet network, including:

[0063] Obtain the face image of the driver to be detected;

[0064] Inputting the driver face image to be detected into a trained driver fatigue detection model, wherein the driver fatigue detection model includes a backbone network, a PIP regressor, and a deformable decoder, wherein the backbone network is a ResNet backbone network that replaces the PIPNet network by embedding a CA attention module into a ScalableViT-S network;

[0065] The driver's face image to be detected is extracted through the backbone network to obtain a feature map;

[0066] The feature map is passed through the PIP regressor to predict the coordinates of each key point and the 10 nearest key point neighbors of each key point to obtain the initial key point coordinates;

[0067] Based on the initial key point coordinates, the deformable decoder samples the last three layers of feature maps and fine-tunes the initial key point coordinates to obtain 68 facial key point coordinates.

[0068] Based on the coordinates of the 68 facial key points and the preset threshold, it is determined whether the driver is in a fatigue driving state and a driver fatigue detection result is obtained.

[0069] In some embodiments, the training method of the driver fatigue detection model includes:

[0070] Step 1: Obtain a dataset with 68 facial key points, generate the key point coordinates of the average face based on the dataset, and label the dataset;

[0071] Step 2: Build a driver fatigue detection model;

[0072] Step 3: Use the labeled dataset and the key point coordinates of the average face to train the constructed driver fatigue detection model to obtain a trained driver fatigue detection model.

[0073] Furthermore, in some embodiments, obtaining a dataset having 68 facial key points also includes preprocessing the dataset having 68 facial key points, wherein the preprocessing includes: expanding the driver's face picture in the dataset by 10% according to a preset bounding box, cropping the face image, and then resizing it to 256×256.

[0074] Furthermore, in some embodiments, during the training process of the constructed driver fatigue detection model using a labeled data set and the key point coordinates of the average face, the coordinates of the 10 nearest key point neighbors of each key point are calculated based on the key point coordinates of the average face generated by the training set, which are used to assist the coordinate regression of the neighbor key points.

[0075] In some embodiments, the facial image of the driver to be detected is subjected to feature extraction by the backbone network to obtain a feature map, including: the CA attention module in the ScalableViT-S network uses a self-attention mechanism for feature extraction instead of the traditional convolutional network using convolution kernels for feature extraction; the CA attention module integrates channel attention and spatial attention to learn the relationship between channels and the location information of key points.

[0076] In some embodiments, the feature map is passed through a PIP regressor to predict the coordinates of each key point and each key point's 10 nearest key point neighbors to obtain the initial key point coordinates, including:

[0077] The PIP regressor uses heatmap regression at low resolution to predict the grid where each keypoint is located, the offset of the keypoint relative to the upper left corner of the grid on the x-axis and y-axis, and the offset of the 10 nearest neighbor keypoints of each keypoint relative to the upper left corner of the grid on the x-axis and y-axis;

[0078] The initial key point coordinates are obtained by averaging the directly predicted coordinates of each key point and the coordinates of the key point assisted by the neighboring key points.

[0079] In some specific embodiments, Figure 1 As shown in FIG, a driver fatigue detection method based on an improved PIPNet network includes the following steps:

[0080] Step 1: During the training and testing phases, the 3 million dataset was divided into training, validation, and test sets. Faces were cropped and resized to 256×256. The keypoint coordinates of the average face were generated based on the training set. In the application, the images extracted using the MTCNN network were resized to 256×256 and fed into the improved PIPNet network to detect the coordinates of the driver's 68 facial keypoints. Figure 2 These are the 68 facial key points according to this embodiment.

[0081] The x-axis and y-axis coordinates of each key point of the average face are calculated according to formula (1). The MTCNN network is used in the application to extract the driver's face image in the video:

[0082]

[0083]

[0084] in Represents the x-axis value of the i-th key point of the average face, Num represents the number of faces in the training set, x j Indicates the value of the j-th facial key point on the x-axis. and y jSimilarly;

[0085] Step 2: ResNet requires convolution operations on the entire image, which is computationally inefficient. However, the multi-head self-attention mechanism in ScalableViT-S allows the model to focus on areas of interest when processing images. Embedding the attention mechanism module into the ScalableViT-S network as the backbone network improves global vision perception and position capture perception while maintaining the number of model parameters and computational complexity, helping the model to more accurately locate key points on the driver's face. Figure 3 It is the CA attention module embedded in the improved PIPNet according to this embodiment; Figure 4 This is the improved PIPNet network backbone structure according to this embodiment; the ScalableViT-S network replaces ResNet. The ScalableViT-S network uses a self-attention mechanism for feature extraction, replacing the convolution kernels used in traditional convolutional networks. This provides enhanced overall visual perception capabilities. Coordinate Attention is a lightweight attention module that combines channel attention and spatial attention to effectively learn inter-channel relationships and keypoint location information.

[0086] Step 3: The network predicts the coordinates of each key point and the coordinates of its 10 nearest key point neighbors. The coordinates of each initial key point are calculated according to formula (2):

[0087]

[0088] where x i 、y i represents the x-axis and y-axis values ​​of the i-th initial key point of the average face. x and y are the x-axis and y-axis values ​​of each key point directly predicted by the network. K means that each key point has K neighbors to assist in finding the coordinate value. j 、y j Indicates the x-axis and y-axis values ​​of the i-th key point predicted by the K-th neighbor;

[0089] Step 4: In order to obtain more accurate key point coordinates, the last three layers of feature maps are sampled using the initial key point coordinates in step 3 as sampling points, and the initial key point coordinates are fine-tuned to obtain 68 facial key point coordinates. Figure 5 According to the deformable decoder structure diagram in this embodiment, there are three layers of deformable decoders, and each layer is connected in series with a self-attention module and a multi-scale deformable attention module.

[0090] Step 5: Use the improved PIPNet network to detect 68 facial key points and determine whether the driver is in a fatigued driving state based on the preset threshold. Let EAR be the percentage of the area of ​​the pupil covered by the eyelid and PERCLOS be the proportion of eye closure per unit time. Calculate the average EAR of the driver's left and right eyes. When the EAR value is less than 20%, it is considered closed eyes. Use the MAR indicator to measure the degree of the driver's mouth opening. When the MAR value is greater than 0.60, it is determined to be a yawning state. Set it to 30 seconds as 1 unit. When the PERCLOS value is greater than 80% or the frequency of yawning is greater than 0.60, it is determined to be a yawning state. If the value is greater than 80%, it is determined whether the driver is in a fatigue state. Calculate according to formula (3):

[0091]

[0092] where ‖x 40 -x 37 ‖ represents the width of the eyes, Indicates the height of the eyes.

[0093] ‖x 65 -x 61 ‖ represents the width of the mouth, Indicates the height of the mouth. K represents the number of closed-eye frames within 30 seconds, and N represents the total number of frames within 30 seconds;

[0094] In some specific application examples, the method should be based on the vehicle camera to collect the driver's facial data in real time. The MTCNN network is used to extract the driver's face image in the video, and then the improved PIPNet network is used to detect the driver's facial key points in each frame. The PERCLOS value and When PERCLOS is greater than 80% or If the value is greater than 80%, the driver is judged to be in a fatigue state.

[0095] Example 2

[0096] In a second aspect, based on Example 1, this embodiment provides an improved driver fatigue detection device based on the PIPNet network, including a processor and a storage medium;

[0097] The storage medium is used to store instructions;

[0098] The processor is configured to operate according to the instructions to perform the method according to embodiment 1.

[0099] Example 3

[0100] In a third aspect, based on embodiment 1, this embodiment provides a device, including:

[0101] Memory;

[0102] processor;

[0103] as well as

[0104] computer programs;

[0105] The computer program is stored in the memory and is configured to be executed by the processor to implement the method described in embodiment 1.

[0106] Example 4

[0107] In a fourth aspect, based on Example 1, this embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the method described in Example 1 is implemented.

[0108] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0109] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0110] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0111] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A driver fatigue detection method based on an improved PIPNet network, characterized in that: include: Obtain the driver's face image to be detected; Inputting the driver face image to be detected into a trained driver fatigue detection model, wherein the driver fatigue detection model includes a backbone network, a PIP regressor, and a deformable decoder, wherein the backbone network is a ResNet backbone network that replaces the PIPNet network by embedding a CA attention module into a ScalableViT-S network; The driver's face image to be detected is extracted through the backbone network to obtain a feature map; The feature map is passed through the PIP regressor to predict the coordinates of each key point and the 10 nearest key point neighbors of each key point, and the initial key point coordinates are obtained, including: The PIP regressor uses heatmap regression at low resolution to predict the grid where each keypoint is located, the offset of the keypoint relative to the upper left corner of the grid on the x-axis and y-axis, and the offset of the 10 nearest neighbor keypoints of each keypoint relative to the upper left corner of the grid on the x-axis and y-axis; The coordinates of each key point directly predicted and the coordinates of the key point predicted by the neighboring key points are averaged to obtain the initial key point coordinates; where x i 、y i Indicates the x-axis and y-axis values ​​of the i-th initial key point. x and y are the x-axis and y-axis values ​​of each key point directly predicted. K means that each key point has K neighboring key points to assist in finding the coordinate value. j 、y j Indicates the x-axis and y-axis values ​​of the i-th key point predicted by the K-th neighbor key point; Based on the initial key point coordinates, the deformable decoder samples the last three layers of feature maps and fine-tunes the initial key point coordinates to obtain 68 facial key point coordinates, including: the deformable decoder includes three layers of deformable decoders, each layer of the deformable decoder is connected in series with a self-attention module and a multi-scale deformable attention module; the deformable decoder uses the initial key point as a reference point, downsamples the driver's face image to be detected into feature maps of 8, 16, and 32 and samples them, outputs the offset of each key point on the x-axis and y-axis, and adds it to the initial key point coordinates to obtain the final 68 facial key point coordinates; Based on the coordinates of the 68 facial key points and the preset threshold, it is determined whether the driver is in a fatigue driving state, and the driver fatigue detection result is obtained, including: obtaining the eye and mouth key points based on the 68 facial key points to calculate the eye aspect ratio EAR, mouth aspect ratio MAR, and eye closure time percentage PERCLOS; in response to EAR being less than 0.20 or MAR being greater than 0.6 or PERCLOS being greater than 0.8, it is determined that the driver is in a fatigue driving state.

2. The improved driver fatigue detection method based on PIPNet network according to claim 1 is characterized in that: The training method of the driver fatigue detection model includes: Step 1: Obtain a dataset with 68 facial key points, generate the key point coordinates of the average face based on the dataset, and label the dataset; Step 2: Build a driver fatigue detection model; Step 3: Use the labeled dataset and the key point coordinates of the average face to train the constructed driver fatigue detection model to obtain a trained driver fatigue detection model.

3. The improved driver fatigue detection method based on PIPNet network according to claim 2 is characterized in that: Obtaining a data set with 68 facial key points also includes preprocessing the data set with 68 facial key points, wherein the preprocessing includes: enlarging the driver's face picture in the data set by 10% according to a preset bounding box to crop the face image, and then resizing it to 256×256.

4. The driver fatigue detection method based on the improved PIPNet network according to claim 2 is characterized in that: During the training process of the constructed driver fatigue detection model using the labeled dataset and the key point coordinates of the average face, the coordinates of the 10 nearest key point neighbors of each key point are calculated based on the key point coordinates of the average face generated by the training set, which are used to assist the coordinate regression of the neighbor key points.

5. The improved driver fatigue detection method based on PIPNet network according to claim 1 is characterized in that: The facial image of the driver to be detected is subjected to feature extraction through the backbone network to obtain a feature map, including: the CA attention module in the ScalableViT-S network uses a self-attention mechanism for feature extraction instead of the convolution kernel used in traditional convolutional networks; the CA attention module combines channel attention and spatial attention to learn the relationship between channels and the location information of key points.

6. The improved driver fatigue detection method based on PIPNet network according to claim 1 is characterized in that: Obtaining a face image of the driver to be detected, including: using the MTCNN network to extract the face image of the driver from the video.

7. An electronic device, characterized in that: include: Memory; processor; as well as computer programs; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images