Driver driving behavior detection method

Abstract

The invention provides a driver driving behavior detection method, and belongs to the technical field of behavior detection, and the method comprises the steps: obtaining a safety equipment image when a driver drives through a camera disposed in a vehicle unit, and carrying out the enhancement processing of the safety equipment image, and obtaining an enhanced driver driving image; performing face feature point detection on the driver by using a Retina Face algorithm, and analyzing the driving state of the driver by extracting eye features and mouth features of the driver; an improved Small-Goals-Yov8 model is constructed, and the weight of the improved Small-Goals-Yov8 model is utilized to carry out prediction; and according to the extracted driving state and the prediction result, alarming and reminding the non-standard driving behavior of the driver to complete the detection of the driving behavior of the driver. The problems that an existing driver driving behavior detection device is high in deployment difficulty, high in cost and difficult to maintain are solved.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior detection, and in particular relates to a method for detecting a driver's driving behavior. Background Art

[0002] The importance of traffic safety to society is self-evident. Maintaining good traffic order and regulating driving behavior can not only reduce casualties and property losses caused by traffic accidents, but also promote social stability and economic development. Most road traffic accidents are caused by drivers' irregular driving behaviors. Drivers' irregular driving behaviors include: talking on the phone, taking their hands off the steering wheel, smoking, eating, closing their eyes for a long time, yawning, etc. Therefore, detecting and regulating drivers' driving behaviors, improving drivers' driving concentration, and reducing the occurrence of traffic accidents have become important research directions.

[0003] Currently, there are two solutions for driver behavior detection devices: automatic recognition and manual monitoring. Manual monitoring equipment consumes a lot of human resources and has high requirements for network quality. On the other hand, the deployment cost of intelligent recognition equipment is high, the vehicle needs to be modified, and it is difficult to maintain. Summary of the invention

[0004] In view of the above-mentioned deficiencies in the prior art, the present invention provides a driver driving behavior detection method to solve the problems of high difficulty in deployment, high cost and difficult maintenance of the current driver driving behavior detection device.

[0005] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0006] This solution provides a driver driving behavior detection method, comprising the following steps:

[0007] S1. Obtain the safety equipment image of the driver while driving through the camera deployed in the vehicle unit, and enhance the safety equipment image to obtain an enhanced driver driving image;

[0008] S2. Based on the enhanced driving image of the driver, the driver's facial feature points are detected using the Retina Face algorithm, and the driver's driving status is analyzed by extracting the driver's eye features and mouth features;

[0009] S3, construct an improved Small-Goals-Yolov8 model, and use the improved Small-Goals-Yolov8 model weights for prediction;

[0010] S4. Based on the driving status extracted in step S2 and the prediction result in step S3, the driver's irregular driving behavior is warned and reminded to complete the detection of the driver's driving behavior.

[0011] The beneficial effects of the present invention are as follows: the present invention combines the driver driving behavior detection based on the retina algorithm Retina Face and the yolov8 algorithm to improve the accuracy of driver behavior recognition in blurred images; finally, an alarm is issued for the detected irregular driving behavior to remind the driver to drive safely.

[0012] Furthermore, the step S1 includes the following steps:

[0013] S101, obtaining an image of the driver's safety equipment while driving through a camera deployed in a vehicle unit;

[0014] S102, transforming the safety equipment image through histogram equalization, and redistributing brightness values ​​of image pixels;

[0015] S103, remapping pixel values ​​based on a grayscale histogram of the safety equipment image;

[0016] S104, performing enhancement processing according to step S102-step S103 to obtain an enhanced driver driving image.

[0017] The beneficial effect of the above further solution is that: this step enhances the collected image, improves the representation of the behavior feature points, and thus improves the detection accuracy.

[0018] Furthermore, step S2 includes the following steps:

[0019] S201, defining 150 frames as one round of fatigue assessment according to the enhanced driving image of the driver;

[0020] S202, in a round of fatigue assessment, for eye features, the eye opening and closing states are determined by analyzing the eye states based on the aspect ratio, and the first fatigue level of the driver is assessed using the retina algorithm Retina Face;

[0021] S203. In a round of fatigue assessment, for mouth features, whether there is yawning behavior is determined based on the opening and closing degree threshold, and the second fatigue level of the driver is assessed using the duty cycle algorithm.

[0022] The beneficial effect of the above further solution is that: this step improves the reliability of fatigue level judgment by judging yawning behavior through the mouth opening and closing threshold.

[0023] Furthermore, the eye state analysis based on aspect ratio is used to determine the eye opening and closing state, and the retina algorithm Retina Face is used to evaluate the driver's fatigue level, which is specifically as follows:

[0024] Set the EYE value and determine whether the EYE value is greater than a threshold value. If so, the eyes are treated as being in a closed state, and within a round of fatigue assessment, the first fatigue level of the driver is assessed according to the time taken for the eyes to be closed. Otherwise, the eyes are not in a closed state.

[0025] The expression of the EYE value is as follows:

[0026] EYE=(h1+h2) / (2×w)

[0027] Where EYE represents the eye opening ratio, h1 and h2 represent the vertical distances of the eyelids on both sides of the pupil, and w represents the maximum horizontal width of the eye.

[0028] The expression of the first fatigue level is as follows:

[0029]

[0030] Among them, perclos represents the first fatigue level, s represents the number of frames of eye closure, and S represents the total number of frames in a round of fatigue assessment.

[0031] The beneficial effect of the above further scheme is that: this step improves the reliability of fatigue degree judgment by judging eye closing behavior through the eye opening degree threshold.

[0032] Furthermore, the second fatigue level of the driver is evaluated by judging whether there is yawning behavior and using the load rate algorithm DutyCycle, which is specifically:

[0033] The MAR value is used to determine whether there is yawning behavior. If so, the second fatigue level of the driver is evaluated according to the time occupied by yawning within a round of fatigue assessment. Otherwise, there is no yawning behavior.

[0034] The expression of the MAR value is as follows:

[0035] MAR=(w1+w2) / (2×s')

[0036] Where MAR represents the mouth opening and closing ratio, w1 and w2 represent the vertical distances on both sides of the lips, and s' represents the maximum horizontal width of the mouth;

[0037] The expression of the second fatigue level is as follows:

[0038]

[0039] Among them, yawn represents the proportion of yawning frames, n represents the number of yawning frames, and N represents the total number of frames in a round of fatigue assessment.

[0040] The beneficial effect of the above further solution is that the image is digitized by an algorithm, thereby improving the visibility of the detected values.

[0041] Furthermore, the structure of the Small-Goals-Yolov8 model is as follows: a single-layer regression structure SA-ones is introduced on the Yolov8 framework, a cross-scale feature fusion module CCFM is introduced on the shallow layer of the network, a deep convolution module SPD-Conv is used on the convolution layer and the pooling layer, and the network neck is connected layer by layer through multi-scale feature fusion, by fusing feature maps from different stages of the backbone.

[0042] The beneficial effect of the above further scheme is that introducing these modules into the Yolov8 framework can effectively improve the detection accuracy, robustness and generalization ability of the model, especially in complex scenes and multi-scale target detection tasks, which can significantly improve the performance of the model.

[0043] Further, the single-layer regression structure SA-ones includes a first unit with a step size of 1 and a second unit with a step size of 2, and the main branches of the first unit and the second unit each include a first convolutional layer, a first depth-separable convolutional layer, and a second convolutional layer connected in sequence;

[0044] The input end of the first unit performs channel segmentation on the received image, and the side branch of the first unit splices the channel segmented image with the main branch image in terms of dimension;

[0045] The image received at the input end of the second unit is processed by the main branch of the second unit and the side branch of the second unit respectively, and the processing results are spliced ​​through the main branch of the second unit and the side branch of the second unit, and the side branch of the second unit includes a second depthwise separable convolutional layer and a third convolutional layer connected in sequence;

[0046] The concatenation result of the first unit and the second unit uses channel shuffling to transmit information.

[0047] The beneficial effects of the above further scheme are: Improve feature expression ability: The SA-ones structure can capture the global correlation between features, thereby improving the network's feature expression ability at different scales and contexts. Enhance the model's discriminative ability: The self-attention mechanism can help the model better identify key features and improve the accuracy of target detection and classification. Reduce information loss: By focusing on the global information of the input data, the SA-ones structure can more effectively retain key feature information and reduce false detection or missed detection caused by information loss.

[0048] Further, the cross-scale feature fusion module CCFM includes a fourth convolutional layer, a fifth convolutional layer and a repetitive block RepBlock connected to the fifth convolutional layer, and the fourth convolutional layer is spliced ​​with the repetitive block RepBlock;

[0049] The SPD-Conv module includes sequential connections from spatial to depth layers and non-strided convolutional layers.

[0050] The beneficial effects of the above further scheme are: Improving the expressiveness of shallow features: Introducing the CCFM module in the shallow layer of the network can better process low-level features (such as edges, textures, etc.) and improve the recognition of features. Enhancing the complementarity of inter-channel features: The CCFM module can adjust feature weights across channels, making the feature information between different channels more complementary, thereby improving the feature fusion effect. Improving the generalization ability of the model: By optimizing shallow features, the CCFM module can improve the performance of the model on different data sets, thereby improving the generalization ability.

[0051] Furthermore, the step S4 is specifically as follows:

[0052] According to the driving status extracted in step S2 and the prediction result in step S3, alarms and reminders are issued when there are yawning, long-term eye closing, phone calls, hands off the steering wheel, smoking, and eating behaviors, thereby completing the detection of the driver's driving behavior.

[0053] The beneficial effects of the above further scheme are: Real-time monitoring and early warning: The system can detect dangerous behaviors that may cause traffic accidents in real time, and issue warnings in time to remind drivers to pay attention, thereby reducing the risk of accidents caused by distracted or fatigued driving. Prevent potential dangers: By monitoring and analyzing the driver's facial expressions, body movements and other behavioral characteristics, the system can warn of potential dangerous conditions in advance, such as fatigue, distraction, etc., to help the driver take corrective measures; Reduce the risk of fatigue driving: By detecting behaviors such as yawning and closing eyes for a long time, the system can identify fatigue status and remind drivers to rest in time to reduce the possibility of accidents caused by fatigue driving; Reduce distracted driving behavior: When detecting distracting behaviors such as making phone calls, taking hands off the steering wheel, smoking, eating, etc., the system can issue warnings to encourage drivers to concentrate and reduce accidents caused by distraction; User habit formation: Long-term use of the system can help drivers gradually form good driving habits and reduce the occurrence of bad driving behaviors. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 The figure is a flow chart of the method of the present invention.

[0055] Figure 2 A schematic diagram of the Retina Face structure provided by an embodiment of the present invention.

[0056] Figure 3 A schematic diagram of eye states based on aspect ratio provided by an embodiment of the present invention.

[0057] Figure 4 A schematic diagram of a mouth state based on a threshold value of opening and closing degree provided in an embodiment of the present invention.

[0058] Figure 5 A schematic diagram of the SA-ones structure is provided for an embodiment of the present invention.

[0059] Figure 6 A CCFM structural diagram is provided for an embodiment of the present invention.

[0060] Figure 7 The embodiments of the present invention provide a Small-Goals-Yolov8 model and box loss, target loss, classification loss, precision, recall, average precision and mean average precision curves during training of the training and validation sets. DETAILED DESCRIPTION

[0061] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0062] Example

[0063] like Figure 1 As shown, the present invention provides a method for detecting a driver's driving behavior, and the implementation method thereof is as follows:

[0064] S1. Using a camera deployed in a vehicle unit, obtain a safety equipment image of the driver while driving, and enhance the safety equipment image to obtain an enhanced driver driving image. The implementation method is as follows:

[0065] S101, obtaining an image of the driver's safety equipment while driving through a camera deployed in a vehicle unit;

[0066] S102, transforming the safety equipment image through histogram equalization, and redistributing brightness values ​​of image pixels;

[0067] S103, remapping pixel values ​​based on a grayscale histogram of the safety equipment image;

[0068] S104, performing enhancement processing according to step S102-step S103 to obtain an enhanced driver driving image.

[0069] In this embodiment, a camera deployed in a vehicle unit is used to obtain a safety equipment image of the driver while driving, and a DCE (Data Comparison Enhancement) transformation processing method is used to enhance the safety equipment image to obtain an enhanced driver driving image; the DCE transformation is to transform the image through histogram equalization, redistribute the brightness values ​​of the image pixels, and thus increase the contrast of the image. This method remaps the pixel values ​​based on the grayscale histogram of the image, so that the pixel values ​​originally concentrated in certain grayscale levels are distributed throughout the grayscale range, thereby improving the image contrast and thus improving the data quality.

[0070] S2. Based on the enhanced driving image of the driver, the driver's facial feature points are detected using the Retina Face algorithm, and the driver's driving state is analyzed by extracting the driver's eye features and mouth features. The implementation method is as follows:

[0071] S201, defining 150 frames as one round of fatigue assessment according to the enhanced driving image of the driver;

[0072] S202, in a round of fatigue assessment, for eye features, the eye opening and closing states are determined by analyzing the eye states based on the aspect ratio, and the first fatigue level of the driver is assessed using the retina algorithm Retina Face;

[0073] The eye state analysis based on aspect ratio is used to determine the eye opening and closing state, and the retina algorithm Retina Face is used to evaluate the driver's fatigue level, which is specifically as follows:

[0074] Set the EYE value and determine whether the EYE value is greater than a threshold value. If so, the eyes are treated as being in a closed state, and within a round of fatigue assessment, the first fatigue level of the driver is assessed according to the time taken for the eyes to be closed. Otherwise, the eyes are not in a closed state.

[0075] S203, in a round of fatigue assessment, for mouth features, judging whether there is yawning behavior based on the opening and closing degree threshold, and using the duty cycle algorithm to assess the second fatigue level of the driver;

[0076] The method is based on judging whether there is yawning behavior and using the load rate algorithm Duty Cycle to evaluate the second fatigue level of the driver. Specifically, the MAR value is used to judge whether there is yawning behavior. If so, within one round of fatigue evaluation, the second fatigue level of the driver is evaluated according to the time occupied by yawning. Otherwise, there is no yawning behavior.

[0077] In this embodiment, the retina algorithm Retina Face uses a face detection network based on a single-layer regression structure SA-ones to extract facial features through feature point detection. Compared with the previous face detection method based on a double-layer regression structure Two-stage, the biggest advantage of the retina algorithm Retina Face is that its detection speed will not be affected by the number of faces in the image. If a non-fixed network mobileNet is selected in the backbone feature extraction network, the system's detection speed can meet the requirements of actual driving scenarios while ensuring accuracy.

[0078] In this embodiment, the structure of Retina Face is as follows Figure 2 As shown in the figure, the retina algorithm Retina Face is essentially based on the visual network RetinaNet structure, using feature pyramid technology to achieve multi-scale information fusion. It proposes detection boxes on the three layers of feature maps of the feature pyramid network feature pyramid net, generates detection boxes at three different scales, and introduces node anchor sizes of different sizes at each scale to ensure the detection rate of objects of different sizes.

[0079] In this embodiment, 150 frames are defined as one round of fatigue assessment in consideration of balancing time and computing resources. When performing fatigue assessment, continuous images need to be analyzed to determine the driver's state, and 150 frames are selected as one round of assessment to ensure that there is enough data to accurately assess the driver's state while not consuming too much computing resources.

[0080] In this embodiment, for eye features, the open and closed state is determined by eye state analysis based on aspect ratio, and the fatigue degree algorithm Perclos is used to evaluate the driver's fatigue degree. The aspect ratio eye state analysis refers to determining whether the eyes are in a closed state according to the set EYE value. Figure 3 As shown:

[0081] EYE=(h1+h2) / (2×w)

[0082] Where EYE represents the eye opening ratio, h1 and h2 represent the vertical distance between the eyelids on both sides of the pupil, and w represents the maximum horizontal width of the eye. By fitting the EYE values ​​of 35 subjects when their eyes are just closed, the threshold is 10.1327. When the EYE value is greater than the threshold, the eyes are judged to be closed. Threshold calculation formula:

[0083]

[0084] Among them, HOLD1 represents the eye opening threshold, i represents accumulation starting from 1, and h1 iIndicates the vertical distance of the eyelid to the left of the pupil, h2 i Indicates the vertical distance of the eyelid to the right of the pupil, w i Indicates the maximum horizontal width of the eye.

[0085] The Perclos algorithm refers to the time (perclos) that the eyes are closed within a round of fatigue assessment. The calculation formula for the first fatigue level is as follows:

[0086]

[0087] Among them, perclos represents the first fatigue level, s represents the number of frames of eye closure, and S represents the total number of frames in a round of fatigue assessment.

[0088] In this embodiment, for mouth features, a threshold based on the degree of opening and closing is used to determine whether yawning occurs, and the duty cycle algorithm is used to assess the driver's fatigue level. The degree of opening and closing threshold refers to determining whether yawning occurs based on the set MAR value. Figure 4 As shown:

[0089] MAR=(w1+w2) / (2×s')

[0090] Wherein, MAR represents the ratio of mouth opening and closing, w1 and w2 represent the vertical distances on both sides of the lips, and s' represents the maximum horizontal width of the mouth.

[0091] In this embodiment, the threshold value is 1.0235 by simulating the MAR value of 8 subjects yawning. When the EAR value is less than the threshold value, it is judged as yawning. Threshold calculation formula:

[0092]

[0093] Among them, HOLD2 mouth opening and closing threshold, i means accumulation starting from 1, w1 i Indicates the vertical distance to the left of the lip, w2 i Indicates the vertical distance to the right of the lip, s i Indicates the maximum horizontal width of the mouth.

[0094] In this embodiment, the duty cycle of the load rate algorithm refers to the time yawn occupied in a round of fatigue assessment. The expression of the second fatigue level is as follows:

[0095]

[0096] Among them, yawn represents the proportion of yawning frames, n represents the number of yawning frames, and N represents the total number of frames in a round of fatigue assessment.

[0097] S3. Build an improved Small-Goals-Yolov8 model and use the improved Small-Goals-Yolov8 model weights for prediction;

[0098] In this embodiment, the small-goals-Yolov8 model: introduces a single-layer regression structure SA-ones on the Yolov8 framework, introduces a cross-scale feature fusion module CCFM in the shallow layer of the network, uses a deep convolution module SPD-Conv on the convolution layer and the pooling layer, and the network neck is connected layer by layer through multi-scale feature fusion, and the feature maps from different stages of the backbone Backbone are fused.

[0099] In this embodiment, the single-layer regression structure SA-ones includes a first unit with a step size of 1 and a second unit with a step size of 2. The main branches of the first unit and the second unit both include a first convolutional layer, a first depth-separable convolutional layer and a second convolutional layer connected in sequence; the input end of the first unit performs channel segmentation on the received image, and the side branch of the first unit splices the channel-segmented image with the main branch image in dimension; the image received at the input end of the second unit is processed by the main branch of the second unit and the side branch of the second unit respectively, and the processing results are spliced ​​by the main branch of the second unit and the side branch of the second unit, and the side branch of the second unit includes a second depth-separable convolutional layer and a third convolutional layer connected in sequence; the splicing result of the first unit and the second unit uses channel shuffling for information transmission.

[0100] The cross-scale feature fusion module CCFM includes a fourth convolutional layer, a fifth convolutional layer and a repetitive block RepBlock connected to the fifth convolutional layer, and the fourth convolutional layer is spliced ​​with the repetitive block RepBlock; the SPD-Conv module includes sequential connections from space to depth layers and non-strided convolutional layers.

[0101] In this embodiment, applying spatial attention and channel attention separately can achieve better performance, but fusing them is bound to increase the computational cost. To solve this problem, the SA-ones structure based on the regression structure SA is introduced, such as Figure 5 As shown in the figure, the structure contains two different units, with strides of 1 and 2 respectively; the main branches of the two units are composed of 3 convolutional layers. The first unit performs channel division after data input, and the side branch directly splices the segmentation structure with the main branch; the second unit data input passes through two branches, and the side branch consists of 3*3 depth-separable convolution and 1 ordinary convolution. It can effectively reduce the computational cost when fusing spatial attention and channel attention. The structure is divided into Q sub-features according to the channel dimension, that is, X=[X1,X2,X3,…,X Q], the sub-features are divided into X along the channel dimension K1 With X K2 To obtain channel weights, global average pooling (GAP) is used to generate statistical information s' in a channel manner. The sub-feature X can be placed in the spatial dimensions H (height) and W (weight) to calculate:

[0102]

[0103] Among them, s' represents the eigenvalue, F GAP Represents the eigenvalue calculation function, i means accumulation starts from 1, and j means accumulation starts from 1.

[0104] Use the Sigmoid activation function to get the output value of the channel weight, where W1 is used for scaling and b1 is used for shifting s":

[0105] X′ K1 =σ[F C (s)]·X K1 =σ(W1s″+b1)

[0106] Among them, X K1 ' represents the output of the first unit, σ represents the activation function symbol, F C (s) represents the activation function weight.

[0107] To obtain the spatial weight, group normalization (GN) is used to obtain spatial statistical information, and the Sigmoid activation function is used to obtain the output value of the spatial weight:

[0108] X K2 '=σ[W2·GN(X K2 )+b2]·X K2

[0109] Among them, X K2 ' represents the output of the second unit, σ represents the activation function symbol, and b2 represents the displacement value.

[0110] In this embodiment, in order to improve the lightweight data features, a cross-scale feature fusion module CCFM (Cross-Scale Feature Fusion Module) is introduced to suppress background noise, and the target features are highlighted by explicitly modeling the interdependence between channels and adaptively recalibrating the feature response in the channel direction, so as to enhance the lightweight features and improve the model robustness. Figure 6As shown in the figure, the cross-scale feature fusion module CCFM consists of 2 1×1 convolutions and N repeated blocks (RepBlock). Since the real-time detection transformer RT-DETR can be scaled, the depth and width of the feature pyramid Hybrid Encoder are controlled by the number of repeated blocks in CCFM and the encoding dimension of the Encoder respectively. At the same time, the network backbone is adjusted accordingly to improve the accuracy.

[0111] In this embodiment, in order to remove redundant pixel information, the image information is processed through the convolution and pooling layers to obtain a small-resolution feature map, which will lead to the loss of fine-grained information. Therefore, the spatial-to-depth convolution SPD-Conv is introduced to replace the ordinary convolution and maximum pooling layers to solve the problem of fine-grained information loss. The spatial-to-depth convolution SPD-Conv structure, which consists of a spatial-to-depth layer SPD and a non-strided convolution layer, uses image conversion technology to downsample the feature maps inside and outside the CNN. Consider an intermediate feature map X of any size S×S×C1, and slice the sub-feature map sequence into:

[0112] f 0,0 =X[0:S:scale,0:S:scale]

[0113] f 1,0 =X[1:S:scale,0:S:scale]

[0114]

[0115] f scale-1,0 =X[scale-1:S:scale,0:S:scale]

[0116] Among them, f scale-1,0 represents the sub-map value, scale represents the scaling ratio, and S represents the convolutional layer size.

[0117] For any given feature map X, the submap f x,y The feature map X(i,j) is composed of all feature maps, i+j and j+y can be divided by the ratio. This design retains the discriminative feature information while using the down-sampled feature map, which brings significant accuracy improvement. For any given feature map X, the sub-map f x,y All feature maps form a feature map X(i,j), where i+j and j+y can be divided by the ratio, where i, j, and y represent the feature coordinate values ​​of the three dimensions respectively. This design retains the discriminative feature information while using the down-sampled feature map, which brings significant accuracy improvement.

[0118] In this embodiment, the training results of the Small-Goals-Yolov8 model are as follows: Figure 7 As shown in Table 1, after 100 iterations, the experimental results are shown in Table 1. The box value decreases through training rounds, from 1.4732 at the beginning to 0.43218 after 100 rounds of training. This shows that the training loss is low, and the model prediction box is basically close to the actual box; the cls value is 0.23742 after 100 rounds of training, indicating that the model has a good classification effect; the precision value indicates the recognition accuracy, which can reach 93.748% after 100 rounds of training; the recall value indicates the model recall rate, which is 89.712% after 100 rounds of training, indicating that the detection accuracy of positive samples is high; the mAP value is 93.095% after 100 rounds of training, indicating that the model has high detection accuracy.

[0119] Table 1

[0120]

[0121] In this embodiment, the trained Small-Goals-Yolov8 model is used to predict whether there are behaviors such as making phone calls, taking hands off the steering wheel, smoking, and eating.

[0122] S4. Based on the driving status extracted in step S2 and the prediction result in step S3, the driver's irregular driving behavior is warned and reminded to complete the detection of the driver's driving behavior.

[0123] In this embodiment, based on the results obtained in step S2 and step S3, it is determined and counted whether there is yawning, closing eyes for a long time, making phone calls, taking hands off the steering wheel, smoking, eating, etc. The detected irregular driving behavior is warned in the form of voice to remind the driver to drive safely.

Claims

1. A method for detecting a driver's driving behavior, characterized in that: The following steps are involved: S1. Obtain the safety equipment image of the driver while driving through the camera deployed in the vehicle unit, and enhance the safety equipment image to obtain an enhanced driver driving image; S2. Based on the enhanced driving image of the driver, the driver's facial feature points are detected using the retina algorithm Retina Face, and the driver's driving status is analyzed by extracting the driver's eye features and mouth features; S3. Build an improved Small-Goals-Yolov8 model and use the improved Small-Goals-Yolov8 model weights for prediction; S4. Based on the driving status extracted in step S2 and the prediction result in step S3, the driver's irregular driving behavior is warned and reminded to complete the detection of the driver's driving behavior.

2. The driver driving behavior detection method according to claim 1, characterized in that: The step S1 comprises the following steps: S101, obtaining an image of the driver's safety equipment while driving through a camera deployed in a vehicle unit; S102, transforming the safety equipment image through histogram equalization, and redistributing brightness values ​​of image pixels; S103, remapping pixel values ​​based on a grayscale histogram of the safety equipment image; S104, performing enhancement processing according to step S102-step S103 to obtain an enhanced driver driving image.

3. The driver driving behavior detection method according to claim 1, characterized in that: The step S2 comprises the following steps: S201, defining 150 frames as one round of fatigue assessment according to the enhanced driving image of the driver; S202, in a round of fatigue assessment, for eye features, the eye opening and closing states are determined by analyzing the eye states based on the aspect ratio, and the first fatigue level of the driver is assessed using the retina algorithm Retina Face; S203. In a round of fatigue assessment, for mouth features, whether there is yawning behavior is determined based on the opening and closing degree threshold, and the second fatigue level of the driver is assessed using the duty cycle algorithm.

4. The driver driving behavior detection method according to claim 3, characterized in that: The eye state analysis based on aspect ratio is used to determine the eye opening and closing state, and the retina algorithm Retina Face is used to evaluate the driver's fatigue level, which is specifically as follows: Set the EYE value and determine whether the EYE value is greater than a threshold value. If so, the eyes are treated as being in a closed state, and within a round of fatigue assessment, the first fatigue level of the driver is assessed according to the time taken for the eyes to be closed. Otherwise, the eyes are not in a closed state. The expression of the EYE value is as follows: EYE=(h1+h2) / (2×w) Where EYE represents the eye opening ratio, h1 and h2 represent the vertical distances of the eyelids on both sides of the pupil, and w represents the maximum horizontal width of the eye. The expression of the first fatigue level is as follows: Among them, perclos represents the first fatigue level, s represents the number of frames of eye closure, and S represents the total number of frames in a round of fatigue assessment.

5. The driver driving behavior detection method according to claim 3, characterized in that: The method is based on judging whether there is yawning behavior and using the load rate algorithm Duty Cycle to evaluate the second fatigue level of the driver, which is specifically: The MAR value is used to determine whether there is yawning behavior. If so, the second fatigue level of the driver is evaluated according to the time occupied by yawning within a round of fatigue assessment. Otherwise, there is no yawning behavior. The expression of the MAR value is as follows: MAR=(w1+w2) / (2×s') Where MAR represents the mouth opening and closing ratio, w1 and w2 represent the vertical distances on both sides of the lips, and s' represents the maximum horizontal width of the mouth; The expression of the second fatigue level is as follows: Among them, yawn represents the proportion of yawning frames, n represents the number of yawning frames, and N represents the total number of frames in a round of fatigue assessment.

6. The driver driving behavior detection method according to claim 1, characterized in that: The structure of the Small-Goals-Yolov8 model is as follows: a single-layer regression structure SA-ones is introduced on the framework of Yolov8, a cross-scale feature fusion module CCFM is introduced on the shallow layer of the network, a deep convolution module SPD-Conv is used on the convolution layer and the pooling layer, and the network neck is connected layer by layer through multi-scale feature fusion, and feature maps from different stages of the backbone are fused.

7. The driver driving behavior detection method according to claim 6, characterized in that: The single-layer regression structure SA-ones includes a first unit with a step size of 1 and a second unit with a step size of 2, and the main branches of the first unit and the second unit both include a first convolutional layer, a first depth-separable convolutional layer, and a second convolutional layer connected in sequence; The input end of the first unit performs channel segmentation on the received image, and the side branch of the first unit splices the channel segmented image with the main branch image in terms of dimension; The image received at the input end of the second unit is processed by the main branch of the second unit and the side branch of the second unit respectively, and the processing results are spliced ​​through the main branch of the second unit and the side branch of the second unit, and the side branch of the second unit includes a second depthwise separable convolutional layer and a third convolutional layer connected in sequence; The concatenation result of the first unit and the second unit uses channel shuffling to transmit information.

8. The driver driving behavior detection method according to claim 6, characterized in that: The cross-scale feature fusion module CCFM includes a fourth convolutional layer, a fifth convolutional layer and a repetitive block RepBlock connected to the fifth convolutional layer, and the fourth convolutional layer is spliced ​​with the repetitive block RepBlock; The SPD-Conv module includes sequential connections from spatial to depth layers and non-strided convolutional layers.

9. The driver driving behavior detection method according to claim 1, characterized in that: The step S4 is specifically as follows: According to the driving status extracted in step S2 and the prediction result in step S3, alarms and reminders are issued when there are yawning, long-term eye closing, phone calls, hands off the steering wheel, smoking, and eating behaviors, thereby completing the detection of the driver's driving behavior.

Images