Automobile driver driving behavior identification method and device

Abstract

The invention provides an automobile driver driving behavior identification method and device. The method comprises the following steps: constructing an original identification model comprising an encoder based on a depth separable convolutional network and a decoder based on a compression excitation network; training the original recognition model through the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model; deploying the driver distraction identification model to a vehicle-mounted hardware platform of a target vehicle; in the process that a driver drives the target vehicle, video stream data collected by a cabin camera on the target vehicle is obtained; performing driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result; and determining the current driving behavior of the driver according to the identification result. Therefore, the method and the device can identify the driving behavior of the driver in real time, are good in accuracy and stability, do not need the driver to wear detection equipment, and ensure the driving experience and comfort of the driver.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and more specifically, to a method and device for identifying driving behavior of a car driver. Background Art

[0002] At present, the development of new energy vehicles has led to the rapid progress of autonomous driving technology. Its highly electronic design, such as advanced electronic stability control and electronic power steering systems, provides a solid foundation for autonomous driving. Currently, autonomous driving is at the L2 level, requiring the driver to remain alert and ready to take over the vehicle at any time. However, distracted driving by the driver increases the risk of accidents, and the reasons for distraction include using mobile phones and talking. Traditional distracted driving identification methods are usually based on monitoring the driver's physiological signals, such as EEG signals and ECG signals. This method can provide relatively accurate distracted driving identification results, but it usually requires the driver to wear relevant instruments and equipment, which not only brings additional burden and inconvenience to the driver, but may also affect the driver's driving experience and comfort. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method and device for identifying the driving behavior of a car driver, which can identify the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0004] The first aspect of the present application provides a method for identifying a driving behavior of a vehicle driver, comprising: Obtain training configuration data and driving behavior sample data of real driving scenarios; Constructing an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; The original recognition model is trained by using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model; Deploying the driver distraction recognition model onto an onboard hardware platform of a target vehicle; While the driver is driving the target vehicle, obtaining video stream data collected by a cockpit camera on the target vehicle; Perform driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result; The current driving behavior of the driver is determined according to the recognition result.

[0005] In the above implementation process, the method can identify the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0006] Furthermore, the acquisition of training configuration data and driving behavior sample data of real driving scenarios includes: Obtain training configuration data and raw driving behavior data of real driving scenarios; The driving behavior original data is subjected to driving behavior category labeling processing to obtain driving behavior sample data.

[0007] Furthermore, the training configuration data includes at least a data batch, a loss function, a model training algorithm, a gradient update algorithm, and a training termination condition for each training; The loss function is a cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination condition includes at least the number of iterations and the early termination loss function value; Among them, the driving behavior categories include safe driving behavior, smoking behavior, drinking water behavior, phone call behavior, text message behavior and fatigue driving behavior.

[0008] Furthermore, the constructing of the original recognition model includes: Construct normalization layers, channel-wise convolution layers, point-wise convolution layers, and pooling layers; Constructing a depth-separable convolutional coding layer according to the normalization layer, the channel-by-channel convolution layer, the point-by-point convolution layer and the pooling layer; The depth-wise separable convolutional coding layers are stacked to construct an encoder; Constructing a decoder based on a compressed excitation network; wherein the decoder comprises at least a compressed excitation module and a forward layer; the compressed excitation module comprises a global average pooling layer, a first fully connected layer and a second fully connected layer; According to the encoder and the decoder, an original recognition model is constructed.

[0009] Furthermore, the method further comprises: Determine whether the current driving behavior is distracted driving behavior; wherein the distracted driving behavior is one of smoking, drinking, making phone calls, sending text messages, and fatigue driving; If yes, a distracted driving warning prompt message is outputted through the vehicle-mounted speaker of the target vehicle.

[0010] Further, after outputting the distracted driving warning prompt information through the vehicle-mounted speaker of the target vehicle, the method further includes: After a preset time period, re-determining whether the driver still has distracted driving behavior based on the driver distraction recognition model; If yes, the target vehicle is switched to an automatic control mode, and the target vehicle is automatically controlled based on the automatic control mode.

[0011] Furthermore, the method further comprises: When the current driving behavior is fatigue driving behavior, determining whether a destination is preset; If yes, automatically controlling the target vehicle to travel to the destination; If not, a safe parking position is determined according to the current position of the target vehicle, and the target vehicle is automatically controlled to travel to the safe parking position to park, and the target vehicle is controlled to perform a double flash operation.

[0012] A second aspect of the present application provides a vehicle driver driving behavior recognition device, the vehicle driver driving behavior recognition device comprising: A first acquisition unit, used to acquire training configuration data and driving behavior sample data of a real driving scenario; A construction unit, used to construct an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; A training unit, configured to train the original recognition model using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model; A deployment unit, used to deploy the driver distraction recognition model onto an on-board hardware platform of a target vehicle; A second acquisition unit is used to acquire video stream data collected by a cockpit camera on the target vehicle while the driver is driving the target vehicle; an identification unit, configured to perform driving behavior identification processing according to the driver distraction identification model and the video stream data to obtain an identification result; A determination unit is used to determine the current driving behavior of the driver according to the recognition result.

[0013] Furthermore, the first acquiring unit includes: An acquisition subunit, used to acquire training configuration data and raw driving behavior data of real driving scenarios; The labeling subunit is used to label the driving behavior original data according to the driving behavior category to obtain driving behavior sample data.

[0014] Furthermore, the training configuration data includes at least a data batch, a loss function, a model training algorithm, a gradient update algorithm, and a training termination condition for each training; The loss function is a cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination condition includes at least the number of iterations and the early termination loss function value; Among them, the driving behavior categories include safe driving behavior, smoking behavior, drinking water behavior, phone call behavior, text message behavior and fatigue driving behavior.

[0015] Furthermore, the construction unit comprises: Construct subunits for constructing normalization layers, channel-wise convolution layers, point-wise convolution layers, and pooling layers; The construction subunit is further used to construct a depth-separable convolutional coding layer according to the normalization layer, the channel-by-channel convolutional layer, the point-by-point convolutional layer and the pooling layer; A stacking subunit, used for stacking the depth-separable convolutional coding layers to construct an encoder; The construction subunit is further used to construct a decoder based on the compression excitation network; wherein the decoder at least includes a compression excitation module and a forward layer; the compression excitation module includes a global average pooling layer, a first fully connected layer and a second fully connected layer; The construction subunit is further used to construct an original recognition model according to the encoder and the decoder.

[0016] Furthermore, the vehicle driver driving behavior recognition device further comprises: A judgment unit, used to judge whether the current driving behavior is distracted driving behavior; wherein the distracted driving behavior is one of smoking behavior, drinking water behavior, making phone calls, sending text messages and fatigue driving behavior; The output unit is used to output distracted driving warning prompt information through the vehicle-mounted speaker of the target vehicle when the current driving behavior is distracted driving behavior.

[0017] Furthermore, the vehicle driver driving behavior recognition device further comprises: The judgment unit is further configured to re-judge whether the driver still has distracted driving behavior according to the driver distraction recognition model after the output unit outputs the distracted driving warning prompt information through the vehicle-mounted speaker of the target vehicle and after a preset time period; The control unit is used to switch the target vehicle to an automatic control mode when the driver still has distracted driving behavior, and automatically control the target vehicle based on the automatic control mode.

[0018] Furthermore, the vehicle driver driving behavior recognition device further comprises: A judging unit, configured to judge whether a destination is preset when the current driving behavior is fatigue driving behavior; A control unit, for automatically controlling the target vehicle to travel to the destination when a destination is preset; The control unit is further used to determine a safe parking position according to the current position of the target vehicle, automatically control the target vehicle to drive to the safe parking position to park, and control the target vehicle to turn on the double flash operation.

[0019] The third aspect of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for identifying the driving behavior of a car driver as described in any one of the first aspect of the present application.

[0020] A fourth aspect of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the method for identifying the driving behavior of a vehicle driver described in any one of the first aspect of the present application is executed.

[0021] The beneficial effects of this application are: by collecting large-scale real driving scene data and performing detailed behavior annotation, a recognition model that is closer to the actual application scenario can be trained. Such a model can better adapt to the diversity of real road conditions and driving behaviors, and improve the performance and generalization ability of the model in actual scenarios; It is also possible to use channel-by-channel convolution and point-by-point convolution through a lightweight encoder to reduce the number of parameters and computational complexity while maintaining model performance. Among them, channel-by-channel convolution reduces the number of parameters of the convolution kernel, while point-by-point convolution introduces cross-channel information interaction and enhances feature representation capabilities. This makes the model lightweight, which is conducive to reducing the number of parameters and computational complexity, and is suitable for low-latency distraction recognition and resource-constrained devices; It is also possible to introduce a compression-excitation module into the compression-excitation decoder to adaptively adjust the weights between feature channels, further improving the performance of the model. The module can learn the importance of each feature channel and selectively enhance the focus on important features while suppressing the reliance on unimportant features. It can be seen that this selective enhancement improves the model's discrimination and generalization performance, while reducing the number of parameters and the complexity of the model through the global average pooling operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flow chart of a method for identifying a driving behavior of a vehicle driver provided in an embodiment of the present application; Figure 2 A schematic diagram of the main design process of a method for identifying the driving behavior of a vehicle driver provided in an embodiment of the present application; Figure 3 A flowchart of another method for identifying a driving behavior of a vehicle driver provided in an embodiment of the present application; Figure 4 An overall block diagram of a method for identifying a vehicle driver's driving behavior provided in an embodiment of the present application; Figure 5 A schematic diagram of the structure of a vehicle driver driving behavior recognition device provided in an embodiment of the present application; Figure 6 A schematic diagram of the structure of another device for identifying driving behavior of a car driver provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0025] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0026] Example 1 Please see Figure 1 , Figure 1 The following is a flow chart of a method for identifying a driving behavior of a vehicle driver provided in this embodiment. The method for identifying a driving behavior of a vehicle driver includes: S101. Acquire training configuration data and driving behavior sample data of real driving scenarios.

[0027] In this embodiment, the training configuration data includes at least the data batch, loss function, model training algorithm, gradient update algorithm and training termination condition for each training; The loss function is the cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination conditions include at least the number of iterations and the early termination loss function value.

[0028] S102: construct an original recognition model.

[0029] In this embodiment, the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compressed excitation network.

[0030] In this embodiment, in the traditional convolutional neural network, each convolution kernel needs to perform convolution operations on all channels of the input image, so the number of parameters of each convolution kernel is large. Depthwise separable convolution decomposes the traditional convolution operation into two steps: channel-by-channel convolution and point-by-point convolution. Channel-by-channel convolution only applies one convolution kernel to each input channel, while point-by-point convolution is used to integrate the output of channel-by-channel convolution between channels. This decomposition can greatly reduce the number of parameters, thereby reducing the complexity of the model.

[0031] In this embodiment, the compression-excitation network is used to enhance the feature representation capability in the convolutional neural network. Its role is to adjust the channel response of the feature map by learning adaptive channel weights to enhance the focus on important features and suppress unimportant features. It can reduce the number of parameters and the amount of computation in the network. By learning channel weights, the channel dimension of the feature map can be reduced, thereby reducing the number of parameters and the amount of computation in subsequent layers. This is very useful for model compression and acceleration in lightweight models and resource-constrained environments.

[0032] Specifically, the compression-excitation network can adaptively adjust the weight of each channel by learning the inter-channel correlation of the feature map, thereby improving the performance of the network on specific tasks. By first compressing the channel dimension of the feature map and then stimulating the feature response of each channel, the compression-excitation network can significantly improve the representation and generalization capabilities of the network while reducing the computational complexity of the model.

[0033] S103: Train the original recognition model using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model.

[0034] In this embodiment, the method can construct a novel encoder-decoder new energy vehicle driver distraction recognition model based on the above encoder and decoder, and train the driver distraction recognition model by using supervised learning by inputting the training set annotated by S1 into the model.

[0035] S104: deploying the driver distraction recognition model onto the on-board hardware platform of the target vehicle.

[0036] S105. While the driver is driving the target vehicle, obtain video stream data collected by a cockpit camera on the target vehicle.

[0037] S106: Perform driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result.

[0038] S107: Determine the driver's current driving behavior according to the recognition result.

[0039] Please see Figure 2 , Figure 2 The main design process of a method for identifying driving behavior of a car driver is shown, wherein the method includes the training process of the new energy vehicle driver distracted driving identification model and the application process of the new energy vehicle driver distracted driving identification model.

[0040] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0041] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0042] It can be seen that the method for identifying the driving behavior of a car driver described in this embodiment can identify the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0043] Example 2 Please see Figure 3 , Figure 3 The following is a flow chart of a method for identifying a driving behavior of a vehicle driver provided in this embodiment. The method for identifying a driving behavior of a vehicle driver includes: S201. Acquire training configuration data and original driving behavior data of a real driving scenario.

[0044] In this embodiment, the original driving behavior data is a large-scale data set, which usually includes drivers of different genders and ages and test roads in different scenarios.

[0045] In this embodiment, the training data set is derived from the high-resolution camera recording of the front cockpit of a new energy vehicle, which uses a Hikvision high-definition camera to ensure that more accurate driver behavior data can be obtained. The data set collection process is designed to simulate real driving scenarios to provide a wide range of data sets for training and evaluating driver distraction recognition models.

[0046] S202: Perform driving behavior category labeling processing on the original driving behavior data to obtain driving behavior sample data.

[0047] In this embodiment, the training configuration data includes at least the data batch, loss function, model training algorithm, gradient update algorithm and training termination condition for each training; The loss function is the cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination conditions include at least the number of iterations and the early termination loss function value; Among them, driving behavior categories include safe driving behavior, smoking behavior, drinking water behavior, phone call behavior, text message behavior and fatigue driving behavior.

[0048] For example, during the collection process, the researchers drove vehicles equipped with collection equipment to cover different types of roads, including urban streets, highways, and rural roads. More than 2,000 volunteer drivers participated at the same time, with a male-female ratio of 6:4 and an age range of 20 to 60 years old. The driving behaviors collected included: safe driving, smoking, drinking water, making phone calls, sending messages, and fatigue driving. Safe driving is defined as the driver keeping both hands on the steering wheel and keeping his eyes focused while driving the vehicle. This helps ensure that the dataset has rich scene diversity to better train the driver distraction recognition model to adapt to various environments.

[0049] The final collected data includes a 3-channel image U with a resolution of 1280*720 pixels from the front camera of the cockpit. （H ×W×C） , where H is height, W is width, and C is the number of channels. The corresponding labels are 6 types of driving behaviors. For the convenience of training, they are represented and labeled with one-hot vectors. For example, safe driving is represented as [1, 0, 0, 0, 0, 0]. During training, all data are shuffled and the training set and test set are divided into 7:3.

[0050] S203, construct a normalization layer, a channel-by-channel convolution layer, a point-by-point convolution layer, and a pooling layer.

[0051] S204, constructing a depth-separable convolutional coding layer according to the normalization layer, the channel-by-channel convolution layer, the point-by-point convolution layer, and the pooling layer.

[0052] S205, stacking the depth-wise separable convolutional coding layers to construct an encoder.

[0053] In this embodiment, the encoder design based on the deep separable convolutional neural network includes the following aspects: ① Standardization layer: Standardization can limit the range of input data to a smaller interval, which helps to accelerate the convergence speed of the model. Because during the training process, if the range of input data is too large, it will cause gradient explosion or gradient disappearance problems, affecting the training effect and speed of the model. In addition, standardization can reduce the swing in the gradient descent process and make the optimization process more stable. When the range of input data is limited to a smaller interval, the gradient update is smoother, reducing the instability of the optimization algorithm when updating parameters. For the input image U （H×W×C） , the standardization operation is expressed as follows: (1) Among them, u hwc Represents the original image U （H×W×C） The pixel value of channel c at position (h, w), The converted image The pixel value of channel c at position (h, w) is scaled to the range [0, 1] by dividing the actual pixel value by the maximum pixel value of 255.

[0054] ② Channel-by-channel convolution layer: Unlike ordinary convolution, channel-by-channel convolution uses only one independent convolution kernel to process each channel of each input image. Therefore, after channel-by-channel convolution, the number of output channels is equal to the number of input channels. Since each input channel is convolved with only one independent convolution kernel, the number of parameters for channel-by-channel convolution is small. Since the number of parameters for channel-by-channel convolution is small, the amount of calculation is also small. Each input channel is convolved with only one independent convolution kernel, so the amount of calculation is only related to the input image size and the number of input channels, and has nothing to do with the number of output channels. The calculation process is as follows: (2) Among them, O 1 represents the first layer channel-by-channel convolution output, Then it represents the corresponding convolution kernel, Represents the RELU activation function. Due to the channel-by-channel convolution, O 1 The number of channels of is consistent with the number of channels of U ̄.

[0055] ③ Point-by-point convolution layer: Point-by-point convolution can achieve cross-channel information interaction by performing convolution operations on the channel dimension. This interaction can help the model capture the correlation and dependency between different channels, thereby improving the feature representation capability. By introducing cross-channel information interaction, the model can better learn the combined features between channels, making the feature representation richer and more expressive. Its expression is as follows: (3) here represents the output of the first point-by-point convolutional layer, Represents a 1×1 convolution kernel.

[0056] ④Pooling layer: The pooling layer can reduce the size of the feature map by performing aggregating operations on the input feature map within the region, thereby reducing the computational complexity of the model. This downsampling operation helps to reduce the number of model parameters and memory consumption, while improving the computational efficiency of the model. In addition, the pooling layer can help the model extract more abstract and high-level features. By aggregating the features of the local area, the pooling layer can retain the most significant features in the input feature map and has a certain robustness to slight changes in spatial position. This helps the model better understand and capture the important features of the image.

[0057] (4) in, It represents the output of the first pooling layer, Z and S represent the size and step size of the pooling kernel, respectively. Maximum pooling is used here to learn significant features.

[0058] ⑤ Encoder stacking: The above depth-wise separable convolutional coding layer can be stacked N times, that is, after building N such encoder blocks, the subsequent stacking no longer requires a normalization layer. Finally, the output of the last encoder block is As the output feature representation of the entire encoder, it is sent to the decoder.

[0059] S206. Construct a decoder based on the compression-excitation network.

[0060] In this embodiment, the decoder includes at least a compression excitation module and a forward layer; the compression excitation module includes a global average pooling layer, a first fully connected layer and a second fully connected layer.

[0061] S207: Construct an original recognition model based on the encoder and the decoder.

[0062] In this embodiment, the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compressed excitation network.

[0063] In this embodiment, the design of a decoder based on a compression-excitation network includes the following steps: ① Compression-excitation layer: First, the input Using global average pooling of channels, the global spatial information is compressed into a channel descriptor, and the I×J×K feature map containing global information is directly compressed into a 1×1×K feature vector Z, which is expressed as follows: (5) Among them, Z k represents the kth value of Z, Represents input The value at position (i, j) in channel k.

[0064] In the compression-excitation module, the role of subsequent fully connected layers is to adaptively scale and reweight the input features. First, the two fully connected layers are used to learn the relationship between channels. The first fully connected layer reduces the dimensionality of the input features and maps them to a smaller dimensional space. This helps reduce the number of parameters and computational complexity. Then, the second fully connected layer maps the reduced dimensionality features back to the original dimensional space.

[0065] The first fully connected layer compresses K channels into K / r channels to reduce the amount of computation, where r refers to the compression ratio. Then, a RELU nonlinear activation layer is passed, and the second fully connected layer restores the number of channels to K channels. The weight S is obtained through Sigmoid activation. The dimension of S is 1×1×C, which is used to characterize the weights of the K feature maps in the feature map O ̃.

[0066] (6) Among them, W 1 , W 2 represents the learnable weight matrix, and δ represents the Sigmoid activation function.

[0067] The last step is the reweighting operation, which adds the attention weights obtained above to the features of each channel: (7) in, Representation output The kth channel of .

[0068] ②Forward layer: The above operation obtains is a three-dimensional vector. To facilitate the subsequent calculation of the probability of each driving behavior, Flattened to a one-dimensional vector, we get , and then use a fully connected layer to scale its length to 6, corresponding to 6 driving behaviors: (8) Among them, Y represents the output of the fully connected layer, W Y and a represent the learnable weight matrices and biases.

[0069] Finally, the Softmax activation function is used to compress Y into another real-valued vector, where each element ranges between 0 and 1, and the sum of all elements is 1. That is, Y is converted into a probability distribution: (9) Represents the final probability output vector The ith value of .

[0070] S208. Train the original recognition model using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model.

[0071] In this embodiment, the model training process based on the lightweight convolutional neural network may include: ① Data preprocessing: set the batch size for each training.

[0072] ②Define the loss function: The cross entropy loss function is widely used in classification tasks because it can effectively measure the difference between the probability distribution of the model output and the true label, and can make the model better fit the training data during the optimization process. The mathematical expression of the cross entropy loss function is as follows: (10) Where p is the true label distribution (a one-hot encoded vector), q is the model’s predicted probability distribution, and i represents the index of the category.

[0073] ③ Gradient update: Adam is a commonly used adaptive learning rate optimization algorithm that combines the ideas of momentum gradient descent and RMSProp algorithm. The following is the formula of Adam optimization algorithm: Let t be the current iteration number, η be the learning rate, β 1 and β 2 is the decay rate parameter, and ϵ is a small constant. First, calculate the first-order moment estimate m and the second-order moment estimate v of the gradient: (11) (12) Among them, g t is the gradient of the current iteration, and then the first-order moment estimate and the second-order moment estimate are bias corrected: (13) (14) Finally update the model parameters: (15) θ t ,θ （t-1） are the model parameters before and after updating respectively.

[0074] ④Training termination: Set the number of iterations and the early termination loss function value. When the model reaches the set number of iterations or reaches the set loss function value in advance, the training stops.

[0075] ⑤Model saving: Save the final model parameters at the end of training as the deployment model for in-vehicle driver distraction recognition.

[0076] S209: deploying the driver distraction recognition model onto the on-board hardware platform of the target vehicle.

[0077] S210. While the driver is driving the target vehicle, obtain video stream data collected by a cockpit camera on the target vehicle.

[0078] S211, performing driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result.

[0079] S212: Determine the driver's current driving behavior according to the recognition result.

[0080] As an optional implementation, when the current driving behavior is fatigue driving behavior, it is determined whether a destination is preset; If yes, the target vehicle is automatically controlled to travel to the destination; If not, a safe parking position is determined according to the current position of the target vehicle, and the target vehicle is automatically controlled to travel to the safe parking position to park, and the target vehicle is controlled to turn on the double flash operation.

[0081] S213: Determine whether the current driving behavior is distracted driving behavior. If so, execute step S214; if not, end this process.

[0082] In this embodiment, the distracted driving behavior is one of smoking, drinking, making phone calls, sending messages, and fatigue driving.

[0083] S214: Output distracted driving warning information through the onboard speaker of the target vehicle.

[0084] S215. After the preset time period, re-determine whether the driver still has distracted driving behavior based on the driver distraction recognition model. If so, execute step S216; if not, end this process.

[0085] S216: Switch the target vehicle to an automatic control mode, and automatically control the target vehicle based on the automatic control mode.

[0086] In this embodiment, the method can deploy the trained on-board driver distraction recognition model to the on-board hardware platform. The model reads the video stream data collected by the cockpit camera to obtain the probability of the driver being in various driving behaviors. The actual driving behavior of the current driver is the behavior with the largest probability value. When the driver is detected to be in a safe driving behavior, the vehicle does nothing. When it is detected that the driver is in the remaining 5 distracted driving behaviors, the vehicle first mobilizes the speaker to warn the driver. If the driver is still distracted driving, the new energy vehicle will temporarily take over the control of the vehicle, and the vehicle will autonomously choose to change lanes, slow down, and brake. When it is detected that the driver is in fatigue driving for a long time, the vehicle autonomously drives to the destination according to the navigation. If the destination is not set in advance, it will choose to slow down, then choose a safe location to park and turn on the double flash.

[0087] Please see Figure 4 , Figure 4 The overall block diagram of a method for identifying a vehicle driver's driving behavior is shown.

[0088] In this embodiment, the innovative point of the method is: ① Data collection and labeling: We used the front camera of new energy vehicles to obtain large-scale real driving scene data and performed detailed behavior labeling. This helps to train a recognition model that is closer to the actual application scenario. By using real driving scene data, we can better simulate actual road conditions and driving behaviors. These data contain rich background information, lighting conditions, vehicle driving status and other factors, and can better capture the diversity and complexity of the real world compared to synthetic data or restricted data sets.

[0089] ② Lightweight encoder design: The lightweight encoder design is to reduce the number of parameters and computational complexity while maintaining model performance to achieve low-latency distraction recognition. A separate convolution operation is performed on each input channel through channel-by-channel convolution. Since channel-by-channel convolution is an independent convolution operation on each channel, the number of parameters of the convolution kernel can be greatly reduced. Point-by-point convolution uses a 1x1 convolution kernel to perform a convolution operation on the output of the channel-by-channel convolution. Point-by-point convolution is a convolution performed on the channel dimension, mapping the output feature map of the depth convolution to the final output feature map. The role of point-by-point convolution is to introduce cross-channel information interaction, thereby enhancing the feature representation capability.

[0090] ③Compression-excitation decoder: The compression-excitation module is introduced to adaptively adjust the weights between feature channels, further improving the performance of the model. Through the compression-excitation module, the model can automatically learn the importance of each feature channel. This enables the model to selectively enhance the focus on important features and suppress dependence on unimportant features. This selective enhancement can improve the model's discrimination ability and generalization performance. In addition, the compression-excitation module uses a global average pooling operation in the compression stage to compress the feature map of each channel into a scalar. This operation greatly reduces the number of parameters in the model and effectively reduces the complexity of the model.

[0091] In this embodiment, the execution subject of the method may be a computing device such as a computer or a server, and no limitation is made in this embodiment.

[0092] In this embodiment, the execution subject of the method may also be a smart device such as a smart phone, a tablet computer, etc., which is not limited in this embodiment.

[0093] It can be seen that the method for identifying the driving behavior of a car driver described in this embodiment can identify the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0094] Example 3 Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of a vehicle driver driving behavior recognition device provided in this embodiment. Figure 5 As shown, the vehicle driver driving behavior recognition device comprises: A first acquisition unit 310 is used to acquire training configuration data and driving behavior sample data of a real driving scenario; A construction unit 320 is used to construct an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; A training unit 330, configured to train the original recognition model using driving behavior sample data and training configuration data to obtain a driver distraction recognition model; A deployment unit 340, for deploying the driver distraction recognition model onto an onboard hardware platform of a target vehicle; The second acquisition unit 350 is used to acquire the video stream data collected by the cockpit camera of the target vehicle while the driver is driving the target vehicle; The recognition unit 360 is used to perform driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result; The determination unit 370 is used to determine the current driving behavior of the driver according to the recognition result.

[0095] In this embodiment, the explanation of the vehicle driver driving behavior recognition device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.

[0096] It can be seen that the automobile driver driving behavior recognition device described in this embodiment can recognize the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0097] Example 4 Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a vehicle driver driving behavior recognition device provided in this embodiment. Figure 6 As shown, the vehicle driver driving behavior recognition device comprises: A first acquisition unit 310 is used to acquire training configuration data and driving behavior sample data of a real driving scenario; A construction unit 320 is used to construct an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; A training unit 330, configured to train the original recognition model using driving behavior sample data and training configuration data to obtain a driver distraction recognition model; A deployment unit 340, for deploying the driver distraction recognition model onto an onboard hardware platform of a target vehicle; The second acquisition unit 350 is used to acquire the video stream data collected by the cockpit camera of the target vehicle while the driver is driving the target vehicle; The recognition unit 360 is used to perform driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result; The determination unit 370 is used to determine the current driving behavior of the driver according to the recognition result.

[0098] As an optional implementation manner, the first acquiring unit 310 includes: An acquisition subunit 311 is used to acquire training configuration data and original driving behavior data of a real driving scenario; The labeling subunit 312 is used to label the original driving behavior data with driving behavior categories to obtain driving behavior sample data.

[0099] In this embodiment, the training configuration data includes at least the data batch, loss function, model training algorithm, gradient update algorithm and training termination condition for each training; The loss function is the cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination conditions include at least the number of iterations and the early termination loss function value; Among them, driving behavior categories include safe driving behavior, smoking behavior, drinking water behavior, phone call behavior, text message behavior and fatigue driving behavior.

[0100] As an optional implementation, the construction unit 320 includes: A construction subunit 321 is used to construct a normalization layer, a channel-by-channel convolution layer, a point-by-point convolution layer, and a pooling layer; The construction subunit 321 is further used to construct a depth-separable convolutional coding layer according to the normalization layer, the channel-by-channel convolution layer, the point-by-point convolution layer and the pooling layer; A stacking subunit 322, used for stacking the depth-separable convolutional coding layers to construct an encoder; The construction subunit 321 is further used to construct a decoder based on the compression excitation network; wherein the decoder includes at least a compression excitation module and a forward layer; the compression excitation module includes a global average pooling layer, a first fully connected layer and a second fully connected layer; The construction subunit 321 is also used to construct an original recognition model according to the encoder and the decoder.

[0101] As an optional implementation, the vehicle driver driving behavior recognition device further includes: The judging unit 380 is used to judge whether the current driving behavior is distracted driving behavior; wherein the distracted driving behavior is one of smoking behavior, drinking behavior, making phone calls, sending messages and fatigue driving behavior; The output unit 390 is used to output distracted driving warning information through the vehicle-mounted speaker of the target vehicle when the current driving behavior is distracted driving behavior.

[0102] As an optional implementation, the vehicle driver driving behavior recognition device further includes: The judgment unit 380 is further configured to re-judge whether the driver still has distracted driving behavior according to the driver distraction recognition model after the output unit 390 outputs the distracted driving warning prompt information through the vehicle-mounted speaker of the target vehicle and after a preset time period; The control unit 400 is used to switch the target vehicle to an automatic control mode when the driver still has distracted driving behavior, and automatically control the target vehicle based on the automatic control mode.

[0103] As an optional implementation, the vehicle driver driving behavior recognition device further includes: A judgment unit 380, for judging whether a destination is preset when the current driving behavior is fatigue driving behavior; The control unit 400 is used to automatically control the target vehicle to travel to the destination when a destination is preset; The control unit 400 is also used to determine a safe parking position according to the current position of the target vehicle, automatically control the target vehicle to drive to the safe parking position to park, and control the target vehicle to turn on the double flash operation.

[0104] In this embodiment, the explanation of the vehicle driver driving behavior recognition device can refer to the description in Embodiment 1 or Embodiment 2, and will not be further elaborated in this embodiment.

[0105] It can be seen that the automobile driver driving behavior recognition device described in this embodiment can recognize the driver's driving behavior in real time with good accuracy and stability, and does not require the driver to wear detection equipment, thereby ensuring the driver's driving experience and comfort.

[0106] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for identifying the driving behavior of a car driver in Embodiment 1 or Embodiment 2 of the present application.

[0107] An embodiment of the present application provides a computer-readable storage medium storing computer program instructions. When the computer program instructions are read and executed by a processor, the method for identifying the driving behavior of a car driver in Embodiment 1 or Embodiment 2 of the present application is executed.

[0108] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0109] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0110] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program codes.

[0111] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0112] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0113] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A method for identifying driving behavior of a car driver, characterized in that: include: Obtain training configuration data and driving behavior sample data of real driving scenarios; Constructing an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; The original recognition model is trained by using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model; Deploying the driver distraction recognition model onto an onboard hardware platform of a target vehicle; While the driver is driving the target vehicle, obtaining video stream data collected by a cockpit camera on the target vehicle; Perform driving behavior recognition processing according to the driver distraction recognition model and the video stream data to obtain a recognition result; The current driving behavior of the driver is determined according to the recognition result.

2. The method for identifying the driving behavior of a vehicle driver according to claim 1, characterized in that: The step of acquiring training configuration data and driving behavior sample data of real driving scenarios includes: Obtain training configuration data and raw driving behavior data of real driving scenarios; The driving behavior original data is subjected to driving behavior category labeling processing to obtain driving behavior sample data.

3. The method for identifying the driving behavior of a vehicle driver according to claim 2, characterized in that: The training configuration data includes at least the data batch, loss function, model training algorithm, gradient update algorithm and training termination condition for each training; The loss function is a cross entropy loss function; The model training algorithm is a supervised learning training algorithm; The gradient update algorithm is the Adam adaptive learning rate optimization algorithm; The training termination condition includes at least the number of iterations and the early termination loss function value; Among them, the driving behavior categories include safe driving behavior, smoking behavior, drinking water behavior, phone call behavior, text message behavior and fatigue driving behavior.

4. The method for identifying the driving behavior of a vehicle driver according to claim 1, characterized in that: The constructing of the original recognition model comprises: Construct normalization layers, channel-wise convolution layers, point-wise convolution layers, and pooling layers; Constructing a depth-separable convolutional coding layer according to the normalization layer, the channel-by-channel convolution layer, the point-by-point convolution layer and the pooling layer; The depth-wise separable convolutional coding layers are stacked to construct an encoder; Constructing a decoder based on a compressed excitation network; wherein the decoder comprises at least a compressed excitation module and a forward layer; the compressed excitation module comprises a global average pooling layer, a first fully connected layer and a second fully connected layer; According to the encoder and the decoder, an original recognition model is constructed.

5. The method for identifying the driving behavior of a vehicle driver according to claim 1, characterized in that: The method further comprises: Determine whether the current driving behavior is distracted driving behavior; wherein the distracted driving behavior is one of smoking, drinking, making phone calls, sending text messages, and fatigue driving; If yes, a distracted driving warning prompt message is outputted through the vehicle-mounted speaker of the target vehicle.

6. The method for identifying the driving behavior of a vehicle driver according to claim 4, characterized in that: After outputting the distracted driving warning prompt information through the vehicle-mounted speaker of the target vehicle, the method further includes: After a preset time period, re-determining whether the driver still has distracted driving behavior based on the driver distraction recognition model; If yes, the target vehicle is switched to an automatic control mode, and the target vehicle is automatically controlled based on the automatic control mode.

7. The method for identifying the driving behavior of a vehicle driver according to claim 1, characterized in that: The method further comprises: When the current driving behavior is fatigue driving behavior, determining whether a destination is preset; If yes, automatically controlling the target vehicle to travel to the destination; If not, a safe parking position is determined according to the current position of the target vehicle, and the target vehicle is automatically controlled to travel to the safe parking position to park, and the target vehicle is controlled to perform a double flash operation.

8. A driving behavior recognition device for a car driver, characterized in that: The vehicle driver driving behavior recognition device comprises: A first acquisition unit, used to acquire training configuration data and driving behavior sample data of a real driving scenario; A construction unit, used to construct an original recognition model; wherein the original recognition model includes an encoder based on a deep separable convolutional network and a decoder based on a compression excitation network; A training unit, configured to train the original recognition model using the driving behavior sample data and the training configuration data to obtain a driver distraction recognition model; A deployment unit, used to deploy the driver distraction recognition model onto an on-board hardware platform of a target vehicle; A second acquisition unit is used to acquire video stream data collected by a cockpit camera on the target vehicle while the driver is driving the target vehicle; an identification unit, configured to perform driving behavior identification processing according to the driver distraction identification model and the video stream data to obtain an identification result; A determination unit is used to determine the current driving behavior of the driver according to the recognition result.

9. An electronic device, characterized in that: The electronic device comprises a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the method for identifying the driving behavior of a vehicle driver according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: The readable storage medium stores computer program instructions, and when the computer program instructions are read and executed by a processor, the method for identifying the driving behavior of a vehicle driver according to any one of claims 1 to 7 is executed.

Images