Distraction driving behavior detection method based on neural network model and related device

Abstract

The application relates to the technical field of data processing, and discloses a distraction driving behavior detection method based on a neural network model and related devices, which comprises the following steps: obtaining a distraction driving behavior image and a driving cabin image; performing label processing on the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset; improving an original detection model to obtain an initial distraction driving behavior detection model; dividing the distraction driving behavior dataset into a training set and a verification set according to a preset proportion, training the initial distraction driving behavior detection model, and obtaining a target distraction driving behavior detection model; performing pretreatment on the driving cabin image to obtain a distraction detection image; and performing distraction driving behavior detection on the distraction detection image through the target distraction driving behavior detection model to obtain a distraction driving behavior detection result, which can effectively improve the accuracy of the detection result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing and computer vision, and particularly relates to a distraction driving behavior detection method based on a neural network model and a related device. BACKGROUND

[0002] According to the definition of the International Organization for Standardization (ISO), distraction driving refers to a phenomenon that the attention is directed to activities irrelevant to normal driving during driving, thereby causing the driving operation ability to decrease. At present, the driver distraction behavior detection method is mainly based on the physiological indicators of the driver, such as monitoring specific physiological characteristics through a skin-attached sensor, and based on the behaviors of the driver, such as the posture, gaze direction of the eyes, and hand movements of the driver, to detect distraction driving. However, only obtaining physiological indicators or obtaining behavior actions cannot comprehensively evaluate the state of the driver, thereby making the accuracy of the distraction driving behavior detection result not high, and failing to achieve the purpose of more accurately and efficiently detecting distraction driving. SUMMARY

[0003] The present application provides a distraction driving behavior detection method based on a neural network model and a related device to solve the technical problem of low accuracy of the distraction driving behavior detection result.

[0004] In a first aspect, a distraction driving behavior detection method based on a neural network model is provided, comprising:

[0005] obtaining a distraction driving behavior image and a driver cabin image;

[0006] labeling the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset;

[0007] improving an original detection model to obtain an initial distraction driving behavior detection model;

[0008] dividing the distraction driving behavior dataset into a training set and a validation set according to a preset proportion, training the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model;

[0009] preprocessing the driver cabin image to obtain a distraction detection image;

[0010] detecting the distraction driving behavior of the distraction detection image through the target distraction driving behavior detection model to obtain a distraction driving behavior detection result.

[0011] In a second aspect, a distraction driving behavior detection device based on a neural network model is provided, comprising:

[0012] an obtaining module configured to obtain a distraction driving behavior image and a driver cabin image;

[0013] The first processing module is configured to perform annotation processing on the distracted driving behaviors in the distracted driving behavior images to obtain a distracted driving behavior dataset.

[0014] The second processing module is configured to improve an original detection model to obtain an initial distracted driving behavior detection model.

[0015] The third processing module is configured to divide the distracted driving behavior dataset into a training set and a verification set according to a preset ratio, train the initial distracted driving behavior detection model, and obtain a target distracted driving behavior detection model.

[0016] The fourth processing module is configured to perform preprocessing on the cab images to obtain distracted detection images.

[0017] The detection module is configured to perform distracted driving behavior detection on the distracted detection images by using the target distracted driving behavior detection model to obtain distracted driving behavior detection results.

[0018] In a third aspect, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the distracted driving behavior detection method based on the neural network model when executing the computer program.

[0019] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the distracted driving behavior detection method based on the neural network model when executed by a processor.

[0020] In the scheme implemented by the distracted driving behavior detection method based on the neural network model and the related device, the distracted driving behavior images and the cab images are obtained, the distracted driving behavior dataset is obtained by performing annotation processing on the distracted driving behaviors in the distracted driving behavior images, the initial distracted driving behavior detection model is obtained by improving the original detection model, the training set and the verification set are obtained by dividing the distracted driving behavior dataset according to the preset ratio, the target distracted driving behavior detection model is obtained by training the initial distracted driving behavior detection model, the distracted detection images are obtained by performing preprocessing on the cab images, the distracted driving behavior detection results are obtained by performing distracted driving behavior detection on the distracted detection images by using the target distracted driving behavior detection model, and the accuracy of the distracted driving behavior detection results can be effectively improved, which is conducive to achieving the purpose of more accurate and efficient distracted driving behavior detection. BRIEF DESCRIPTION OF DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0022] Figure 1 is a flowchart of a method for detecting distracted driving behavior based on a neural network model in an embodiment of the present application;

[0023] Figure 2 is a network structure diagram of a target distracted driving behavior detection model of a method for detecting distracted driving behavior based on a neural network model in an embodiment of the present application;

[0024] Figure 3 is a structure diagram of an EMA attention mechanism module of an apparatus for detecting distracted driving behavior based on a neural network model in an embodiment of the present application;

[0025] Figure 4 is a structure diagram of an apparatus for detecting distracted driving behavior based on a neural network model in an embodiment of the present application;

[0026] Figure 5 is a structure diagram of an apparatus for detecting distracted driving behavior based on a neural network model in an embodiment of the present application;

[0027] Figure 6 is a structure diagram of a computer device in an embodiment of the present application;

[0028] Figure 7 is another structure diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0030] Please refer to Figure 1 , Figure 1 is a flowchart of a method for detecting distracted driving behavior based on a neural network model in an embodiment of the present application, comprising the following steps:

[0031] S10: Obtain distracted driving behavior images and driving room images.

[0032] The distraction driving behavior image can include one or more images corresponding to the distraction driving behavior. The distraction driving behavior image can be an image including one or more distraction driving behaviors. The distraction driving behavior can include, but is not limited to, drinking water, smoking, making a phone call, chatting, etc., and the present application does not limit this.

[0033] The cab image can include one or more images corresponding to the driving process in the cab. It can be understood that the cab image can be an image to be subjected to distraction detection, and the present application does not limit this.

[0034] S20: Labeling the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset.

[0035] After obtaining the distraction driving behavior image, the distraction driving behavior in each distraction driving behavior image, such as drinking water, smoking, making a phone call, chatting, etc., can be labeled to obtain a distraction driving behavior dataset. Optionally, the above labeling process can be performed using labeling software (such as the image labeling open source tool LabelImg), and the present application does not limit this.

[0036] For example, the distraction driving behavior is divided into drinking water, smoking, and making a phone call. The present application can use the 100-Driver public dataset, which is a large-scale, diversified, and gesture-based distraction driver dataset. The present application can select 6000 images therefrom, label the water cup, cigarette, and mobile phone as targets using the labeling tool LabelImg, and generate a corresponding.txt file for the images, and the present application does not limit this.

[0037] S30: Improving the original detection model to obtain an initial distraction driving behavior detection model.

[0038] The original detection model can be a detection model based on an improved eighth version of a single detector (You Only Look Once version 8, YOLOv8), and the present application does not limit this. It should be noted that YOLOv8 is usually composed of convolutional layers, pooling layers, fully connected layers, etc., and automatically extracts features in an image through learning of a large amount of data, and performs target detection and classification, which is a target detection algorithm based on deep learning. Taking the improved YOLOv8 as an example, the original detection model is improved to construct the distraction driving behavior detection model of the present application to obtain an initial distraction driving behavior detection model, and the specific steps can be as follows:

[0039] (1) In the feature fusion module of the original detection model, such as the Channel to Pixel (C2f) module, an Exponential Moving Average (EMA) attention mechanism is added. First, the C2f module is composed of a convolution block that receives the input feature map and generates an intermediate feature map. The generated feature map is split into two parts, one of which is directly passed to the final Concat block, and the other of which is passed to multiple Bottleneck blocks for further processing. Second, the directly passed feature map and the processed feature map are fused through the Concat block, which allows the model to utilize multi-scale and multi-level information. Finally, the final feature map is generated through a convolution block, which provides rich feature representations for subsequent detection and classification tasks.

[0040] However, due to the downsampling operation, the C2f module may not effectively extract features of small targets, resulting in poor detection of small targets. Therefore, the EMA attention mechanism is added before the Bottleneck module, which can directly process a portion of the output from the preliminary convolution layer and optimize the feature representation before the data flows into the Bottleneck layer. By applying the attention mechanism first to enhance the expression of important features, and then performing deeper feature processing and optimization, the model can have stronger feature extraction capability and improve the detection accuracy of small targets.

[0041] (2) Delete the large target detection head. The original YOLOv8 has three sizes of detection scales, 20x20, 40x40, and 80x80. For the 20x20 detection scale, it can be used to detect large targets; for the 40x40 detection scale, it can be used to detect medium-sized targets; for the 80x80 detection scale, it can be used to detect small targets. The method provided in the present application is mainly used for detecting the distraction behavior of drivers in a driving environment. The size of the target to be detected is fixed, such as detecting specific objects (e.g., mobile phones, water cups, and cigarettes) in the driver's cabin. These types of objects have extremely limited pixel range areas in the picture. Therefore, the present application proposes to remove the 20x20 detection scale to delete the detection of larger objects, and to retain the 40x40 and 80x80 detection scales to implement the detection of medium and small targets. By deleting the large target detection head, the model can better match the size of the target to be detected, aiming to reduce the computational load of the network while ensuring detection accuracy, thereby speeding up the network operation.

[0042] (3) using a focal enhanced intersection over union loss (Focal-EIoU Loss) to replace a complete intersection over union loss (CIoU Loss) in an original detection model. The method provided in the application uses the Focal-EIoU Loss to replace the original CIoU Loss as a regression loss function on the basis of the original YOLOv8 network model. Although the CIoU considers the overlapping area, the center point distance, the width-height ratio of the bounding box and the like, and adds a measurement of the width-height ratio of the predicted frame and the real frame on the basis of a distance intersection over union loss (DIoU) loss, the regression speed of the predicted frame can be accelerated to a certain extent. However, in the regression process of the predicted frame, once the width-height ratio of the predicted frame and the real frame presents a linear proportion, the width and the height of the predicted frame cannot be increased or decreased at the same time, and then the regression optimization cannot be continued.

[0043] The EIoU considers the overlapping area, the center point distance, the real difference of the width-height side length, and solves the ambiguous definition of the width-height ratio on the basis of the CIoU. Meanwhile, in order to make the EIoU pay more attention to high-quality samples, the application introduces the Focal Loss to solve the problem of sample imbalance on this basis. Therefore, using the Focal-EIoU Loss as the regression loss function can not only improve the convergence speed of the model, but also effectively handle the common sample imbalance problem in the target detection task, so as to improve the overall precision and quality of the detection model.

[0044] S40: dividing the distraction driving behavior data set according to a preset ratio into a training set and a verification set, training the initial distraction driving behavior detection model, and obtaining a target distraction driving behavior detection model.

[0045] The preset ratio can be understood as a ratio of the training set and the verification set for model training which is set in advance. Optionally, the preset ratio can be set by the user or set by default by the system, and the application does not limit this. For example, in the embodiment of the application, the distraction driving behavior data set can be divided into a training set and a verification set according to a ratio of 8:2, so as to realize the training of the initial distraction driving behavior detection model.

[0046] S50: preprocessing the cab image to obtain a distraction detection image.

[0047] A series of preprocessing operations are performed on the collected cab image, which can improve the image quality, reduce irrelevant information interference, make it more suitable for model processing and analysis, and provide more reliable data basis for subsequent model training and detection. Among them, the preprocessing can include but is not limited to removing noise, adjusting image size, normalizing pixel value, etc. The present application does not make any limitation.

[0048] The random graying method can be used to enhance the generalization ability of the model, so that the model can better adapt to different input data. Converting the image from color to grayscale image can reduce the dimension of data and reduce the computational complexity. The random graying method is adopted in the present application, which means that instead of simply converting all images to grayscale images in a fixed way, the grayscale processing is performed with a certain randomness. By using the random graying method, the model can be exposed to more different forms of input data, avoiding the model's excessive dependence on specific color information, so that the model can learn more essential image features, rather than relying only on color differences, thereby enhancing the generalization ability of the model, making it better adapt to different lighting conditions, different shooting devices and other factors that may cause color changes.

[0049] S60: performing distraction driving behavior detection on the distraction detection image to be detected by the target distraction driving behavior detection model to obtain a distraction driving behavior detection result.

[0050] After obtaining the trained target distraction driving behavior detection model, the target distraction driving behavior detection model can be used to perform distraction driving behavior detection on the preprocessed distraction detection image to be detected, so as to obtain more accurate distraction driving behavior detection results. The specific distraction driving behavior detection steps can be referred to the detailed description in the following embodiments, which will not be repeated here.

[0051] In this embodiment, by obtaining the distraction driving behavior image and the cab image, the distraction driving behavior in the distraction driving behavior image is labeled to obtain a distraction driving behavior dataset. The original detection model is improved to obtain an initial distraction driving behavior detection model. The distraction driving behavior dataset is further divided into a training set and a validation set according to a preset proportion, and the initial distraction driving behavior detection model is trained to obtain a target distraction driving behavior detection model. Thus, the cab image can be preprocessed to obtain a distraction detection image to be detected, and the target distraction driving behavior detection model can be used to perform distraction driving behavior detection on the distraction detection image to be detected to obtain a distraction driving behavior detection result, which can effectively improve the accuracy of the distraction driving behavior detection result and is conducive to achieving the purpose of more accurate and efficient distraction driving behavior detection.

[0052] In a possible implementation, the distraction driving behavior detection on the to-be-detected image is performed by using the target distraction driving behavior detection model, and a distraction driving behavior detection result is obtained, which refers to a process of how to apply the trained target distraction driving behavior detection model. As shown in Figure 2 Figure 2 A network structure diagram of a target distraction driving behavior detection model is exemplarily shown. Optionally, the distraction driving behavior detection on the to-be-detected image is performed by using the target distraction driving behavior detection model, and a distraction driving behavior detection result is obtained, including the following steps:

[0053] A1: receiving an input to-be-detected image;

[0054] A2: performing initial feature extraction on the to-be-detected image by using a convolution layer set to obtain an initial distraction feature image;

[0055] A3: performing attention feature extraction on the initial distraction feature image by using a feature fusion module to which an exponential moving average (EMA) attention mechanism is added to obtain an attention distraction feature image;

[0056] A4: performing multi-scale feature fusion on the attention distraction feature image by using a multi-scale feature fusion module to obtain a fusion distraction feature image;

[0057] A5: performing an up-sampling operation on the fusion distraction feature image to obtain an up-sampled distraction feature image;

[0058] A6: performing fusion enhancement on the up-sampled distraction feature image by using a feature fusion module to obtain an enhanced distraction feature image;

[0059] A7: determining a distraction driving behavior detection result according to the enhanced distraction feature image.

[0060] The network starts from an input layer, and receives input image data, i.e., a to-be-detected image. The input image can first pass through several convolution layers (Conv), which can be responsible for extracting low-level features to obtain an initial distraction feature image.

[0061] After the convolution layer, a C2f module to which an EMA attention mechanism is added, referred to as a C2f-EMA module, is used to perform attention feature extraction, and an attention distraction feature image can be obtained. Then, a Spatial Pyramid Pooling-Fast (SPPF) module, i.e., a module for multi-scale feature fusion, is used to help the network process features under different receptive fields, so as to obtain a fusion distraction feature image. Optionally, the SPPF module can be used for pooling operation to help the model learn more robust features, which is not limited in the present application.​

[0062] Next, an up-sampling operation (UpSample) is performed, and further feature fusion (Concat) of low-level features and high-level features is performed. This processing procedure is similar to a feature pyramid structure, which can be used to combine multi-scale features, thereby improving the detection performance for small targets. A C2f module can be used for further fusion and enhancement of the features to obtain an enhanced distraction feature image. It can be understood that the C2f module can reduce redundant calculations and improve inference efficiency by optimizing the channel dimension.

[0063] After feature fusion, the network can be divided into two branches, such as a detection head (Detect) P3 and a detection head (Detect) P4 in Figure 2 , which are used for feature images of different scales, respectively. It can be understood that the final network output of the target detection result, i.e., the distraction driving behavior detection result, can include target categories, positions, etc., which are not limited by the present application.

[0064] Referring to Figure 3 , Figure 3 an exemplary structural diagram of an EMA attention mechanism module is shown. Optionally, the attention feature extraction of the initial distraction feature image by the feature fusion module added with the exponential moving average (EMA) attention mechanism to obtain the attention distraction feature image can include the following steps:

[0065] A31: grouping the initial distraction feature image to obtain a first initial distraction feature image and a second initial distraction feature image;

[0066] A32: performing two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image;

[0067] A33: performing activation processing on the two-dimensional global pooling distraction feature image by 1x1 convolution and an activation function to obtain a first target distraction feature image;

[0068] A34: performing feature extraction on the second initial distraction feature image by convolution to obtain a convolution distraction feature image;

[0069] A35: performing activation processing on the convolution distraction feature image by two-dimensional global pooling and an activation function to obtain a second target distraction feature image;

[0070] A36: merging the first target distraction feature image and the second target distraction feature image by a cross-dimension interaction module to obtain a cross-dimension distraction feature image;

[0071] A37: performing channel weighting on the cross-dimension distraction feature image by a weight generated by an activation function to obtain an attention distraction feature image.

[0072] In the EMA module, the input (i.e., the initial distraction feature image) can be first grouped, as shown in Figure 3 The input CxHxW can be divided into G groups C / GxHxW. Then, the processing is performed through different branches, as shown in Figure 3 The upper branch can perform two-dimensional global pooling, 1x1 convolution, and activation processing (Sigmoid activation), and the lower branch can perform 3x3 convolution, two-dimensional global pooling, and activation processing (Softmax activation). The output features of the two branches can be then modulated by a Sigmoid function and a normalization operation, and finally merged by a cross-dimension interaction module and further processed by a 1x1 convolution to capture the pixel-level pair-wise relationship. After the final Sigmoid adjustment, such as channel weighting using the weight generated by the Sigmoid, the output feature map with the same dimension as the input is output. Among them, the output feature map enhances or weakens the original input feature, thereby obtaining an attention distraction feature image.

[0073] EMA is a high-efficiency multi-scale attention mechanism, which aims to reduce the computational overhead while preserving the key information of each channel, grouping the channel dimension into multiple sub-features, and making the spatial semantic features uniformly distributed within each feature group. Specifically, in addition to encoding global information to recalibrate the channel weights in each parallel branch, the output features of the two parallel branches are further aggregated by a cross-dimension interaction method.

[0074] Optionally, the step of performing two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image can include: performing two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image by the following formula:

[0075]

[0076] wherein Z C represents the two-dimensional global pooling distraction feature image corresponding to each channel; C represents the number of input channels; H represents the height of the input first initial distraction feature image in the spatial dimension; W represents the width of the input first initial distraction feature image in the spatial dimension; j represents the height index; i represents the width index; x c (i,j) represents the input feature value at the spatial dimension (i,j) position on the channel C.

[0077] Optionally, the improvement of the original detection model to obtain the initial distracted driving behavior detection model can include: adopting a focal Euclidean intersection over union loss function Focal-EIoU Loss as a regression loss function of the initial distracted driving behavior detection model; a formula of the focal Euclidean intersection over union loss function Focal-EIoU Loss is as follows:

[0078] L Focal-EIoU = IoU β L EIoU

[0079]

[0080] wherein, L Focal-EIoU represents the focal Euclidean intersection over union loss function; IoU represents an intersection over union of a predicted box and a real box; γ is a parameter for controlling an abnormal value suppression degree; IoU γ represents an IoU-based weighted item for adaptively adjusting the loss of different IoU values; L EIoU represents an enhanced intersection over union loss function; ρ 2 represents a Euclidean distance of a calculation element; b represents a center point of the predicted box; b gt represents a center point of the real box; c represents a length of a diagonal line of a minimum circumscribed rectangle of the predicted box and the real box; w represents a width of the predicted box; w gt represents a width of the real box; h represents a height of the predicted box; h gt represents a height of the real box; C w represents a width of the minimum circumscribed rectangle; C h represents a height of the minimum circumscribed rectangle.

[0081] The application further provides a target distracted driving behavior detection device for realizing the driver distracted driving detection method based on the improved YOLOv8. Optionally, the distracted driving detection algorithm (i.e., the target distracted driving behavior detection model) can be transplanted into the target distracted driving behavior detection device, and a deep learning acceleration module is used to accelerate the neural network processing.

[0082] The target distraction driving behavior detection device can mainly include image information acquisition, image processor, data storage, display and voice prompt modules. The main function of the image acquisition information module can be to capture and collect the images and video data of the driver in real time, collect the distraction driving behaviors of the driver, such as drinking water, making a phone call, smoking, etc.; the image processor module can be responsible for analyzing and processing the image and video data transmitted by the image acquisition information module, and the core task is to extract useful information from the original data and classify and identify; the data storage module can be used to save the collected image data, analysis results and historical records; the display module can be responsible for displaying the detection results and related information of the system to the driver or other relevant personnel, and the voice prompt module can be used to remind the driver to pay attention to safe driving through voice output.

[0083] Further, the image acquisition information module is usually composed of a camera and a sensor, and the camera is generally installed inside the vehicle to capture the key features of the driver's facial expression, eye movement, head position and hand movement. Since the driving environment is divided into day and night, in order to better detect the distraction driving behavior of the driver, the application needs to use a high-definition infrared camera for image acquisition, which can ensure the image quality in most strong light, weak light and night environment, thereby reducing the influence on the distraction driving detection performance.

[0084] Further, the image processor module usually adopts high-performance processors and advanced image processing algorithms, which plays a crucial role in the distraction driving detection device. It is not only the core of data processing, but also needs to have multiple functions to ensure the reliability and real-time performance of the device. Mainly including image preprocessing, feature extraction, behavior analysis, system self-learning and optimization functions. Image preprocessing is the first step of the image processor module, which aims to optimize the original image to improve the accuracy of subsequent analysis; feature extraction is to identify and extract key feature points and regions from the preprocessed image, such as driver's hand, face and other regions, to provide basic data for subsequent behavior analysis; behavior analysis is based on the extracted feature data, using machine learning and deep learning algorithms to analyze the driver's behavior pattern and judge whether there is distraction behavior; system self-learning means that the image processor module improves its analysis ability and accuracy through continuous learning and optimization.

[0085] Further, the data storage module plays a key role in the distraction driving detection device, responsible for saving and managing various types of data generated by the image processor module. It usually needs to have high capacity and high speed characteristics to meet the needs of real-time data processing. This module mainly involves short-term storage: saving recent images and detection results for backtracking analysis in case of accidents or abnormal detection; long-term storage: recording driver behavior data and system detection results for future audit, research and algorithm optimization; data backup and synchronization: ensuring data security, preventing loss, and supporting data synchronization with external systems (such as vehicle computers, cloud platforms) and other functions.

[0086] Further, the display module is responsible for presenting the analysis results to the driver or other users in a visual manner. It usually includes in-vehicle display screens and mobile device display interfaces. The voice prompt module communicates detection results and warning information to the driver through voice, reminding the driver to pay attention to safe driving. When distraction behavior or dangerous situations are detected, the system will issue voice warnings to the driver, such as: when the driver is drinking water, the system will broadcast "Please do not drink water"; when the driver is making a phone call, the system will broadcast "Please do not make a phone call"; when the driver is smoking, the system will broadcast "Please do not smoke".

[0087] For example, as shown in Figure 4 , a block diagram of a distraction driving behavior detection device is shown. Figure 4 For example, as shown in Figure 4 , a block diagram of a distraction driving behavior detection device is shown.

[0088] 1. RTC (Real-Time Clock) & Power Control: Real-time clock and power management module, providing time tracking and power control functions.

[0089] 2. GPIO (General-Purpose Input / Output): General-purpose input / output port, used for simple digital signal interaction with other devices.

[0090] 3. ADC (Analog-to-Digital Converter): Analog-to-digital converter, used to convert analog signals to digital signals.

[0091] 4、SDIO (Secure Digital Input / Output) Controller: SDIO controller for connecting storage devices like SD cards.

[0092] 5、USB 2.0 Host / Device: USB 2.0 host / device interface for connecting peripherals like keyboards, storage devices, etc.

[0093] 6、UART (Universal Asynchronous Receiver / Transmitter): UART for serial communication.

[0094] 7、LCD / MIPI Controller: LCD or MIPI interface controller for driving display devices.

[0095] 8、Ethernet PHY: Ethernet physical layer interface for network connectivity.

[0096] 9、Sensor IF (CSI / SLVDS): Sensor interface for connecting image sensors, CSI (Camera Serial Interface) and SLVDS (Serial Low-voltage Differential Signaling) are transmission protocols.

[0097] 10、I2C (Inter-Integrated Circuit): Communication bus for inter-chip communication, commonly used in sensors, clock chips, etc.

[0098] 11、SIF (Serial Interface): Serial interface, usually used for simple device communication.

[0099] 12、AMP (Amplifier): Amplifier for audio signal amplification.

[0100] 13、MD Engine (Motion Detection Engine): Motion detection engine for detecting moving objects in video.

[0101] 14、3DNR (3D Noise Reduction): 3D noise reduction technology for video noise reduction.

[0102] 15、Anti-fog: Anti-fog treatment for image clarity optimization.

[0103] 16、OSD (On-Screen Display) / Graphics: On-Screen Display module, responsible for superimposing graphics onto video streams (e.g., subtitles).

[0104] 17、ARM Cortex A9: Processor core, responsible for processing system logic and application tasks.

[0105] 18、H.264 / H.265 Video Codec: Video encoder, used for compressing and decoding video, H.264 and H.265 are common high-efficiency video compression standards.

[0106] 19、DLA (Deep Learning Accelerator): Deep learning accelerator, used to accelerate neural network calculations.

[0107] 20、PEG Engine: JPEG image encoding engine, used for compressing images.

[0108] 21、Flash Controller: Flash memory controller, used to manage and operate flash memory storage devices.

[0109] 22、SDRAM Controller: Dynamic Random Access Memory controller, used to manage external memory.

[0110] 23、DDR III DRAM: Double Data Rate Third Generation Dynamic Random Access Memory, used to provide high-speed storage.

[0111] 24、SPI (Serial Peripheral Interface): Serial Peripheral Interface, used for external device communication.

[0112] 25、NOR / NAND: NOR / NAND flash memory storage, commonly used for program code and data storage.

[0113] 26、Key / Battery Detect: Keyboard and battery detection module, used to detect the connection of external devices and battery status.

[0114] Compared with the prior art, the application has the following beneficial effects: 1. Adding an EMA attention mechanism in the C2f module helps to enhance the feature expression capability, so that the model can better focus on important areas in the image; 2. Removing the head (Head) in YOLOv8 P5 detection head reduces the computational amount and parameter amount of the model, thereby reducing the complexity and computational cost of the model; 3. Using Focal-EIoU Loss instead of CIoU Loss as the regression loss function improves the accuracy and convergence speed of the model. 4. Finally, the distraction driving detection algorithm is transplanted into the device, and the deep learning acceleration module is used to accelerate the neural network processing, and the software and hardware system is designed and implemented, and the performance of the distraction driving detection system is comprehensively tested and analyzed.

[0115] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0116] In an embodiment, a distraction driving behavior detection device based on a neural network model is provided, which corresponds to the distraction driving behavior detection method based on a neural network model in the above embodiment. As shown in the figure, the distraction driving behavior detection device based on a neural network model includes an acquisition module 101, a first processing module 102, a second processing module 103, a third processing module 104, a fourth processing module 105 and a detection module 106. The functions of each functional module are described in detail as follows: Figure 5

[0117] The acquisition module 101 is configured to acquire distraction driving behavior images and cab images;

[0118] The first processing module 102 is configured to perform annotation processing on the distraction driving behavior in the distraction driving behavior images to obtain a distraction driving behavior dataset;

[0119] The second processing module 103 is configured to improve an original detection model to obtain an initial distraction driving behavior detection model;

[0120] The third processing module 104 is configured to divide the distraction driving behavior dataset into a training set and a validation set according to a preset ratio, train the initial distraction driving behavior detection model, and obtain a target distraction driving behavior detection model;

[0121] The fourth processing module 105 is configured to pre-process the cab images to obtain distraction detection images;

[0122] ​The detection module 106 is configured to perform distraction driving behavior detection on the to-be-detected image by using the target distraction driving behavior detection model, and obtain a distraction driving behavior detection result.

[0123] In an embodiment, the detection module 106 is configured to perform distraction driving behavior detection on the to-be-detected image by using the target distraction driving behavior detection model, and obtain a distraction driving behavior detection result, and specifically configured to:

[0124] receive an input to-be-detected image;

[0125] perform initial feature extraction on the to-be-detected image by using a convolutional layer set, and obtain an initial distraction feature image;

[0126] perform attention feature extraction on the initial distraction feature image by using a feature fusion module to which an exponential moving average (EMA) attention mechanism is added, and obtain an attention distraction feature image;

[0127] perform multi-scale feature fusion on the attention distraction feature image by using a multi-scale feature fusion module, and obtain a fused distraction feature image;

[0128] perform an upsampling operation on the fused distraction feature image, and obtain an upsampled distraction feature image;

[0129] perform fusion enhancement on the upsampled distraction feature image by using a feature fusion module, and obtain an enhanced distraction feature image;

[0130] determine a distraction driving behavior detection result according to the enhanced distraction feature image.

[0131] In an embodiment, the detection module 106 is configured to perform attention feature extraction on the initial distraction feature image by using a feature fusion module to which an exponential moving average (EMA) attention mechanism is added, and obtain an attention distraction feature image, and specifically configured to:

[0132] group the initial distraction feature image to obtain a first initial distraction feature image and a second initial distraction feature image;

[0133] perform one-dimensional global pooling on the first initial distraction feature image, and obtain a global-pooled distraction feature image;

[0134] perform feature extraction on the second initial distraction feature image by using convolution, and obtain a convolutional distraction feature image;

[0135] modulate the global-pooled distraction feature image and the convolutional distraction feature image by using an activation function and a normalization operation, and merge the global-pooled distraction feature image and the convolutional distraction feature image by using a cross-dimension interaction module, and obtain a cross-dimension distraction feature image;

[0136] The cross-dimension distraction feature image is adjusted through an activation function to obtain an attention distraction feature image.

[0137] In an embodiment, the detection module 106 is configured to perform two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image, and specifically configured to:

[0138] The two-dimensional global pooling on the first initial distraction feature image is performed to obtain a two-dimensional global pooling distraction feature image through the following formula:

[0139]

[0140] wherein, Z C represents the two-dimensional global pooling distraction feature image corresponding to each channel; C represents the input channel number; H represents the height of the input first initial distraction feature image in the spatial dimension; W represents the width of the input first initial distraction feature image in the spatial dimension; j represents the height index; i represents the width index; x c (i,j) represents the input feature value at the spatial dimension (i,j) position on the channel C.

[0141] In an embodiment, the second processing module is configured to improve the original detection model to obtain an initial distraction driving behavior detection model, and specifically configured to:

[0142] The focal Euclidean intersection over union loss function Focal-EIoU Loss is used as the regression loss function of the initial distraction driving behavior detection model;

[0143] The formula of the focal Euclidean intersection over union loss function Focal-EIoU Loss is as follows:

[0144] L Focal-EIoU = IoU γ L EIoU

[0145]

[0146] wherein, L Focal-EIoU represents the focal Euclidean intersection over union loss function; IoU represents the intersection over union of the predicted box and the real box; γ is a parameter for controlling the degree of abnormal value suppression; IoU γ represents the weighted item based on IoU, which is used for adaptive adjustment of the loss of different IoU values; L EIoU represents the enhanced intersection over union loss function; ρ 2 represents the Euclidean distance of the calculation element; b represents the center point of the predicted box; b gtrepresents the center point of the real box; c represents the length of the diagonal of the minimum bounding rectangle of the prediction box and the real box; w represents the width of the prediction box; w gt represents the width of the real box; h represents the height of the prediction box; h gt represents the height of the real box; c w represents the width of the minimum bounding rectangle; c h represents the height of the minimum bounding rectangle.

[0147] The application provides a distraction driving behavior detection device based on a neural network model. The distraction driving behavior detection device obtains a distraction driving behavior image and a driving cabin image, labels a distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset, improves an original detection model to obtain an initial distraction driving behavior detection model, further divides the distraction driving behavior dataset into a training set and a verification set according to a preset proportion, trains the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model, pre-processes the driving cabin image to obtain a to-be-detected image, detects a distraction driving behavior of the to-be-detected image by using the target distraction driving behavior detection model to obtain a distraction driving behavior detection result, and effectively improves the accuracy of the distraction driving behavior detection result, which is conducive to achieving the purpose of more accurate and efficient distraction driving behavior detection.

[0148] The specific limitations of the distraction driving behavior detection device based on the neural network model can be seen in the limitations of the distraction driving behavior detection method based on the neural network model in the foregoing, which will not be described here. Each module in the distraction driving behavior detection device based on the neural network model can be realized by software, hardware and a combination thereof in whole or in part. Each module can be embedded in or independent of a processor in a computer device in a hardware form, or can be stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform the operations corresponding to each module.

[0149] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in Figure 6As shown in the figure. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile and / or volatile storage media, internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external client through the network connection. The computer program is executed by the processor to realize the functions or steps of the server side of the distraction driving behavior detection method based on the neural network model.

[0150] In one embodiment, a computer device is provided, which can be a client, and its internal structure diagram can be as shown in the figure. Figure 7 As shown in the figure. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes non-volatile storage media, internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the client side of the distraction driving behavior detection method based on the neural network model.

[0151] In one embodiment, a computer device is provided, including a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor executes the computer program to realize the following steps:

[0152] Obtain distraction driving behavior images and driving room images;

[0153] Label the distraction driving behavior in the distraction driving behavior images to obtain a distraction driving behavior dataset;

[0154] Improve the original detection model to obtain an initial distraction driving behavior detection model;

[0155] Divide the distraction driving behavior dataset into a training set and a validation set according to a preset ratio, train the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model;

[0156] Preprocess the driving room images to obtain distraction detection images;

[0157] The target distraction driving behavior detection model is used for distraction driving behavior detection on the to-be-detected distraction image, and a distraction driving behavior detection result is obtained.

[0158] The present application provides a computer device, by obtaining distraction driving behavior image and driving room image, to label the distraction driving behavior in the distraction driving behavior image, and get the distraction driving behavior dataset; and improve the original detection model, get the initial distraction driving behavior detection model, to further divide the distraction driving behavior dataset into training set and validation set according to the preset proportion, train the initial distraction driving behavior detection model, get the target distraction driving behavior detection model, so as to preprocess the driving room image, get the to-be-detected distraction image, and then the target distraction driving behavior detection model can be used for distraction driving behavior detection on the to-be-detected distraction image, and a distraction driving behavior detection result is obtained, which can effectively improve the accuracy of the distraction driving behavior detection result, and is conducive to realizing the purpose of more accurate and efficient distraction driving behavior detection.

[0159] In one embodiment, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the following steps:

[0160] Obtain distraction driving behavior image and driving room image;

[0161] Label the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset;

[0162] Improve the original detection model to obtain an initial distraction driving behavior detection model;

[0163] Divide the distraction driving behavior dataset into a training set and a validation set according to a preset proportion, train the initial distraction driving behavior detection model, and obtain a target distraction driving behavior detection model;

[0164] Preprocess the driving room image to obtain a to-be-detected distraction image;

[0165] The target distraction driving behavior detection model is used for distraction driving behavior detection on the to-be-detected distraction image, and a distraction driving behavior detection result is obtained.

[0166] The application provides a computer readable storage medium, which obtains a distraction driving behavior image and a driver's room image, performs annotation processing on the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior data set, improves an original detection model to obtain an initial distraction driving behavior detection model, further divides the distraction driving behavior data set into a training set and a verification set according to a preset proportion, trains the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model, so that the driver's room image can be preprocessed to obtain a distraction detection image, and then the target distraction driving behavior detection model can be used for distraction driving behavior detection on the distraction detection image to obtain a distraction driving behavior detection result, the accuracy of the distraction driving behavior detection result can be effectively improved, and the purpose of more accurate and efficient distraction driving behavior detection can be achieved.

[0167] It should be noted that the functions or steps that can be achieved by the computer readable storage medium or the computer device described above can be referred to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0168] Those skilled in the art can understand that all or part of the processes in the foregoing embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the foregoing embodiments of the method can be included. Any reference to a memory, storage, database or other medium used in the embodiments provided in the present application can include a non-volatile and / or volatile memory. The non-volatile memory can include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM) or a flash memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM) and memory bus dynamic RAM (RDRAM).

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above.

[0170] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for detecting a distracted driving behavior based on a neural network model, the method comprising: The method comprises: acquiring a distraction driving behavior image and a driver's cabin image; labeling the distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior dataset; improving an original detection model to obtain an initial distraction driving behavior detection model; dividing the distraction driving behavior dataset into a training set and a verification set according to a preset proportion, training the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model; preprocessing the driver's cabin image to obtain a distraction detection image; detecting the distraction driving behavior of the distraction detection image by the target distraction driving behavior detection model to obtain a distraction driving behavior detection result; the improvement of the original detection model to obtain the initial distraction driving behavior detection model comprises: adding an exponential moving average EMA attention mechanism before a bottleneck module in a feature fusion module of the original detection model; deleting a large target detection head in the original detection model; replacing a loss function in the original detection model with a focal-EIoU loss function to obtain the initial distraction driving behavior detection model. 2.The method of claim 1, wherein, the distraction driving behavior detection of the distraction detection image by the target distraction driving behavior detection model to obtain the distraction driving behavior detection result comprises: receiving an input distraction detection image; performing initial feature extraction on the distraction detection image through a convolution layer set to obtain an initial distraction feature image; performing attention feature extraction on the initial distraction feature image through a feature fusion module with an added exponential moving average EMA attention mechanism to obtain an attention distraction feature image; performing multi-scale feature fusion on the attention distraction feature image through a multi-scale feature fusion module to obtain a fusion distraction feature image; performing an upsampling operation on the fusion distraction feature image to obtain an upsampled distraction feature image; performing fusion enhancement on the upsampled distraction feature image through a feature fusion module to obtain an enhanced distraction feature image; determining a distraction driving behavior detection result according to the enhanced distraction feature image. 3.The method of claim 2, wherein, the attention feature extraction on the initial distraction feature image through the feature fusion module with the added exponential moving average EMA attention mechanism to obtain the attention distraction feature image comprises: grouping the initial distraction feature image to obtain a first initial distraction feature image and a second initial distraction feature image; performing two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image; activating the two-dimensional global pooling distraction feature image through a 1x1 convolution and an activation function to obtain a first target distraction feature image; performing feature extraction on the second initial distraction feature image through a convolution to obtain a convolution distraction feature image; activating the convolution distraction feature image through two-dimensional global pooling and an activation function to obtain a second target distraction feature image; merging the first target distraction feature image and the second target distraction feature image through a cross-dimension interaction module to obtain a cross-dimension distraction feature image; The cross-dimension distraction feature image is channel-weighted by a weight generated by an activation function to obtain an attention distraction feature image. 4.The method of claim 3, wherein, The two-dimensional global pooling of the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image comprises: The two-dimensional global pooling of the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image is achieved by the following formula: wherein, represents the two-dimensional global pooling distraction feature image corresponding to each channel; C represents the input channel number; H represents the height of the input first initial distraction feature image in the spatial dimension; W represents the width of the input first initial distraction feature image in the spatial dimension; represents the height index; represents the width index; represents the input feature value at the spatial dimension position on the channel C. 5.The method of claim 3, wherein, The loss function in the original detection model is replaced by the focal Euclidean intersection over union loss function Focal-EIoU Loss to obtain an initial distraction driving behavior detection model. The focal Euclidean intersection over union loss function Focal-EIoU Loss is used as a regression loss function of the initial distraction driving behavior detection model. The formula of the focal Euclidean intersection over union loss function Focal-EIoU Loss is as follows: wherein, represents a focal Euclidean intersection over union loss function; represents an intersection over union between a predicted box and a ground truth box; a parameter for controlling the degree of outlier suppression; represents an IoU-based weighting term for adaptively adjusting the loss of different IoU values; represents an enhanced intersection over union loss function; represents the Euclidean distance of a calculation element; represents the center point of a predicted box; represents the center point of a ground truth box; represents the length of the diagonal of the minimum bounding rectangle of a predicted box and a ground truth box; w represents the width of a predicted box; represents the width of a ground truth box; h represents the height of a predicted box; represents the height of a ground truth box; represents the width of a minimum bounding rectangle; represents the height of a minimum bounding rectangle.

6. A device for detecting a distracted driving behavior based on a neural network model, characterized by, The distraction driving behavior detection device based on the neural network model comprises: An acquisition module is configured to acquire a distraction driving behavior image and a driver's cabin image; A first processing module is configured to perform label processing on a distraction driving behavior in the distraction driving behavior image to obtain a distraction driving behavior data set; A second processing module is configured to improve an original detection model to obtain an initial distraction driving behavior detection model; A third processing module is configured to divide the distraction driving behavior data set into a training set and a validation set according to a preset proportion, train the initial distraction driving behavior detection model, and obtain a target distraction driving behavior detection model; A fourth processing module is configured to pre-process the driver's cabin image to obtain a distraction detection image; A detection module is configured to perform distraction driving behavior detection on the distraction detection image by using the target distraction driving behavior detection model to obtain a distraction driving behavior detection result. The second processing module is configured to improve an original detection model to obtain an initial distraction driving behavior detection model, and specifically configured to: add an exponential moving average EMA attention mechanism before a bottleneck module in a feature fusion module of the original detection model; delete a large target detection head in the original detection model; and replace a loss function in the original detection model by the focal Euclidean intersection over union loss function Focal-EIoU Loss to obtain the initial distraction driving behavior detection model.

7. The neural network model based inattentive driving behavior detection apparatus according to claim 6, characterized by, The detection module is configured to perform distraction driving behavior detection on the distraction detection image by using the target distraction driving behavior detection model to obtain a distraction driving behavior detection result, and specifically configured to: receive an input distraction detection image; perform initial feature extraction on the distraction detection image by using a convolutional layer set to obtain an initial distraction feature image; perform attention feature extraction on the initial distraction feature image by using a feature fusion module to which an exponential moving average EMA attention mechanism is added to obtain an attention distraction feature image; perform multi-scale feature fusion on the attention distraction feature image by using a multi-scale feature fusion module to obtain a fusion distraction feature image; perform an upsampling operation on the fusion distraction feature image to obtain an upsampled distraction feature image; and perform distraction driving behavior detection on the upsampled distraction feature image by using the target distraction driving behavior detection model to obtain a distraction driving behavior detection result. The up-sampling distraction feature image is fused and enhanced by a feature fusion module to obtain an enhanced distraction feature image; A distraction driving behavior detection result is determined according to the enhanced distraction feature image.

8. The neural network model based inattentive driving behavior detection apparatus according to claim 7, characterized by, The detection module is configured to extract attention features of the initial distraction feature image by the feature fusion module added with an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image, and specifically configured to: group the initial distraction feature image to obtain a first initial distraction feature image and a second initial distraction feature image; perform one-dimensional global pooling on the first initial distraction feature image to obtain a global-pooled distraction feature image; extract features of the second initial distraction feature image by convolution to obtain a convolutional distraction feature image; modulate the global-pooled distraction feature image and the convolutional distraction feature image by an activation function and a normalization operation, and merge them by a cross-dimension interaction module to obtain a cross-dimension distraction feature image; adjust the cross-dimension distraction feature image by an activation function to obtain the attention distraction feature image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the neural network model-based distraction driving behavior detection method of any one of claims 1 to 5.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the neural network model-based distraction driving behavior detection method of any one of claims 1 to 5.

Images