Distraction driving behavior detection method based on neural network model and related device

Abstract

The invention relates to the technical field of data processing, and discloses a distraction driving behavior detection method based on a neural network model and a related device, and the method comprises the steps: obtaining a distraction driving behavior image and a cab image; marking the distraction driving behaviors in the distraction driving behavior image to obtain a distraction driving behavior data set; improving the original detection model to obtain an initial distracted driving behavior detection model; dividing the distraction driving behavior data set into a training set and a verification set according to a preset proportion, and training the initial distraction driving behavior detection model to obtain a target distraction driving behavior detection model; preprocessing the cab image to obtain a to-be-distracted detection image; distraction driving behavior detection is carried out on the to-be-distraction detection image through the target distraction driving behavior detection model to obtain a distraction driving behavior detection result, and the accuracy of the detection result can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing and computer vision technology, and in particular to a distracted driving behavior detection method based on a neural network model and a related device. Background Art

[0002] According to the definition of the International Organization for Standardization (ISO), distracted driving refers to a phenomenon in which the driver's attention is directed to activities unrelated to normal driving, which leads to a decrease in driving operation ability. At present, the driver's distracted behavior detection method is mainly based on the driver's physiological indicators, such as monitoring specific physiological characteristics through sensors close to the skin, and based on the driver's behavior, such as the driver's posture, eye gaze direction, hand movements, etc., to perform distracted driving detection. However, simply obtaining physiological indicators or behavioral actions cannot fully evaluate the driver's state, resulting in low accuracy of distracted driving behavior detection results, and cannot achieve the purpose of more accurate and efficient distracted driving detection. Summary of the invention

[0003] The present invention provides a distracted driving behavior detection method and related devices based on a neural network model to solve the technical problem that the distracted driving behavior detection result is not accurate enough.

[0004] In a first aspect, a distracted driving behavior detection method based on a neural network model is provided, comprising:

[0005] Acquire distracted driving behavior images and cab images;

[0006] Annotating the distracted driving behaviors in the distracted driving behavior images to obtain a distracted driving behavior dataset;

[0007] The original detection model is improved to obtain the initial distracted driving behavior detection model;

[0008] Dividing the distracted driving behavior data set into a training set and a validation set according to a preset ratio, training the initial distracted driving behavior detection model, and obtaining a target distracted driving behavior detection model;

[0009] Preprocessing the cab image to obtain an image to be detected as distraction;

[0010] The distracted driving behavior detection is performed on the distracted driving behavior detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0011] In a second aspect, a distracted driving behavior detection device based on a neural network model is provided, comprising:

[0012] An acquisition module, used for acquiring distracted driving behavior images and cab images;

[0013] A first processing module is used to label the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set;

[0014] The second processing module is used to improve the original detection model to obtain an initial distracted driving behavior detection model;

[0015] A third processing module is used to divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, train the initial distracted driving behavior detection model, and obtain a target distracted driving behavior detection model;

[0016] A fourth processing module, used for preprocessing the cab image to obtain an image to be detected for distraction;

[0017] The detection module is used to perform distracted driving behavior detection on the distracted driving detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0018] In a third aspect, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the distracted driving behavior detection method based on the neural network model are implemented.

[0019] In a fourth aspect, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the steps of the distracted driving behavior detection method based on the neural network model are implemented.

[0020] In the scheme implemented by the distracted driving behavior detection method and related device based on the neural network model, a distracted driving behavior image and a cab image are obtained to annotate the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set; and the original detection model is improved to obtain an initial distracted driving behavior detection model, so as to further divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, and the initial distracted driving behavior detection model is trained to obtain a target distracted driving behavior detection model, so that the cab image can be preprocessed to obtain a to-be-distracted detection image, and then the to-be-distracted detection image can be subjected to distracted driving behavior detection by the target distracted driving behavior detection model to obtain a distracted driving behavior detection result, which can effectively improve the accuracy of the distracted driving behavior detection result and is conducive to achieving the purpose of more accurate and efficient distracted driving behavior detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 is a flow chart of a distracted driving behavior detection method based on a neural network model in one embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of the network structure of a target distracted driving behavior detection model of a distracted driving behavior detection method based on a neural network model in one embodiment of the present invention;

[0024] Figure 3 It is a structural schematic diagram of an EMA attention mechanism module of a distracted driving behavior detection device based on a neural network model in one embodiment of the present invention;

[0025] Figure 4 is a schematic diagram of the structure of a distracted driving behavior detection device based on a neural network model in one embodiment of the present invention;

[0026] Figure 5 is a structural schematic diagram of a distracted driving behavior detection device based on a neural network model in one embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of a structure of a computer device in one embodiment of the present invention;

[0028] Figure 7 It is another structural schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0030] See also Figure 1 As shown, Figure 1 A flowchart of a distracted driving behavior detection method based on a neural network model provided in an embodiment of the present invention includes the following steps:

[0031] S10: Obtain distracted driving behavior images and cab images.

[0032] The distracted driving behavior image may include one or more images corresponding to the distracted driving behavior, and the distracted driving behavior image may be an image including one or more distracted driving behaviors. Distracted driving behaviors may include but are not limited to drinking, smoking, making phone calls, chatting, etc., and this application does not limit this.

[0033] The cab image may include one or more images corresponding to the driving process in the cab. It is understandable that the cab image may be an image to be detected for distraction, and the present application does not limit this.

[0034] S20: labeling the distracted driving behaviors in the distracted driving behavior image to obtain a distracted driving behavior dataset.

[0035] After obtaining distracted driving behavior images, the distracted driving behaviors in each distracted driving behavior image, such as drinking, smoking, making phone calls, chatting, etc., can be labeled to obtain a distracted driving behavior dataset. Optionally, labeling software (such as the image labeling open source tool LabelImg) can be used to perform the above labeling processing, and this application does not limit this.

[0036] For example, taking the distracted driving behaviors of drinking, smoking, and making phone calls as an example, this application can use the 100-Driver public dataset, which can be a large-scale, diverse, posture-based distracted driver dataset. This application can select 6,000 images from it, use the labeling tool LabelImg to label water cups, cigarettes, and mobile phones as targets, and generate a .txt file corresponding to the image, and this application does not limit this.

[0037] S30: Improving the original detection model to obtain an initial distracted driving behavior detection model.

[0038] Among them, the original detection model can be a detection model based on the improved eighth version of the single detector (You Only Look Once version 8, YOLOv8), and this application does not limit this. It should be noted that YOLOv8 is usually composed of convolutional layers, pooling layers, fully connected layers, etc. It automatically extracts features in images by learning a large amount of data, and detects and classifies targets. It is a target detection algorithm based on deep learning. Taking the original detection model as the improved YOLOv8 as an example, this application improves the original detection model to construct the distracted driving behavior detection model of this application and obtain the initial distracted driving behavior detection model. The specific steps can be as follows:

[0039] (1) Add the Exponential Moving Average (EMA) attention mechanism to the feature fusion module of the original detection model, such as the Channel to Pixel (C2f) module. First, the C2f module consists of a convolution block that receives the input feature map and generates an intermediate feature map. The generated feature map is split into two parts, one part is directly passed to the final concatenation (Concat) block, and the other part is passed to multiple bottleneck (Botleneck) blocks for further processing. Secondly, the directly passed feature map and the processed feature map are fused through the Concat block, which allows the model to comprehensively utilize multi-scale and multi-level information. Finally, the final feature map is generated through the convolution block, which can provide rich feature representation for subsequent detection and classification tasks.

[0040] However, due to the downsampling operation, the C2f module will not be able to extract features of small objects well, which will lead to poor detection of small objects. Therefore, this method adds the EMA attention mechanism before the Bottleneck module, which can directly process part of the output from the preliminary convolution layer and optimize the feature representation before the data flows into the Bottleneck layer. By first applying the attention mechanism to enhance the expression of important features, and then performing deeper feature processing and optimization, it can have a stronger feature extraction ability and improve the detection accuracy of small objects.

[0041] (2) Delete the large target detection head. The original YOLOv8 has three detection scales, namely 20×20, 40×40, and 80×80. The 20×20 detection scale can be used to detect large targets; the 40×40 detection scale can be used to detect medium-sized targets; and the 80×80 detection scale can be used to detect small targets. The method provided in this application is mainly used to detect the driver's distracted behavior in the in-vehicle driving environment. The size of the target to be detected is fixed. For example, specific objects in the cab (such as mobile phones, water cups, and cigarettes) can be detected. The pixel range area occupied by these types of objects in the picture is extremely limited. Therefore, this application proposes to remove the 20×20 detection scale to delete the detection of larger objects, and retain the two detection scales of 40×40 and 80×80 to realize the detection of small and medium-sized targets. By deleting the large target detection head, the model can better match the size of the target to be detected, aiming to reduce the amount of calculation required by the network while ensuring the detection accuracy, thereby speeding up the operation of the network.

[0042] (3) The focal Euclidean intersection over Union loss (Focal Enhanced Intersection over Union Loss, Focal-EIoU Loss) is used to replace the complete intersection over Union loss (Complete Intersection over Union Loss, CIoU Loss) in the original detection model. The method provided in this application uses Focal-EIoU Loss to replace the original CIoU Loss as the regression loss function based on the original YOLOv8 network model. Although CIoU takes into account factors such as overlapping area, center point distance, and aspect ratio of the bounding box, it adds a measure of the aspect ratio of the predicted box and the true box on the basis of the distance intersection over Union loss (Distance Intersection over Union Loss, DIoU) loss, which can speed up the regression speed of the predicted box to a certain extent. However, in the prediction box regression process, once the aspect ratio of the predicted box and the true box is in linear proportion, the width and height of the predicted box cannot be increased or decreased at the same time, and the regression optimization cannot continue.

[0043] EIoU takes into account the overlapping area, center point distance, width, height and side length, and solves the fuzzy definition of aspect ratio on the basis of CIoU. At the same time, in order to make EIoU pay more attention to high-quality samples, this application introduces Focal Loss on this basis to solve the problem of sample imbalance. Therefore, using Focal-EIoU Loss as the regression loss function can not only improve the convergence speed of the model, but also effectively deal with the common sample imbalance problem in target detection tasks, so as to improve the overall accuracy and quality of the detection model.

[0044] S40: dividing the distracted driving behavior dataset into a training set and a validation set according to a preset ratio, training the initial distracted driving behavior detection model, and obtaining a target distracted driving behavior detection model.

[0045] Among them, the preset ratio can be understood as the ratio of the training set and the validation set for model training. Optionally, the preset ratio can be set by the user or by the system default setting, and this application does not limit this. Exemplarily, in an embodiment of the present application, the distracted driving behavior data set can be divided into a training set and a validation set in a ratio of 8:2 to achieve the training of the initial distracted driving behavior detection model.

[0046] S50: Preprocess the cab image to obtain an image to be detected for distraction.

[0047] Performing a series of preprocessing operations on the collected cab images can improve the image quality, reduce the interference of irrelevant information, make it more suitable for model processing and analysis, and provide a more reliable data basis for subsequent model training and detection. Among them, preprocessing can include but is not limited to operations such as removing noise, adjusting image size, and normalizing pixel values, which are not limited in this application.

[0048] A random graying method can be used to enhance the generalization ability of the model so that the model can better adapt to different input data. Converting an image from a color to a grayscale image can reduce the dimension of the data and reduce the computational complexity. The present application adopts a random graying method, which means that not all images are simply converted into grayscale images in a fixed manner, but graying is performed with a certain randomness. By adopting a random graying method, the model can be exposed to more different forms of input data, avoiding the model from being overly dependent on specific color information, so that the model can learn more essential image features, rather than relying solely on color differences, thereby enhancing the generalization ability of the model, so that it can better adapt to different lighting conditions, different shooting equipment, and other factors that may cause color changes.

[0049] S60: Perform distracted driving behavior detection on the distracted driving detection image using a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0050] After obtaining the trained target distracted driving behavior detection model, the target distracted driving behavior detection model can be used to perform distracted driving behavior detection on the preprocessed distracted detection image, thereby obtaining a more accurate distracted driving behavior detection result. The specific distracted driving behavior detection steps can be found in the detailed description of the following embodiment, and this application will not repeat them here.

[0051] In this embodiment, a distracted driving behavior image and a cab image are acquired to annotate the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set; and an original detection model is improved to obtain an initial distracted driving behavior detection model, so that the distracted driving behavior data set is further divided into a training set and a validation set according to a preset ratio, and the initial distracted driving behavior detection model is trained to obtain a target distracted driving behavior detection model, so that the cab image can be preprocessed to obtain a to-be-distracted detection image, and then the to-be-distracted detection image can be subjected to distracted driving behavior detection by the target distracted driving behavior detection model to obtain a distracted driving behavior detection result, which can effectively improve the accuracy of the distracted driving behavior detection result and is conducive to achieving a more accurate and efficient distracted driving behavior detection purpose.

[0052] In a possible implementation, the distracted driving behavior detection is performed on the distracted driving behavior detection image by using the target distracted driving behavior detection model to obtain the distracted driving behavior detection result, which refers to the process of how to apply the trained target distracted driving behavior detection model. Figure 2 As shown, Figure 2 The network structure diagram of a target distracted driving behavior detection model is exemplarily shown. Optionally, performing distracted driving behavior detection on the distracted detection image by using the target distracted driving behavior detection model to obtain a distracted driving behavior detection result includes the following steps:

[0053] A1: receiving the input image to be distracted;

[0054] A2: extracting initial features from the image to be detected as distraction through a set of convolutional layers to obtain an initial distraction feature image;

[0055] A3: extracting attention features from the initial distraction feature image through a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image;

[0056] A4: performing multi-scale feature fusion on the attention distraction feature image through a multi-scale feature fusion module to obtain a fused distraction feature image;

[0057] A5: performing an upsampling operation on the fused distraction feature image to obtain an upsampled distraction feature image;

[0058] A6: fusing and enhancing the upsampled distraction feature image through a feature fusion module to obtain an enhanced distraction feature image;

[0059] A7: Determine a distracted driving behavior detection result based on the enhanced distraction feature image.

[0060] The network starts at the input layer, receiving input image data, that is, the image to be distracted. The input image can first pass through several convolutional layers (Conv), which can be responsible for extracting low-level features to obtain an initial distraction feature image.

[0061] After the convolutional layer, the C2f module with the EMA attention mechanism added, such as the C2f-EMA module, is used to extract attention features, and an attention distraction feature image can be obtained. Then, the fast spatial pyramid pooling (Spatial Pyramid Pooling-Fast, SPPF) module, which is a module for multi-scale feature fusion, helps the network process features under different receptive fields, thereby obtaining a fused distraction feature image. Optionally, the SPPF module can perform pooling operations to help the model learn more robust features, which is not limited in this application.

[0062] Next, the upsampling operation (UpSample) is performed, and the low-level features are further fused (Concat) with the high-level features. This process is similar to the feature pyramid structure, which can be used to combine multi-scale features to improve the detection performance for small targets. The C2f module can be used to further fuse and enhance the features to obtain an enhanced distraction feature image. It can be understood that the C2f module can reduce redundant calculations and improve reasoning efficiency by optimizing the channel dimension.

[0063] After feature fusion, the network can be divided into two branches, such as Figure 2 The detection head (Detect) P3 and the detection head (Detect) P4 in the network are used for feature map output of different scales. It can be understood that the final network output target detection result, that is, the distracted driving behavior detection result, can include target category, position, etc., and this application does not limit this.

[0064] See also Figure 3 , Figure 3 The structural diagram of the EMA attention mechanism module is shown as an example. Optionally, extracting the attention feature of the initial distraction feature image by adding a feature fusion module of the exponential moving average EMA attention mechanism to obtain the attention distraction feature image may include the following steps:

[0065] A31: grouping the initial distraction feature images to obtain a first initial distraction feature image and a second initial distraction feature image;

[0066] A32: performing two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooled distraction feature image;

[0067] A33: Activate the two-dimensional global pooling distraction feature image through a 1×1 convolution and an activation function to obtain a first target distraction feature image;

[0068] A34: extracting features from the second initial distraction feature image through convolution to obtain a convolution centroid feature image;

[0069] A35: performing two-dimensional global pooling and activation function activation processing on the convolution centroid feature image to obtain a second target distraction feature image;

[0070] A36: merging the first target distraction feature image and the second target distraction feature image through a cross-dimensional interaction module to obtain a cross-dimensional distraction feature image;

[0071] A37: Perform channel weighting on the cross-dimensional distraction feature image through the weights generated by the activation function to obtain an attention distraction feature image.

[0072] In the EMA module, the input (i.e., the initial distracting feature image) can be first grouped, such as Figure 3 As shown in , the input C×H×W can be divided into G groups C / G×H×W. Then, it is processed by different branches, such as Figure 3 As shown, the upper branch can perform two-dimensional global pooling, 1×1 convolution and activation processing (Sigmoid activation), and the lower branch can perform 3×3 convolution, two-dimensional global pooling and activation processing (Softmax activation). The output features of the two branches can be modulated by the Sigmoid function and normalization operation, and finally merged through the cross-dimensional interaction module and further processed by 1×1 convolution to capture the pairwise relationship at the pixel level. After the final Sigmoid adjustment, such as using the weights generated by Sigmoid for channel weighting, a feature map of the same dimension as the input is output. Among them, the output feature map is used to enhance or weaken the original input features, thereby obtaining an attention distraction feature image.

[0073] EMA is an efficient multi-scale attention mechanism that aims to reduce computational overhead while retaining the key information of each channel, grouping the channel dimension into multiple sub-features, and making the spatial semantic features evenly distributed within each feature group. Specifically, in addition to encoding global information to recalibrate the channel weights in each parallel branch, the output features of the two parallel branches are further aggregated through a cross-dimensional interaction method.

[0074] Optionally, performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooling distracting feature image may include: performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooling distracting feature image by the following formula:

[0075]

[0076] Among them, Z C represents the two-dimensional global pooling distracting feature image corresponding to each channel; C represents the number of input channels; H represents the height of the first initial distracting feature image of the input in the spatial dimension; W represents the width of the first initial distracting feature image of the input in the spatial dimension; j represents the height index; i represents the width index; x c (i, j) represents the input feature value at the position (i, j) of the spatial dimension on channel C.

[0077] Optionally, the improving the original detection model to obtain the initial distracted driving behavior detection model may include: using a focal Euclidean intersection-over-union loss function Focal-EIoU Loss as a regression loss function of the initial distracted driving behavior detection model; the formula of the focal Euclidean intersection-over-union loss function Focal-EIoU Loss is as follows:

[0078] L Focal-EIoU =IoU β L EIoU

[0079]

[0080] Among them, L Focal-EIoU represents the focal Euclidean intersection over union loss function; IoU represents the intersection over union ratio of the predicted box and the true box; γ is a parameter that controls the degree of suppression of outliers; IoU γ represents the weighted term based on IoU, which is used to adaptively adjust the loss of different IoU values; L EIoU represents the enhanced intersection-over-union loss function; ρ 2 represents the Euclidean distance of the calculated element; b represents the center point of the prediction box; b gt represents the center point of the real box; c represents the length of the diagonal of the minimum circumscribed rectangle of the predicted box and the real box; w represents the width of the predicted box; w gt represents the width of the real box; h represents the height of the predicted box; h gt Indicates the height of the real box; C w Indicates the width of the minimum enclosing rectangle; C h Indicates the height of the minimum enclosing rectangle.

[0081] The present invention also proposes a target distracted driving behavior detection device, which is used to implement a driver distracted driving detection method based on improved YOLOv8. Optionally, the distracted driving detection algorithm (i.e., the target distracted driving behavior detection model) can be transplanted into the target distracted driving behavior detection device, and the deep learning acceleration module can be used to accelerate the processing of the neural network.

[0082] Among them, the target distracted driving behavior detection device can mainly include modules such as image information acquisition, image processor, data storage, display and voice prompt. The main function of the image acquisition information module can be to capture and collect the driver's image and video data in real time, and collect the driver's distracted driving behavior, such as drinking water, making phone calls, smoking, etc.; the image processor module can be responsible for analyzing and processing the image and video data transmitted by the image acquisition information module. Its core task is to extract useful information from the original data and classify and identify it; the data storage module can be used to save the collected image data, analysis results and historical records; the display module can be responsible for displaying the system's detection results and related information to the driver or other relevant personnel, and the voice prompt module can be used to remind the driver to pay attention to safe driving through voice output.

[0083] Furthermore, the image acquisition information module is usually composed of a camera and a sensor. The camera is generally installed inside the vehicle to capture the driver's facial expressions, eye movements, head positions, hand movements and other key features. Since the driving environment is divided into day and night, in order to better detect the driver's distracted driving behavior, the present invention needs to use a high-definition infrared camera for image acquisition. The camera can ensure the image quality in most strong light, weak light, and dark environments, thereby reducing the impact on distracted driving detection performance.

[0084] Furthermore, the image processor module usually uses a high-performance processor and advanced image processing algorithms, and plays a vital role in the distracted driving detection device. It is not only the core of data processing, but also needs to have multiple functions to ensure the reliability and real-time performance of the device. It mainly includes image preprocessing, feature extraction, behavior analysis, system self-learning and optimization and other functions. Image preprocessing is the first step of the image processor module, which aims to optimize the original image to improve the accuracy of subsequent analysis; feature extraction is to identify and extract key feature points and areas from the preprocessed image, such as: driver's hands, face and other areas, to provide basic data for subsequent behavior analysis; behavior analysis is based on the extracted feature data, using machine learning and deep learning algorithms to analyze the driver's behavior pattern and determine whether he has distracted behavior; system self-learning means that the image processor module improves its analysis ability and accuracy through continuous learning and optimization.

[0085] Furthermore, the data storage module plays a key role in the distracted driving detection device, responsible for saving and managing various types of data generated by the image processor module, and usually needs to have the characteristics of high capacity and high speed to meet the needs of real-time data processing. This module mainly involves short-term storage: saving recent images and detection results for retrospective analysis in the event of an accident or detection anomaly; long-term storage: recording driver behavior data and system detection results for future auditing, research and algorithm optimization; data backup and synchronization: ensuring data security, preventing loss, and supporting data synchronization with external systems (such as on-board computers, cloud platforms), etc.

[0086] Furthermore, the display module is responsible for presenting the analysis results intuitively to the driver or other users. It usually includes an in-vehicle display screen and a mobile device display interface. The voice prompt module conveys the detection results and warning information to the driver through voice, reminding the driver to drive safely. When distracting behavior or dangerous situations are detected, the driver is warned by voice, such as: when the driver drinks water, the system will broadcast "Do not drink water"; when the driver makes a phone call, the system will broadcast "Do not make a phone call"; when the driver smokes, the system will broadcast "Do not smoke".

[0087] For example, Figure 4 As shown, Figure 4 An architectural diagram of a distracted driving behavior detection device is exemplarily shown. Figure 4 The NT96580 chip is used as the core of the image processing and video encoding device, and a variety of peripherals and interface modules are integrated. The chip is connected to the camera sensor (Lens & Sensor) and peripheral devices (such as display (Liquid Crystal Display, LCD), audio equipment, Ethernet, Universal Serial Bus (Universal Serial Bus, USB) devices, etc.) to form a complete embedded system for image acquisition, processing and output. The description of each module is as follows:

[0088] 1. RTC (Real-Time Clock) & Power Control: Real-time clock and power management module, providing time tracking and power control functions.

[0089] 2. GPIO (General-Purpose Input / Output): General-Purpose Input / Output port, used for simple digital signal interaction with other devices.

[0090] 3. ADC (Analog-to-Digital Converter): Analog-to-digital converter, used to convert analog signals into digital signals.

[0091] 4. SDIO (Secure Digital Input / Output) Controller: SDIO controller, used to connect storage devices such as SD cards.

[0092] 5. USB 2.0 Host / Device: Supports USB 2.0 host / device interface, used to connect peripherals such as keyboards, storage devices, etc.

[0093] 6. UART (Universal Asynchronous Receiver / Transmitter): Universal asynchronous receiver / transmitter, used for serial communication.

[0094] 7. LCD / MIPI Controller: LCD display or MIPI interface controller, used to drive display devices.

[0095] 8. Ethernet PHY: Ethernet physical layer interface, used for network connection.

[0096] 9. Sensor IF (CSI / SLVDS): sensor interface, used to connect image sensors. CSI (CameraSerial Interface) and SLVDS (Serial Low-voltage Differential Signaling) are transmission protocols.

[0097] 10. I2C (Inter-Integrated Circuit): Communication bus used for inter-chip communication, commonly used in sensors, clock chips and other devices.

[0098] 11. SIF (Serial Interface): Serial interface, usually used for simple device communication.

[0099] 12. AMP (Amplifier): Amplifier, used to amplify audio signals.

[0100] 13. MD Engine (Motion Detection Engine): Motion detection engine, used to detect moving objects in the video.

[0101] 14. 3DNR (3D Noise Reduction): three-dimensional noise reduction technology, used for video noise reduction.

[0102] 15. Anti-fog: Anti-fog treatment for optimizing image clarity.

[0103] 16. OSD (On-Screen Display) / Graphics: Screen display control module, responsible for overlaying graphics into the video stream (such as subtitles).

[0104] 17. ARM Cortex A9: Processor core, responsible for handling system logic and application tasks.

[0105] 18. H.264 / H.265 Video Codec: Video encoder, used to compress and decode video. H.264 and H.265 are common and efficient video compression standards.

[0106] 19. DLA (Deep Learning Accelerator): Deep learning accelerator, used to accelerate the calculation of neural networks.

[0107] 20. PEG Engine: JPEG image encoding engine, used to compress images.

[0108] 21. Flash Controller: Flash controller is used to manage and operate flash storage devices.

[0109] 22. SDRAM Controller: Dynamic random access memory controller, used to manage external memory.

[0110] 23. DDR IIIDRAM: Double data rate third generation dynamic random access memory, used to provide high-speed storage.

[0111] 24. SPI (Serial Peripheral Interface): Serial peripheral interface, used for external device communication.

[0112] 25. NOR / NAND: NOR / NAND flash storage is commonly used for program code and data storage.

[0113] 26. Key / Battery Detect: Keyboard and battery detection module, used to detect the connection and battery status of external devices.

[0114] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Adding the EMA attention mechanism in the C2f module helps to enhance the feature expression capability, so that the model can better focus on the important areas in the image; 2. The P5 detection head is deleted from the head (Head) in YOLOv8 to reduce the amount of calculation and parameters of the model, thereby reducing the complexity and computational cost of the model; 3. Focal-EIoU Loss is used to replace CIoU Loss as the regression loss function to improve the accuracy and convergence speed of the model. 4. Finally, the distracted driving detection algorithm is transplanted into the device, and the deep learning acceleration module is used to accelerate the processing of the neural network. The hardware and software system is designed and implemented, and the performance of the distracted driving detection system is comprehensively tested and analyzed.

[0115] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0116] In one embodiment, a distracted driving behavior detection device based on a neural network model is provided, and the distracted driving behavior detection device based on a neural network model corresponds one-to-one to the distracted driving behavior detection method based on a neural network model in the above embodiment. Figure 5 As shown, the distracted driving behavior detection device based on the neural network model includes an acquisition module 101, a first processing module 102, a second processing module 103, a third processing module 104, a fourth processing module 105 and a detection module 106. The functional modules are described in detail as follows:

[0117] An acquisition module 101 is used to acquire a distracted driving behavior image and a cab image;

[0118] A first processing module 102 is used to label the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior dataset;

[0119] The second processing module 103 is used to improve the original detection model to obtain an initial distracted driving behavior detection model;

[0120] A third processing module 104 is used to divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, train the initial distracted driving behavior detection model, and obtain a target distracted driving behavior detection model;

[0121] The fourth processing module 105 is used to pre-process the cab image to obtain an image to be detected for distraction;

[0122] The detection module 106 is configured to perform distracted driving behavior detection on the distracted driving detection image using a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0123] In one embodiment, the detection module 106 is used to perform distracted driving behavior detection on the distracted driving detection image using a target distracted driving behavior detection model to obtain a distracted driving behavior detection result, specifically for:

[0124] Receiving an input image to be detected as distraction;

[0125] Extract initial features from the image to be detected for distraction through a set of convolutional layers to obtain an initial distraction feature image;

[0126] The initial distraction feature image is subjected to attention feature extraction by a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image;

[0127] Performing multi-scale feature fusion on the attention distraction feature image through a multi-scale feature fusion module to obtain a fused distraction feature image;

[0128] Performing an upsampling operation on the fused distraction feature image to obtain an upsampled distraction feature image;

[0129] The upsampled distraction feature image is fused and enhanced by a feature fusion module to obtain an enhanced distraction feature image;

[0130] A distracted driving behavior detection result is determined according to the enhanced distraction feature image.

[0131] In one embodiment, the detection module 106 is used to extract attention features from the initial distraction feature image through a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image, specifically for:

[0132] Grouping the initial distraction feature images to obtain a first initial distraction feature image and a second initial distraction feature image;

[0133] Performing one-dimensional global pooling on the first initial distracting feature image to obtain a global pooled distracting feature image;

[0134] Performing feature extraction on the second initial distraction feature image through convolution to obtain a convolution centroid feature image;

[0135] The global pooled distraction feature image and the convolutional centroid feature image are modulated by an activation function and a normalization operation, and are merged by a cross-dimensional interaction module to obtain a cross-dimensional distraction feature image;

[0136] The cross-dimensional distraction feature image is adjusted through an activation function to obtain an attention distraction feature image.

[0137] In one embodiment, the detection module 106 is used to perform two-dimensional global pooling on the first initial distraction feature image to obtain a two-dimensional global pooling distraction feature image, specifically for:

[0138] The step of performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooling distracting feature image is implemented by the following formula:

[0139]

[0140] Among them, Z C represents the two-dimensional global pooling distracting feature image corresponding to each channel; C represents the number of input channels; H represents the height of the first initial distracting feature image of the input in the spatial dimension; W represents the width of the first initial distracting feature image of the input in the spatial dimension; j represents the height index; i represents the width index; x c (i, j) represents the input feature value at the position (i, j) of the spatial dimension on channel C.

[0141] In one embodiment, the second processing module is used to improve the original detection model to obtain an initial distracted driving behavior detection model, specifically for:

[0142] The focal-Euclidean intersection-over-union loss function is used as the regression loss function of the initial distracted driving behavior detection model;

[0143] The formula of the focal Euclidean intersection-over-union loss function Focal-EIoU Loss is as follows:

[0144] L Focal-EIoU =IoU γ L EIoU

[0145]

[0146] Among them, L Focal-EIoU represents the focal Euclidean intersection over union loss function; IoU represents the intersection over union ratio of the predicted box and the true box; γ is a parameter that controls the degree of suppression of outliers; IoU γ represents the weighted term based on IoU, which is used to adaptively adjust the loss of different IoU values; L EIoU represents the enhanced intersection-over-union loss function; ρ 2 represents the Euclidean distance of the calculated element; b represents the center point of the prediction box; b gtrepresents the center point of the real box; c represents the length of the diagonal of the minimum circumscribed rectangle of the predicted box and the real box; w represents the width of the predicted box; w gt represents the width of the real box; h represents the height of the predicted box; h gt Indicates the height of the real box; C w Indicates the width of the minimum enclosing rectangle; C h Indicates the height of the minimum enclosing rectangle.

[0147] The present invention provides a distracted driving behavior detection device based on a neural network model. By acquiring a distracted driving behavior image and a cab image, the distracted driving behavior in the distracted driving behavior image is annotated to obtain a distracted driving behavior data set. The original detection model is improved to obtain an initial distracted driving behavior detection model, so that the distracted driving behavior data set is further divided into a training set and a verification set according to a preset ratio, and the initial distracted driving behavior detection model is trained to obtain a target distracted driving behavior detection model, so that the cab image can be pre-processed to obtain a to-be-distracted detection image, and then the to-be-distracted detection image can be subjected to distracted driving behavior detection by the target distracted driving behavior detection model to obtain a distracted driving behavior detection result. The accuracy of the distracted driving behavior detection result can be effectively improved, which is conducive to achieving the purpose of more accurate and efficient distracted driving behavior detection.

[0148] For the specific definition of the distracted driving behavior detection device based on the neural network model, please refer to the definition of the distracted driving behavior detection method based on the neural network model above, which will not be repeated here. Each module in the above-mentioned distracted driving behavior detection device based on the neural network model can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0149] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 6As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile and / or volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external client through a network connection. When the computer program is executed by the processor, it realizes the functions or steps on the server side of a distracted driving behavior detection method based on a neural network model.

[0150] In one embodiment, a computer device is provided. The computer device may be a client, and its internal structure diagram may be as follows: Figure 7 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external server via a network connection. When the computer program is executed by the processor, it implements the functions or steps of the client side of a distracted driving behavior detection method based on a neural network model.

[0151] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program:

[0152] Acquire distracted driving behavior images and cab images;

[0153] Annotating the distracted driving behaviors in the distracted driving behavior images to obtain a distracted driving behavior dataset;

[0154] The original detection model is improved to obtain the initial distracted driving behavior detection model;

[0155] Dividing the distracted driving behavior data set into a training set and a validation set according to a preset ratio, training the initial distracted driving behavior detection model, and obtaining a target distracted driving behavior detection model;

[0156] Preprocessing the cab image to obtain an image to be detected as distraction;

[0157] The distracted driving behavior detection is performed on the distracted driving behavior detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0158] The present invention provides a computer device, which obtains a distracted driving behavior image and a cab image to annotate the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set; and improves an original detection model to obtain an initial distracted driving behavior detection model, so as to further divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, train the initial distracted driving behavior detection model to obtain a target distracted driving behavior detection model, so as to pre-process the cab image to obtain a to-be-distracted detection image, and then perform distracted driving behavior detection on the to-be-distracted detection image through the target distracted driving behavior detection model to obtain a distracted driving behavior detection result, which can effectively improve the accuracy of the distracted driving behavior detection result and is conducive to achieving the purpose of more accurate and efficient distracted driving behavior detection.

[0159] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0160] Acquire distracted driving behavior images and cab images;

[0161] Annotating the distracted driving behaviors in the distracted driving behavior images to obtain a distracted driving behavior dataset;

[0162] The original detection model is improved to obtain the initial distracted driving behavior detection model;

[0163] Dividing the distracted driving behavior data set into a training set and a validation set according to a preset ratio, training the initial distracted driving behavior detection model, and obtaining a target distracted driving behavior detection model;

[0164] Preprocessing the cab image to obtain an image to be detected as distraction;

[0165] The distracted driving behavior detection is performed on the distracted driving behavior detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

[0166] The present invention provides a computer-readable storage medium, which obtains a distracted driving behavior image and a cab image to annotate the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set; and improves the original detection model to obtain an initial distracted driving behavior detection model, so as to further divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, and train the initial distracted driving behavior detection model to obtain a target distracted driving behavior detection model, so that the cab image can be preprocessed to obtain a to-be-distracted detection image, and then the to-be-distracted detection image can be subjected to distracted driving behavior detection by the target distracted driving behavior detection model to obtain a distracted driving behavior detection result, which can effectively improve the accuracy of the distracted driving behavior detection result and is conducive to achieving the purpose of more accurate and efficient distracted driving behavior detection.

[0167] It should be noted that the above functions or steps that can be implemented by the computer-readable storage medium or computer device can refer to the relevant descriptions on the server side and the client side in the aforementioned method embodiment. To avoid repetition, they will not be described one by one here.

[0168] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0169] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0170] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A distracted driving behavior detection method based on a neural network model, characterized in that: The method comprises: Acquire distracted driving behavior images and cab images; Annotating the distracted driving behaviors in the distracted driving behavior images to obtain a distracted driving behavior dataset; The original detection model is improved to obtain the initial distracted driving behavior detection model; Dividing the distracted driving behavior data set into a training set and a validation set according to a preset ratio, training the initial distracted driving behavior detection model, and obtaining a target distracted driving behavior detection model; Preprocessing the cab image to obtain an image to be detected as distraction; The distracted driving behavior detection is performed on the distracted driving behavior detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

2. The distracted driving behavior detection method based on a neural network model according to claim 1, characterized in that: The distracted driving behavior detection is performed on the distracted driving detection image by using the target distracted driving behavior detection model to obtain a distracted driving behavior detection result, including: Receiving an input image to be distracted; Extract initial features from the image to be detected for distraction through a set of convolutional layers to obtain an initial distraction feature image; The initial distraction feature image is subjected to attention feature extraction by a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image; Performing multi-scale feature fusion on the attention distraction feature image through a multi-scale feature fusion module to obtain a fused distraction feature image; Performing an upsampling operation on the fused distraction feature image to obtain an upsampled distraction feature image; The upsampled distraction feature image is fused and enhanced by a feature fusion module to obtain an enhanced distraction feature image; A distracted driving behavior detection result is determined according to the enhanced distraction feature image.

3. The distracted driving behavior detection method based on a neural network model according to claim 2, characterized in that: The initial distraction feature image is subjected to attention feature extraction by a feature fusion module with an exponential moving average EMA attention mechanism to obtain an attention distraction feature image, including: Grouping the initial distraction feature images to obtain a first initial distraction feature image and a second initial distraction feature image; Performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooled distracting feature image; Activate the two-dimensional global pooled distraction feature image through a 1×1 convolution and an activation function to obtain a first target distraction feature image; Performing feature extraction on the second initial distraction feature image through convolution to obtain a convolution centroid feature image; Performing two-dimensional global pooling and activation function activation processing on the convolution centroid feature image to obtain a second target distraction feature image; Merging the first target distraction feature image and the second target distraction feature image through a cross-dimensional interaction module to obtain a cross-dimensional distraction feature image; The cross-dimensional distraction feature image is channel-weighted by weights generated by an activation function to obtain an attention distraction feature image.

4. The distracted driving behavior detection method based on a neural network model according to claim 3, characterized in that: The step of performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooled distracting feature image includes: The step of performing two-dimensional global pooling on the first initial distracting feature image to obtain a two-dimensional global pooling distracting feature image is implemented by the following formula: Among them, Z C represents the two-dimensional global pooling distracting feature image corresponding to each channel; C represents the number of input channels; H represents the height of the first initial distracting feature image of the input in the spatial dimension; W represents the width of the first initial distracting feature image of the input in the spatial dimension; j represents the height index; i represents the width index; x c (i, j) represents the input feature value at the position (i, j) of the spatial dimension on channel C.

5. The distracted driving behavior detection method based on a neural network model according to claim 3, characterized in that: The original detection model is improved to obtain an initial distracted driving behavior detection model, including: The focal-Euclidean intersection-over-union loss function is used as the regression loss function of the initial distracted driving behavior detection model; The formula of the focal Euclidean intersection-over-union loss function Focal-EIoU Loss is as follows: L Focal-EIoU =IoU β L EIoU Among them, L Focal-EIoU represents the focal Euclidean intersection over union loss function; IoU represents the intersection over union ratio of the predicted box and the true box; γ is a parameter that controls the degree of suppression of outliers; IoU γ represents the weighted term based on IoU, which is used to adaptively adjust the loss of different IoU values; L EIoU represents the enhanced intersection-over-union loss function; ρ 2 represents the Euclidean distance of the calculated element; b represents the center point of the prediction box; b gt represents the center point of the real box; c represents the length of the diagonal of the minimum circumscribed rectangle of the predicted box and the real box; w represents the width of the predicted box; w gt represents the width of the real box; h represents the height of the predicted box; h gt Indicates the height of the real box; C w Indicates the width of the minimum enclosing rectangle; C h Indicates the height of the minimum enclosing rectangle.

6. A distracted driving behavior detection device based on a neural network model, characterized in that: The distracted driving behavior detection device based on the neural network model includes: An acquisition module, used for acquiring distracted driving behavior images and cab images; A first processing module is used to label the distracted driving behavior in the distracted driving behavior image to obtain a distracted driving behavior data set; The second processing module is used to improve the original detection model to obtain an initial distracted driving behavior detection model; A third processing module is used to divide the distracted driving behavior data set into a training set and a validation set according to a preset ratio, train the initial distracted driving behavior detection model, and obtain a target distracted driving behavior detection model; A fourth processing module, used for preprocessing the cab image to obtain an image to be detected for distraction; The detection module is used to perform distracted driving behavior detection on the distracted driving detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result.

7. The distracted driving behavior detection device based on a neural network model according to claim 6, characterized in that: The detection module is used to perform distracted driving behavior detection on the distracted driving detection image through a target distracted driving behavior detection model to obtain a distracted driving behavior detection result, and is specifically used to: Receiving an input image to be distracted; Extract initial features from the image to be detected for distraction through a set of convolutional layers to obtain an initial distraction feature image; The initial distraction feature image is subjected to attention feature extraction by a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image; Performing multi-scale feature fusion on the attention distraction feature image through a multi-scale feature fusion module to obtain a fused distraction feature image; Performing an upsampling operation on the fused distraction feature image to obtain an upsampled distraction feature image; The upsampled distraction feature image is fused and enhanced by a feature fusion module to obtain an enhanced distraction feature image; A distracted driving behavior detection result is determined according to the enhanced distraction feature image.

8. The distracted driving behavior detection device based on a neural network model according to claim 7, characterized in that: The detection module is used to extract attention features from the initial distraction feature image through a feature fusion module that adds an exponential moving average (EMA) attention mechanism to obtain an attention distraction feature image, specifically for: Grouping the initial distraction feature images to obtain a first initial distraction feature image and a second initial distraction feature image; Performing one-dimensional global pooling on the first initial distracting feature image to obtain a global pooled distracting feature image; Performing feature extraction on the second initial distraction feature image through convolution to obtain a convolution centroid feature image; The global pooled distraction feature image and the convolutional centroid feature image are modulated by an activation function and a normalization operation, and are merged by a cross-dimensional interaction module to obtain a cross-dimensional distraction feature image; The cross-dimensional distraction feature image is adjusted through an activation function to obtain an attention distraction feature image.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the distracted driving behavior detection method based on the neural network model as described in any one of claims 1 to 5 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the distracted driving behavior detection method based on a neural network model as described in any one of claims 1 to 5 is implemented.

Images