Rolov5-based driver drinking and fatigue driving comprehensive detection system

Abstract

The invention discloses a yolov5-based driver drinking and fatigue driving comprehensive detection system. According to the invention, through the comprehensive monitoring system based on the YOLOv5 target detection algorithm, whether the driver is in an unsafe state such as drinking, fatigue driving or smoking can be identified and judged in real time. By integrating various sensor data, such as an alcohol sensor and a smoke sensor, and an image recognition technology of deep learning, the system can realize multi-dimensional state monitoring of a driver, and gives an alarm in time to prevent potential risks. Therefore, the driving safety can be remarkably improved, and particularly, unsafe behaviors such as drinking, fatigue and smoking can be monitored in a complex driving environment. Experimental results show that the system can maintain a high response speed on the basis of high precision, and can adapt to various driving environments. By integrating various sensors and a deep learning algorithm, the system can realize comprehensive monitoring of driver behaviors, and traffic accidents are reduced.

Description

Technical Field

[0001] The invention belongs to the technical field of driving safety detection, and in particular is a comprehensive detection system for driver drinking and fatigue driving based on yolov5. Background Art

[0002] With the increasing emphasis on traffic safety in modern society, how to detect and prevent unsafe driving behaviors of drivers, especially fatigue driving and drunk driving, has become a research hotspot. Existing driver behavior monitoring systems are mainly based on two types of technologies: one is to detect changes in the in-vehicle environment and driver's physical signs (such as alcohol concentration in the car, heart rate detection, etc.) through sensors; the other is to use image recognition technology to analyze the driver's facial expressions and behaviors, such as yawning, closing eyes, etc., and then determine whether the driver is fatigued or has unsafe driving behaviors.

[0003] However, most traditional monitoring methods have the following limitations:

[0004] 1. Single function: Many monitoring systems can only identify a certain type of behavior, such as fatigue driving or drunk driving, and cannot achieve multi-dimensional comprehensive monitoring.

[0005] 2. Limited detection accuracy: Some sensor-based systems are prone to false positives or false negatives due to factors such as environmental changes.

[0006] 3. Recognition delay: Some systems rely on sensors to collect data, have a slow response speed, and are unable to monitor changes in the driver’s status in real time.

[0007] Therefore, there is an urgent need for a comprehensive system that can combine multiple technologies to simultaneously achieve real-time and efficient detection of drivers' drinking, fatigue driving and other unsafe behaviors. Summary of the invention

[0008] The purpose of the present invention is to provide a comprehensive detection system for driver drinking and fatigue driving based on yolov5 in order to solve the above-mentioned problems.

[0009] The technical solution adopted by the present invention is as follows: a comprehensive detection system for driver drinking and fatigue driving based on yolov5, the system comprising: an ESP32-CAM module, an alcohol sensor and a smoke sensor module, an Arduino UNO development board module, a WiFi module, a YOLOv5 image recognition algorithm module and a Flask server module;

[0010] The ESP32-CAM module integrates camera and WiFi functions to capture the driver's facial image in real time and transmit it to the server via WiFi. The ESP32-CAM module strikes a good balance between performance and power consumption and is suitable for embedded systems.

[0011] The alcohol sensor and smoke sensor modules are used to detect the alcohol and smoke concentrations in the car environment, and the data is collected and processed through the Arduino development board.

[0012] As the main control device of the sensor and camera, the Arduino UNO development board module is responsible for the collection, processing and transmission of sensor data.

[0013] The WiFi module realizes the wireless transmission of data from Arduino UNO to the server, supporting real-time data transmission and feedback.

[0014] The YOLOv5 image recognition algorithm module detects the driver's fatigue state (such as closing eyes, yawning) and whether he smokes by training a specific data set, and has high real-time performance. This algorithm is one of the most advanced target detection algorithms currently, and can respond quickly while maintaining high-precision detection results.

[0015] Flask server module: used to receive sensor and image data from Arduino, analyze the driver image based on the YOLOv5 model, and generate a comprehensive judgment result based on the sensor input data.

[0016] In a preferred embodiment, the detection process of the YOLOv5 image recognition algorithm module includes:

[0017] 1. Input image preprocessing: The input image is processed into a size suitable for the network (such as 640x640) through resizing and normalization. This helps improve the computational efficiency of the model.

[0018] formula:

[0019]

[0020] Where x is the pixel value, x norm is the normalized pixel value.

[0021] 2. Feature extraction: YOLOv5 uses the CSPNet (Cross Stage Partial Networks) structure for feature extraction and adopts residual blocks to extract features of different scales to maintain high efficiency.

[0022] Feature mapping function:

[0023] y=f(x)+x

[0024] Among them, f(x) represents the nonlinear mapping in the network layer, and x is the input feature.

[0025] 3. Object detection and classification: The output of YOLOv5 is the object bounding box and category prediction in each grid. It adopts the Anchor mechanism and combines IoU (Intersection over Union) as a measure of object matching.

[0026] Coordinate formula for predicted bounding box:

[0027] B x =σ(t x )+C x ,B y =σ(t y )+C y

[0028]

[0029] Among them, B x and B y is the predicted bounding box center coordinate, B w and B h is the predicted width and height, C x and C y is the grid center coordinate, p w and p h is the width and height of the prior box, and σ is the Sigmoid function.

[0030] 4. Loss function: The loss function of YOLOv5 consists of three parts:

[0031] ο Bounding Box Loss: GIoU (Generalized Intersection over Union) is used as the evaluation criterion for bounding box prediction.

[0032] οClassification loss: based on binary cross-entropy loss (Binary Cross-Entropy Loss).

[0033] οConfidence loss: used to measure whether there is an object in the prediction box, based on Focal Loss.

[0034] 5. Loss function formula:

[0035]

[0036] 6. Reasoning and result output: Filter out the targets through the maximum confidence score, perform NMS (Non-Maximum Suppression) processing, remove overlapping detection boxes, and finally output the detection results.

[0037] In a preferred embodiment, the data received by the Flask server module includes:

[0038] 1. Smoke sensor data: used to detect whether there is smoking in the cab. The data is collected by MQ-2 or similar sensors.

[0039] 2. Alcohol sensor data: collected by sensors such as MQ-3 and used to detect whether the driver has been drinking.

[0040] YOLOv5 image data analysis: Detects driver fatigue (such as closing eyes, yawning) and smoking behavior. The results of YOLOv5 analysis are combined with sensor data to form a comprehensive driving safety judgment.

[0041] In a preferred embodiment, after receiving the sensor data, the Flask server uses the YOLOv5 model in the YOLOv5 image recognition algorithm module to analyze the driver image to determine whether there is unsafe behavior such as yawning, closing eyes or smoking. The data analysis process of the YOLOv5 model is as follows:

[0042] 1. Smoke and alcohol detection:

[0043] The smoke detection formula is:

[0044]

[0045] Among them, R L is the load resistance, R0 is the sensor calibration value, V out is the sensor output voltage, V in is the input voltage.

[0046] The alcohol test formula is:

[0047]

[0048] Among them, A BAC is the blood alcohol content, R alcohol is the resistance during detection, and k is the calibration coefficient.

[0049] 2. Comprehensive judgment logic:

[0050] Based on the analysis results of sensors and YOLOv5, the Flask server combines multiple data to conduct a comprehensive analysis of driving behavior. The following is the judgment logic:

[0051] a. When a high smoke value is detected and YOLOv5 recognizes a cigarette in the driver’s facial area, it is inferred that the driver is smoking.

[0052] b. When the alcohol sensor detects a high alcohol concentration and YOLOv5 does not detect any smoke, it is inferred that the driver is at risk of drunk driving.

[0053] c. When YOLOv5 detects closing eyes or yawning, it infers that the driver may be in a state of fatigue driving.

[0054] Comprehensive formula:

[0055] Let the detected smoke value be V smoke , alcohol concentration is A BAC , the detection result of YOLOv5 is Y fatigue (Fatigue) and Y smoking (Smoking), the comprehensive judgment results are:

[0056] S risk =αV smoke +βA BAC +γY fatigue +δY smoking

[0057] Among them, α, β, γ, and δ are the weights of smoke, alcohol, fatigue, and smoking, respectively.

[0058] The Arduino UNO development board module finally feeds back the above results to the user terminal through the Flask server, displaying the driver's behavior status and risk level in real time.

[0059] In a preferred embodiment, the YOLOv5 model is used to detect the driver's facial behavior (such as closing eyes, yawning and other fatigue driving related behaviors) and smoking behavior. The specific detection process is based on the target detection algorithm:

[0060] YOLOv5 uses a deep convolutional neural network to detect objects in images and output bounding box coordinates and class probabilities.

[0061] Loss function formula:

[0062]

[0063] Classification loss, used to determine the target category.

[0064] Bounding box regression loss, used to predict the accuracy of the object box.

[0065] Confidence loss is used to determine whether there is an object in the box.

[0066] Through the features in the image data, YOLOv5 can recognize the driver's facial expressions, movements (such as closing eyes, yawning, etc.), and whether he is smoking. Combining these recognition results, the system can judge the driver's status in real time.

[0067] Assume the result is:

[0068] Y fatigue : When the driver is detected to be fatigued (such as closing eyes, yawning), it is set to 1, otherwise it is 0.

[0069] Y smoking : When the driver's smoking behavior is detected, it is set to 1, otherwise it is 0.

[0070] In a preferred embodiment, the YOLOv5 model collects the smoke concentration in the air through the smoke sensor and returns an analog voltage value, which is proportional to the smoke concentration. We use the voltage value returned by the sensor as the smoke concentration V smoke , processed by the following formula:

[0071] S smoke =α·V smoke

[0072] Where α is the sensor calibration coefficient, V smoke The voltage value output by the sensor.

[0073] Alcohol sensor (such as MQ-3) detects alcohol concentration:

[0074] Similarly, the alcohol sensor generates a voltage output according to the alcohol content in the air. Assume that its output value is V alcohol , the alcohol concentration can be converted by the following formula:

[0075] S alcohol =β·V alcohol

[0076] Where β is the sensor calibration coefficient, V alcohol The voltage value output by the sensor.

[0077] In a preferred embodiment, the YOLOv5 model fuses the detection results of YOLOv5 with the data of the sensor, and the system can make judgments according to the following logic:

[0078] Fatigue detection:

[0079] If Y fatigue =1, it is judged that the driver is in a fatigue driving state.

[0080] Smoking detection:

[0081] If Y smoking=1, and S smoke If the value is higher than the threshold, it is judged that the driver is smoking.

[0082] Risks of drinking:

[0083] If S alcohol >threshold, the system determines that the driver is at risk of drunk driving.

[0084] Comprehensive judgment formula:

[0085] Based on various inputs, the system's judgment formula is as follows:

[0086] S risk =γ·Y fatigue +δ·Y smoking +α·V smoke +β·V alcohol

[0087] in:

[0088] γ, δ: are the weights of fatigue driving and smoking behaviors, respectively.

[0089] α, β: calibration coefficients for smoke and alcohol concentrations, respectively.

[0090] According to this comprehensive risk value S risk , the system can make corresponding warnings and processing decisions. For example, if S risk If a certain threshold is exceeded, the system will sound an alarm to prompt the driver to stop driving.

[0091] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0092] In the present invention, through the comprehensive monitoring system based on the YOLOv5 target detection algorithm, it is possible to identify and judge in real time whether the driver is in an unsafe state such as drinking, fatigue driving or smoking. By integrating multiple sensor data (such as alcohol sensors, smoke sensors) and deep learning image recognition technology, this system can realize multi-dimensional state monitoring of the driver, and issue alarms in time to prevent potential risks. Thereby, driving safety can be significantly improved, especially in the monitoring of unsafe behaviors such as drinking, fatigue and smoking in complex driving environments. Experimental results show that the system can maintain a high response speed on the basis of high precision, and can adapt to a variety of driving environments. By integrating multiple sensors and deep learning algorithms, the system can realize comprehensive monitoring of driver behavior and reduce the occurrence of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 It is a flow chart of the overall system operation of the present invention. DETAILED DESCRIPTION

[0094] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0095] Reference Figure 1 ,

[0096] A comprehensive detection system for driver drinking and fatigue driving based on yolov5,

[0097] 1. Hardware configuration: This system uses ESP32-CAM module, Arduino UNO development board, alcohol sensor, smoke sensor and other hardware devices for data collection and transmission. The camera is used to capture the driver's image in real time, the sensor detects the in-car environment, and all data is transmitted to the server via WiFi for processing.

[0098] 2. Software implementation: On the server side, the Flask framework and YOLOv5 model are used to analyze images and sensor data in real time. The YOLOv5 model is trained on an NVIDIA 3080 GPU to quickly identify whether the driver is in a fatigued driving state, and combines sensor data to determine whether the driver is drinking or smoking.

[0099] 3. Data processing and feedback: After receiving the data, the Flask server stores the test results in the SQLite database and feeds back the analysis results to the driver in real time. At the same time, the system will also regularly generate test reports for managers to analyze the driver's driving status and further improve the safety of fleet management.

[0100] The system includes: ESP32-CAM module, alcohol sensor and smoke sensor module, Arduino UNO development board module, WiFi module, YOLOv5 image recognition algorithm module and Flask server module;

[0101] The ESP32-CAM module integrates camera and WiFi functions to capture the driver's facial image in real time and transmit it to the server via WiFi. The ESP32-CAM module strikes a good balance between performance and power consumption and is suitable for embedded systems.

[0102] The alcohol sensor and smoke sensor modules are used to detect the alcohol and smoke concentrations in the car environment, and the data is collected and processed through the Arduino development board.

[0103] As the main control device of the sensor and camera, the Arduino UNO development board module is responsible for the collection, processing and transmission of sensor data.

[0104] The WiFi module realizes the wireless transmission of data from Arduino UNO to the server, supporting real-time data transmission and feedback.

[0105] The YOLOv5 image recognition algorithm module detects the driver's fatigue state (such as closing eyes, yawning) and whether he smokes by training a specific data set, and has high real-time performance. This algorithm is one of the most advanced target detection algorithms currently, and can respond quickly while maintaining high-precision detection results.

[0106] Flask server module: used to receive sensor and image data from Arduino, analyze the driver image based on the YOLOv5 model, and generate a comprehensive judgment result based on the sensor input data.

[0107] The detection process of the YOLOv5 image recognition algorithm module includes:

[0108] 1. Input image preprocessing: The input image is processed into a size suitable for the network (such as 640x640) through resizing and normalization. This helps improve the computational efficiency of the model.

[0109] formula:

[0110]

[0111] Where x is the pixel value, x norm is the normalized pixel value.

[0112] 2. Feature extraction: YOLOv5 uses the CSPNet (Cross Stage Partial Networks) structure for feature extraction and adopts residual blocks to extract features of different scales to maintain high efficiency.

[0113] Feature mapping function:

[0114] y=f(x)+x

[0115] Among them, f(x) represents the nonlinear mapping in the network layer, and x is the input feature.

[0116] 3. Object detection and classification: The output of YOLOv5 is the object bounding box and category prediction in each grid. It adopts the Anchor mechanism and combines IoU (Intersection over Union) as a measure of object matching.

[0117] Coordinate formula for predicted bounding box:

[0118] B x =σ(t x )+Cx ,B y =σ(t y )+C y

[0119]

[0120] Among them, B x and B y is the predicted bounding box center coordinate, B w and B h is the predicted width and height, C x and C y is the grid center coordinate, p w and p h is the width and height of the prior box, and σ is the Sigmoid function.

[0121] 4. Loss function: The loss function of YOLOv5 consists of three parts:

[0122] ο Bounding Box Loss: GIoU (Generalized Intersection over Union) is used as the evaluation criterion for bounding box prediction.

[0123] οClassification loss: based on binary cross-entropy loss (Binary Cross-Entropy Loss).

[0124] οConfidence loss: used to measure whether there is an object in the prediction box, based on Focal Loss.

[0125] 5. Loss function formula:

[0126]

[0127] 6. Reasoning and result output: Filter out the targets through the maximum confidence score, perform NMS (Non-Maximum Suppression) processing, remove overlapping detection boxes, and finally output the detection results.

[0128] The data received by the Flask server module includes:

[0129] 1. Smoke sensor data: used to detect whether there is smoking in the cab. The data is collected by MQ-2 or similar sensors.

[0130] 2. Alcohol sensor data: collected by sensors such as MQ-3 and used to detect whether the driver has been drinking.

[0131] YOLOv5 image data analysis: Detects driver fatigue (such as closing eyes, yawning) and smoking behavior. The results of YOLOv5 analysis are combined with sensor data to form a comprehensive driving safety judgment.

[0132] After receiving the sensor data, the Flask server uses the YOLOv5 model in the YOLOv5 image recognition algorithm module to analyze the driver image to determine whether there are unsafe behaviors such as yawning, closing eyes or smoking. The data analysis process of the YOLOv5 model is as follows:

[0133] 1. Smoke and alcohol detection:

[0134] The smoke detection formula is:

[0135]

[0136] Among them, R L is the load resistance, R0 is the sensor calibration value, V out is the sensor output voltage, V in is the input voltage.

[0137] The alcohol test formula is:

[0138]

[0139] Among them, A BAC is the blood alcohol content, R alcohol is the resistance during detection, and k is the calibration coefficient.

[0140] 2. Comprehensive judgment logic:

[0141] Based on the analysis results of sensors and YOLOv5, the Flask server combines multiple data to conduct a comprehensive analysis of driving behavior. The following is the judgment logic:

[0142] a. When a high smoke value is detected and YOLOv5 recognizes a cigarette in the driver’s facial area, it is inferred that the driver is smoking.

[0143] b. When the alcohol sensor detects a high alcohol concentration and YOLOv5 does not detect any smoke, it is inferred that the driver is at risk of drunk driving.

[0144] c. When YOLOv5 detects closing eyes or yawning, it infers that the driver may be in a state of fatigue driving.

[0145] Comprehensive formula:

[0146] Let the detected smoke value be V smoke , alcohol concentration is A BAC, the detection result of YOLOv5 is Y fatigue (Fatigue) and Y smoking (Smoking), the comprehensive judgment results are:

[0147] S risk =αV smoke +βA BAC +γY fatigue +δY smoking

[0148] Among them, α, β, γ, and δ are the weights of smoke, alcohol, fatigue, and smoking, respectively.

[0149] The Arduino UNO development board module finally feeds back the above results to the user terminal through the Flask server, displaying the driver's behavior status and risk level in real time.

[0150] The YOLOv5 model is used to detect the driver's facial behaviors (such as closing eyes, yawning, and other behaviors related to fatigue driving) and smoking behaviors. The specific detection process is based on the target detection algorithm:

[0151] YOLOv5 uses a deep convolutional neural network to detect objects in images and output bounding box coordinates and class probabilities.

[0152] Loss function formula:

[0153]

[0154] Classification loss, used to determine the target category.

[0155] Bounding box regression loss, used to predict the accuracy of the object box.

[0156] Confidence loss is used to determine whether there is an object in the box.

[0157] Through the features in the image data, YOLOv5 can recognize the driver's facial expressions, movements (such as closing eyes, yawning, etc.), and whether he is smoking. Combining these recognition results, the system can judge the driver's status in real time.

[0158] Assume the result is:

[0159] Y fatigue : When the driver is detected to be fatigued (such as closing eyes, yawning), it is set to 1, otherwise it is 0.

[0160] Y smoking : When the driver's smoking behavior is detected, it is set to 1, otherwise it is 0.

[0161] The YOLOv5 model uses the smoke sensor to collect the smoke concentration in the air and return an analog voltage value, which is proportional to the smoke concentration. We use the voltage value returned by the sensor as the smoke concentration V smoke , processed by the following formula:

[0162] S smoke =α·V smoke

[0163] Where α is the sensor calibration coefficient, V smoke The voltage value output by the sensor.

[0164] Alcohol sensor (such as MQ-3) detects alcohol concentration:

[0165] Similarly, the alcohol sensor generates a voltage output according to the alcohol content in the air. Assume that its output value is V alcohol , the alcohol concentration can be converted by the following formula:

[0166] S alcohol =β·V alcohol

[0167] Where β is the sensor calibration coefficient, V alcohol The voltage value output by the sensor.

[0168] The YOLOv5 model fuses the detection results of YOLOv5 with the data of the sensor, and the system can make judgments based on the following logic:

[0169] Fatigue detection:

[0170] If Y fatigue =1, it is judged that the driver is in a fatigue driving state.

[0171] Smoking detection:

[0172] If Y smoking =1, and S smoke If the value is higher than the threshold, it is judged that the driver is smoking.

[0173] Risks of drinking:

[0174] If S alcohol >threshold, the system determines that the driver is at risk of drunk driving.

[0175] Comprehensive judgment formula:

[0176] Based on various inputs, the system's judgment formula is as follows:

[0177] S risk =γ·Y fatigue +δ·Y smoking +α·Vsmoke +β·V alcohol

[0178] in:

[0179] γ, δ: are the weights of fatigue driving and smoking behaviors, respectively.

[0180] α, β: calibration coefficients for smoke and alcohol concentrations, respectively.

[0181] According to this comprehensive risk value S risk , the system can make corresponding warnings and processing decisions. For example, if S risk If a certain threshold is exceeded, the system will sound an alarm to prompt the driver to stop driving.

[0182] In the present invention, through the comprehensive monitoring system based on the YOLOv5 target detection algorithm, it is possible to identify and judge in real time whether the driver is in an unsafe state such as drinking, fatigue driving or smoking. By integrating multiple sensor data (such as alcohol sensors, smoke sensors) and deep learning image recognition technology, this system can realize multi-dimensional state monitoring of the driver, and issue alarms in time to prevent potential risks. Thereby, driving safety can be significantly improved, especially in the monitoring of unsafe behaviors such as drinking, fatigue and smoking in complex driving environments. Experimental results show that the system can maintain a high response speed on the basis of high precision, and can adapt to a variety of driving environments. By integrating multiple sensors and deep learning algorithms, the system can realize comprehensive monitoring of driver behavior and reduce the occurrence of traffic accidents.

[0183] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0184] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A comprehensive detection system for driver drinking and fatigue driving based on yolov5, characterized by: The system includes: ESP32-CAM module, alcohol sensor and smoke sensor module, Arduino UNO development board module, WiFi module, YOLOv5 image recognition algorithm module and Flask server module; The ESP32-CAM module integrates camera and WiFi functions to capture the driver's facial image in real time and transmit it to the server via WiFi. The ESP32-CAM module strikes a good balance between performance and power consumption and is suitable for embedded systems. The alcohol sensor and smoke sensor modules are used to detect the alcohol and smoke concentrations in the car environment, and the data is collected and processed through the Arduino development board; The Arduino UNO development board module serves as the main control device for sensors and cameras, responsible for collecting, processing and transmitting sensor data; The WiFi module enables wireless transmission of data from Arduino UNO to the server, supporting real-time data transmission and feedback; The YOLOv5 image recognition algorithm module detects the driver's fatigue status and smoking status by training a specific data set, and has high real-time performance. This algorithm is one of the most advanced target detection algorithms currently, and can respond quickly while maintaining high-precision detection results. Flask server module: used to receive sensor and image data from Arduino, analyze the driver image based on the YOLOv5 model, and generate a comprehensive judgment result based on the sensor input data.

2. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: The detection process of the YOLOv5 image recognition algorithm module includes: S1. Input image preprocessing: The input image is processed into a size suitable for the network through resizing and normalization; this helps to improve the computational efficiency of the model; formula: Where x is the pixel value, x norm is the normalized pixel value; S2. Feature extraction: YOLOv5 uses the CSPNet structure for feature extraction and adopts residual blocks to extract features of different scales to maintain high efficiency; Feature mapping function: y=f(x)+x Among them, f(x) represents the nonlinear mapping in the network layer, and x is the input feature; S3. Object detection and classification: The output of YOLOv5 is the object bounding box and category prediction in each grid. It uses the Anchor mechanism and combines IoU as a measure of object matching. Coordinate formula for predicted bounding box: B x =σ(t x )+C x ,B y =σ(t y )+C y Among them, B x and B y is the predicted bounding box center coordinate, B w and B h is the predicted width and height, C x and C y is the grid center coordinate, p w and p h is the width and height of the prior box, σ is the Sigmoid function; S4. Loss function: The loss function of YOLOv5 consists of three parts: oBounding box regression loss: GIoU is used as the evaluation criterion for bounding box prediction; oClassification loss: based on binary cross entropy loss; oConfidence loss: used to measure whether there is a target in the prediction box, based on Focal Loss; S5. Loss function formula: S6. Reasoning and result output: Filter out the targets through the maximum confidence score, perform NMS processing, remove overlapping detection boxes, and finally output the detection results.

3. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: The data received by the Flask server module includes:

1. Smoke sensor data: used to detect whether there is smoking in the cab. The data is collected by MQ-2 or similar sensors; 2. Alcohol sensor data: collected by sensors such as MQ-3 to detect whether the driver has drunk alcohol; YOLOv5 image data analysis: Detects driver fatigue and smoking behavior; YOLOv5 analysis results are combined with sensor data to form a comprehensive driving safety judgment.

4. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: After receiving the sensor data, the Flask server uses the YOLOv5 model in the YOLOv5 image recognition algorithm module to analyze the driver image to determine whether there are unsafe behaviors such as yawning, closing eyes or smoking. The data analysis process of the YOLOv5 model is as follows:

1. Smoke and alcohol detection: The smoke detection formula is: Among them, R L is the load resistance, R0 is the sensor calibration value, V out is the sensor output voltage, V in is the input voltage; The alcohol test formula is: Among them, A BAC is the blood alcohol concentration, R alcohol is the resistance during detection, k is the calibration coefficient; 2. Comprehensive judgment logic: Based on the analysis results of sensors and YOLOv5, the Flask server combines multiple data to conduct a comprehensive analysis of driving behavior; the following is the judgment logic: a. When a high smoke value is detected and YOLOv5 recognizes a cigarette in the driver's facial area, it is inferred that the driver is smoking; b. When the alcohol sensor detects a high alcohol concentration and YOLOv5 does not detect any smoke, it is inferred that the driver is at risk of drunk driving; c. When YOLOv5 detects closing eyes or yawning, it infers that the driver may be in a state of fatigue driving; Comprehensive formula: Let the detected smoke value be V smoke , alcohol concentration is A BAC , the detection result of YOLOv5 is Y fatigue and Y smoking , the comprehensive judgment result is: S risk =αV smoke +βA BAC +γY fatigue +δY smoking Among them, α, β, γ, and δ are the weights of smoke, alcohol, fatigue, and smoking, respectively; The Arduino UNO development board module finally feeds back the above results to the user terminal through the Flask server, displaying the driver's behavior status and risk level in real time.

5. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: The YOLOv5 model is used to detect the driver's facial behavior and smoking behavior; the specific detection process is based on the target detection algorithm: YOLOv5 uses a deep convolutional neural network to detect objects in images and output bounding box coordinates and class probabilities; Loss function formula: Classification loss, used to determine the target category; Bounding box regression loss, used to predict the accuracy of the target box; Confidence loss, used to determine whether there is an object in the box; Through the features in the image data, YOLOv5 can recognize the driver's facial expressions, movements, and whether he is smoking. Combining these recognition results, the system can determine the driver's status in real time; Assume the result is: Y fatigue : When the driver is detected to be fatigued, it is set to 1, otherwise it is 0; Y smoking : When the driver's smoking behavior is detected, it is set to 1, otherwise it is 0.

6. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: The YOLOv5 model uses a smoke sensor to collect the smoke concentration in the air and return an analog voltage value, which is proportional to the smoke concentration. We use the voltage value returned by the sensor as the smoke concentration V smoke , processed by the following formula: S smoke =α·V smoke Where α is the sensor calibration coefficient, V smoke is the voltage value output by the sensor; Alcohol sensor detects alcohol concentration: Similarly, the alcohol sensor generates a voltage output according to the alcohol content in the air. Assume that its output value is V alcohol , the alcohol concentration can be converted by the following formula: S alcohol =β·V alcohol Where β is the sensor calibration coefficient, V alcohol The voltage value output by the sensor.

7. A comprehensive detection system for driver drinking and fatigue driving based on yolov5 as claimed in claim 1, characterized in that: The YOLOv5 model fuses the detection results of YOLOv5 with the data of the sensor, and the system can make judgments based on the following logic: Fatigue detection: If Y fatigue =1, the driver is judged to be in a fatigue driving state; Smoking detection: If Y smoking =1, and S smoke If it is higher than the threshold, it is judged that the driver is smoking; Risks of drinking: If S alcohol >threshold, the system determines that the driver is at risk of drunk driving; Comprehensive judgment formula: Based on various inputs, the system's judgment formula is as follows: S risk =γ·Y fatigue +δ·Y smoking +α·V smoke +β·V alcohol in: γ, δ: weights of fatigue driving and smoking behaviors, respectively; α, β: calibration coefficients for smoke and alcohol concentrations, respectively; According to this comprehensive risk value S risk , the system can make corresponding warnings and processing decisions; If S risk If a certain threshold is exceeded, the system will sound an alarm to prompt the driver to stop driving.

Images