Driver detection method for cab

Abstract

The invention provides a driver detection method for a cab, and belongs to the technical field of driver detection, and the method comprises the steps: obtaining the state data of a driver and the change of the environment in a vehicle in real time through arranging a camera, and carrying out the night detection through arranging an infrared camera at a main driver position in the vehicle; the camera and the infrared camera are used for collecting data of the state of a driver in the vehicle and the surrounding environment of the vehicle content, and the data are analyzed and processed to remind the driver of driving safety after being collected. Facial expressions, limb conditions and the like of the driver are considered, the environment in the vehicle is monitored, the infrared sensor and the camera are combined, the environment in the vehicle is better monitored when the vehicle is driven at night, the driver is better reminded to concentrate on driving in time, and the driving safety is improved. Data collection is carried out for drivers in the vehicle, the environment in the vehicle including passengers and the like, the algorithm model can update and perfect the model based on the collected data set, and the real-time effect is better.

Description

Technical Field

[0001] The present invention relates to the technical field of driver detection, and in particular to a driver detection method for a cab. Background Art

[0002] As the amount of technology incorporated into vehicles increases, the amount of information available to the driver and passengers of the vehicle also increases. This information can become distracting to the driver while the vehicle is moving, especially when there is a certain level of desired interaction. As a result, select interactive features within the vehicle may be disabled or limited while the vehicle is in motion.

[0003] The Chinese patent with the announcement number CN 110348350 A discloses a driver state detection method based on facial expression. Through graying, Gamma correction and PCA dimensionality reduction processing, the size of the facial image is reduced and the features are enhanced. On this basis, the present invention constructs a facial expression recognition convolution neural network consisting of a four-layer structure, an average pooling layer, a dropout layer and a softmax classifier connected in sequence, which has a small number of parameters. At the same time, the inception structure design adopts a new form, which splits the traditional regular convolution 3*3 convolution into 1*3 convolution and 3*1 convolution, which saves a lot of parameters, accelerates the calculation and reduces overfitting, and adds a layer of nonlinear expansion model expression ability, which can process more and richer spatial features and increase feature diversity. This inception structure design makes the facial expression recognition convolution neural network more lightweight, and has a better detection effect, that is, it improves the accuracy of driver state detection. The patent has the following defects: 1. Only the driver's facial expression detection is considered, which has certain limitations. 2. Recognize facial expressions based on neural networks. This process takes a long time and cannot be detected and responded to in real time, and has a certain delay.

[0004] The Chinese patent with the announcement number CN 116363636 A discloses a driver state detection method, device, equipment and storage medium. The method comprises: obtaining an image to be detected; inputting the image to be detected into a pre-trained driver state detection model, and determining the driver state detection result corresponding to the image to be detected according to the output generation result; wherein the driver state detection model at least includes a feature extraction module, a multi-task information detection module, an occlusion detection module, a fatigue detection module and a distraction detection module. The technical solution of the embodiment of the present invention avoids the image corresponding to the failure scene from entering the subsequent calculation process, reduces the amount of data calculation for the failure scene, and avoids the output of the detection result corresponding to the failure scene, improves the accuracy of the driver state detection result determined according to the generation result, and improves the robustness of the overall algorithm. The patent has the following defects: 1. Only the driver state expression detection is considered, which has certain limitations. 2. The training model based on feature detection cannot adapt to environmental changes and data updates, cannot update the data model and response in real time, and has a certain delay.

[0005] The above two methods have the following technical disadvantages: first, only the driver's facial image is considered to judge the driver's status, which has certain limitations. Second, the neural network model and feature detection training model cannot adapt to environmental changes, cannot detect data and update the model in real time, and have a certain delay. Therefore, it is necessary to design a comprehensive driver detection method for the cab. Summary of the invention

[0006] The purpose of the present invention is to provide a driver detection method for the cab, which solves the technical problems that the existing detection method only considers the driver's facial image to judge the driver's state, has certain limitations, and the neural network model and feature detection training model cannot adapt to environmental changes, cannot detect data and update the model in real time, and has a certain delay. The driver's behavior and the in-vehicle environment in the cab are monitored to achieve more stable and accurate detection results.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A driver detection method for the cab, which acquires the driver's status data and changes in the in-car environment in real time by setting up a camera, and sets an infrared camera at the main driving position in the car for night detection. The camera and the infrared camera collect data on the driver's status in the car and the surrounding environment of the car. After the data is collected, it is analyzed and processed to remind the driver of driving safety.

[0009] Furthermore, the in-car camera and infrared sensor are combined for all-day monitoring, especially when driving at night, to perceive the driver's facial expressions and body behavior data, and obtain data on the in-car environment, including the dynamics of the front passenger and the rear passengers, and transmit the data to the on-board system. Then, based on the yolov5 algorithm, the in-car environment of the entire vehicle is modeled, trained, analyzed and predicted. The driver's status and any behavior that may affect the driver's driving are analyzed based on the transmitted real-time data. It is determined whether the driver is playing with a mobile phone, driving fatigued, or whether the children in the car are interfering with driving. If so, the assisted driving system will use voice to remind the driver to concentrate on driving.

[0010] 3. The method for detecting a driver in a cab according to claim 1 is characterized in that the specific process of collecting data on the driver's state in the vehicle and the surrounding environment of the vehicle is as follows:

[0011] (1) Real-time image acquisition: The driver’s facial and eye images are acquired in real time through an optical camera installed inside the car.

[0012] (2) Infrared radiation detection: The infrared sensor detects and receives infrared radiation emitted by objects in the vehicle to perceive the vehicle environment. Temperature difference recognition: Due to the temperature difference between different objects, the infrared sensor accurately detects the target object. Environmental status judgment: Based on the detected infrared radiation information, the temperature and humidity status of the vehicle environment are judged, as well as whether there is an abnormal overheating or overcooling.

[0013] (3) The information obtained by the camera and infrared sensor is integrated to obtain more comprehensive and accurate information about the driver's status and the vehicle's interior environment, and the collected data is transmitted to the intelligent driving system.

[0014] Furthermore, the YOLOv5 algorithm model building process is:

[0015] (1) Model design: YOLOv5 is used as the basic model, and its components include input, back-bone, neck and output. Input: YOLOv5 performs data enhancement on the image at the input. To improve the detection effect, the original data is spliced ​​with the enhanced image and then passed into the network for convolution operation. Secondly, YOLOv5 uses adaptive anchor frame calculation and shrinks and enlarges the adaptive image to obtain a suitable anchor frame. Back-bone: Slice the image to obtain a downsampled feature map without information loss, which optimizes the problem of complex model calculation. By reducing the number of parameters, the detection speed of the model is improved, and spatial feature information of different sizes is extracted to achieve feature fusion, enriching the expression ability of the feature map. It can detect multiple targets, which is conducive to improving the robustness and detection accuracy of the model. Neck: It realizes the transmission and fusion of high-level feature information, so as to transmit the obtained more effective information to the prediction layer. Output: The prediction results with higher probability are retained through non-maximum suppression, which solves the problem of screening multiple target frames and achieves target frame deduplication.

[0016] (2) Model training: The feature vectors formed by different behavior data are numbered to form a model set of different in-vehicle behaviors, and the corresponding three types of in-vehicle behavior models are obtained. According to the characteristics and requirements of the data set, the parameters of YOLOv5 are configured, including the input size, in-vehicle model, in-vehicle structure size, and number of training rounds. The YOLOv5 model is trained using the labeled data set. After continuous training, the model achieves the set effect on the validation set.

[0017] (3) Model evaluation and optimization: Use the test set collected in the early stage to evaluate the performance of the model, including indicators such as accuracy, precision, recall, F1 score, ROC curve and AUC value. Accuracy: Accuracy = (TP + TN) / (TP + FP + TN + FN), the ratio of the number of correctly classified samples to the total number of samples. Precision: Precision = TP / (TP + FP), which measures the proportion of samples that are actually positive among the samples predicted by the model as positive samples. Recall: Recall = TP / (TP + FN), which measures the ability of the model to find all positive samples. F1 Score: F1 Score = 2 * (Precision * Recall) / (Precision + Recall). ROC curve and AUC value are AUC-ROC: which evaluates the effect of the classification model. The larger the AUC value, the better the model performance. Optimize the model according to the evaluation results. Optimization includes adjusting model parameters, using data enhancement technology, and trying different model architectures.

[0018] (4) Real-time analysis and prediction: The real-time video stream inside the car is obtained through the camera. The information provided by the infrared sensor is used to supplement the camera data. Especially when the lighting conditions are poor or the driver wears glasses, the real-time video stream is preprocessed as necessary. The preprocessing includes adjusting the resolution and color space conversion. The trained YOLOv5 model is used to detect the target of the preprocessed video frame, identify the key objects of the driver's face, hand movements, mobile phones, and children, and finally determine whether the vehicle content behavior meets the expected three model standards by comparing with the predetermined standards. The YOLOv5 model continuously trains and optimizes itself: the adaptive enhancer of the CPA-Enhancer chain thinking network is introduced to improve the object detection performance, especially in complex environments; the collected data is divided into training sets in real time, and then the training process is started using the divided training sets through the background configuration script. During the training process, the model will continuously adjust the parameters through the back-propagation algorithm to minimize the loss function.

[0019] Furthermore, combined with different scene intentions, three different models are designed for driving behavior in the vehicle, including: the driver's eyes looking straight ahead model, the driver's hands on the steering wheel model, and the front and rear passengers interfering with the driver's behavior model. Video data containing various in-car scenes and driver behaviors are collected. The video data includes data on normal driving, fatigue driving, playing with mobile phones, and interference from children in the car. The labeling tool LabelImg is used to annotate key objects in the video frames, including the driver's face, mobile phone, and children. The annotations will be used to train the YOLOv5 model. Different car contents will form different model data, which will form different feature vectors after quantization encoding. By distinguishing the feature vectors, the distinction of in-car behaviors can be finally obtained.

[0020] Furthermore, the assisted driving system reminds the driver to concentrate on driving:

[0021] The analysis results calculated in real time by the algorithm model are sent to the on-board system. The on-board system plays a voice prompt to the driver based on the received signal instructions, which includes the current situation, such as the driver being distracted and not looking forward or playing with the mobile phone, and reminding the driver to concentrate on driving to avoid traffic accidents.

[0022] The present invention has the following beneficial effects due to the adoption of the above technical solution:

[0023] The present invention not only takes into account the driver's facial expressions, the driver's limbs, etc., but also includes monitoring of the in-car environment. By using a combination of infrared sensors + cameras, better in-car environment monitoring is achieved when driving at night, and the driver can be reminded to concentrate on driving in a timely manner. Data is collected on the in-car driver, in-car environment including passengers, etc. The algorithm model can be updated and improved based on the collected data set, and the real-time effect is better.

[0024] The present invention not only detects the in-car environment from the driver's perspective, but also monitors the in-car environment in all directions, which may affect driving. The addition of infrared sensors can better adapt to nighttime driving and instantly remind the driver of the safety of nighttime driving. The yolov5 algorithm has high precision, faster detection speed, and better scalability. Combined with the background control script, it can analyze the collected data in real time and optimize the model, and monitor and remind the driver to concentrate on driving in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and preferred embodiments. However, it should be noted that many details listed in the specification are only for the purpose of enabling the reader to have a thorough understanding of one or more aspects of the present invention, and these aspects of the present invention can be implemented even without these specific details.

[0027] like Figure 1 As shown, a driver detection method for the cab includes a camera installed in the cab, which can obtain the driver's status data and changes in the interior environment of the car in real time; the infrared camera in the main driving position in the car can perform night detection, and combined with the camera, it can realize the data collection of the driver's status in the car and the surrounding environment of the car when driving at night, and the data is analyzed and processed after collection to remind the driver to drive safely.

[0028] The method of monitoring the driver's status based on the driver's behavior and the in-car environment is:

[0029] First of all, the combination of in-car cameras and infrared sensors can sense the driver's facial expressions, body behaviors and other data throughout the day, especially when driving at night, and obtain data on the in-car environment, including but not limited to the dynamics of the front passenger and the rear passengers, and transmit the data to the on-board system. Then, based on the yolov5 algorithm, the in-car environment of the entire vehicle is modeled, trained, analyzed and predicted. According to the real-time data transmitted, the driver's status and any behavior that may affect the driver's driving are analyzed, and it is determined whether the driver is playing with a mobile phone, driving while fatigued, or there are children in the car that interfere with driving. Finally, the assisted driving system will use voice to remind the driver to concentrate on driving.

[0030] The combination of in-car cameras and infrared sensors can sense the driver's facial expressions, body behaviors and other data throughout the day, especially when driving at night, and obtain data on the in-car environment, including but not limited to the dynamics of the front passenger and rear passengers, and transmit the data to the intelligent driving system.

[0031] The process of obtaining the driver's status and in-vehicle environment includes:

[0032] (1) Real-time image acquisition: The driver’s facial and eye images are acquired in real time through an optical camera installed inside the car.

[0033] (2) Infrared radiation detection: Infrared sensors are used to detect and receive infrared radiation emitted by objects inside the vehicle, thereby realizing perception of the vehicle's interior environment. Temperature difference recognition: Due to the temperature difference between different objects, infrared sensors can accurately detect target objects, such as passengers and pets. Environmental status judgment: Based on the detected infrared radiation information, the temperature, humidity and other conditions of the vehicle's interior environment are judged, as well as whether there are any abnormal conditions, such as overheating or overcooling.

[0034] (3) The information obtained by the camera and infrared sensor is integrated to obtain more comprehensive and accurate information about the driver's status and the vehicle's interior environment, and the collected data is transmitted to the intelligent driving system.

[0035] Based on the yolov5 algorithm, the vehicle's interior environment is modeled, trained, analyzed and predicted. The driver's status and any behaviors that may affect the driver's driving are analyzed based on the real-time data transmitted, and it is determined whether the driver is playing with a mobile phone, driving while fatigued, or there are children in the car and other behaviors that interfere with driving.

[0036] The process of building a model through the YOLOv5 algorithm includes:

[0037] (1) Model design: YOLOv5 is a basic model, and its main components include input, back-bone, neck and output. Input: YOLOv5 performs data enhancement on the image at the input. To improve the detection effect, the original data and the enhanced image are spliced ​​and then passed into the network for convolution operation. Secondly, YOLOv5 uses adaptive anchor frame calculation and shrinks and enlarges the adaptive image to obtain a suitable anchor frame. Back-bone: Slice the image to obtain a downsampled feature map without information loss, optimize the problem of complex model calculation, and to a certain extent improve the detection speed of the model by reducing the number of parameters, extract spatial feature information of different sizes, realize feature fusion, and greatly enrich the expression ability of the feature map. It is very helpful for multi-target detection and is conducive to improving the robustness and detection accuracy of the model. Neck: Realize the transmission and fusion of high-level feature information, so as to transmit the obtained more effective information to the prediction layer. Output: The prediction results with higher probability are retained by non-maximum suppression, which solves the problem of multi-target box screening and realizes target box deduplication.

[0038] Combined with different scene intentions, three different models are designed for different driving behaviors in the vehicle. The three models are: the driver's eyes looking straight ahead model, the driver's hands on the steering wheel model, and the front and rear passengers' interference with the driver's behavior model. Collect video data containing various in-car scenes and driver behaviors. These data should include various situations such as normal driving, fatigue driving, playing with mobile phones, and interference from children in the car. Use the labeling tool LabelImg to annotate key objects in the video frame, including the driver's face, mobile phone, children, etc. These annotations will be used to train the YOLOv5 model. Different car content will form different models.

[0039] The data, after being quantized and encoded, will form different feature vectors. By distinguishing the feature vectors, we can finally distinguish the behaviors inside the car.

[0040] (2) Model training: The feature vectors formed by different behavioral data are numbered to form a model set of different in-car behaviors, and the corresponding three types of in-car behavior models are obtained (model of the driver looking straight ahead, model of whether the driver holds the steering wheel with both hands, and model of front and rear passengers interfering with the driver's behavior). According to the characteristics and requirements of the dataset, configure the parameters of YOLOv5, such as input size, in-car model, in-car structure size, number of training rounds, etc. Use the labeled dataset to train the YOLOv5 model, and after continuous training, until the model achieves satisfactory results on the validation set.

[0041] (3) Model evaluation and optimization: Use the test set collected in the early stage to evaluate the performance of the model, including indicators such as accuracy, precision, recall, F1 score, ROC curve and AUC value. Accuracy: Accuracy = (TP + TN) / (TP + FP + TN + FN), the ratio of the number of correctly classified samples to the total number of samples. Precision: Precision = TP / (TP + FP), measures the proportion of samples that are actually positive among the instances predicted by the model as positive samples. Recall: Recall = TP / (TP + FN), measures the ability of the model to find all positive samples. F1 score (F1 Score): F1 Score = 2 * (Precision * Recall) / (Precision + Recall), which takes into account both precision and recall, and is a commonly used comprehensive performance indicator. ROC curve and AUC value (AUC-ROC): Evaluate the effect of the classification model. The larger the AUC value, the better the model performance. Optimize the model based on the evaluation results, such as adjusting model parameters, using data enhancement techniques, trying different model architectures, etc.

[0042] (4) Real-time analysis and prediction: The real-time video stream inside the car is obtained through the camera. The information provided by the infrared sensor can be used as a supplement to the camera data, especially when the lighting conditions are poor or the driver wears glasses. The real-time video stream is preprocessed as necessary, such as adjusting the resolution and color space conversion. The trained YOLOv5 model is used to detect the target of the preprocessed video frame and identify the key objects such as the driver's face, hand movements, mobile phones, and children. By comparing with the predetermined standards, it is finally determined whether the vehicle content behavior meets the expected three model standards. In addition, the YOLOv5 model can continuously train and optimize itself: introduce adaptive enhancers such as CPA-Enhancer chain thinking network to improve object detection performance, especially in complex environments; divide the collected data into training sets in real time, and then use the divided training sets to start the training process through the background configuration script. During the training process, the model will continuously adjust the parameters through the back-propagation algorithm to minimize the loss function.

[0043] The process of the assisted driving system reminding the driver to concentrate on driving includes: sending a signal to the vehicle system based on the analysis results calculated in real time by the algorithm model. The vehicle system plays a voice prompt to the driver's current driving situation based on the received signal instructions. The prompt information includes the current situation, such as the driver is distracted and not looking forward or playing with the phone, and reminding the driver to concentrate on driving to avoid traffic accidents. Not only the driver's facial expressions and the driver's body conditions are taken into consideration, but also the monitoring of the in-car environment is included. By using a combination of infrared sensors + cameras, better in-car environment monitoring is achieved when driving at night, and the driver is reminded to concentrate on driving in a better and more timely manner. Data collection is carried out on the in-car driver, in-car environment including passengers, etc. The algorithm model can be updated and improved based on the collected data set, and the real-time effect is better.

[0044] The yolov5 algorithm has high precision, faster detection speed, and better scalability. Combined with the background control script, it can perform real-time analysis and model optimization on the collected data, and monitor and remind the driver to concentrate on driving in real time.

[0045] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A driver detection method for a cab, characterized in that: By setting up cameras, the driver's status data and changes in the interior environment of the car can be obtained in real time. An infrared camera is set up at the main driving seat in the car for night detection. The data on the driver's status in the car and the surrounding environment of the car are collected through cameras and infrared cameras. After data collection, it is analyzed and processed to remind the driver to drive safely.

2. A cab-oriented driver detection method according to claim 1, characterized in that: The in-car camera and infrared sensor are combined for all-day monitoring, especially when driving at night, to perceive the driver's facial expressions and body behavior data, and obtain data on the in-car environment, including the dynamics of the front passenger and the rear passengers, and transmit the data to the on-board system. Then, based on the yolov5 algorithm, the in-car environment of the entire vehicle is modeled, trained, analyzed and predicted. The driver's status and any behavior that may affect the driver's driving are analyzed based on the real-time data transmitted, and it is determined whether the driver is playing with a mobile phone, driving fatigued, or whether the children in the car are interfering with driving. If so, the assisted driving system will use voice to remind the driver to concentrate on driving.

3. The method for detecting a driver in a cab according to claim 1, characterized in that: The specific process of data collection on the driver status in the car and the surrounding environment of the car is as follows: (1) Real-time image acquisition: The driver’s facial and eye images are acquired in real time through an optical camera installed inside the car. (2) Infrared radiation detection: The infrared sensor detects and receives infrared radiation emitted by objects in the vehicle to perceive the vehicle environment. Temperature difference recognition: Due to the temperature difference between different objects, the infrared sensor accurately detects the target object. Environmental status judgment: Based on the detected infrared radiation information, the temperature and humidity status of the vehicle environment are judged, as well as whether there is an abnormal overheating or overcooling. (3) The information obtained by the camera and infrared sensor is integrated to obtain more comprehensive and accurate information about the driver's status and the vehicle's interior environment, and the collected data is transmitted to the intelligent driving system.

4. The method for detecting a driver in a cab according to claim 2, characterized in that: The YOLOv5 algorithm model building process is as follows: (1) Model design: YOLOv5 is used as the basic model, and its components include input, back-bone, neck and output. Input: YOLOv5 performs data enhancement on the image at the input. To improve the detection effect, the original data is spliced ​​with the enhanced image and then passed into the network for convolution operation. Secondly, YOLOv5 uses adaptive anchor frame calculation and shrinks and enlarges the adaptive image to obtain a suitable anchor frame. Back-bone: Slice the image to obtain a downsampled feature map without information loss, which optimizes the problem of complex model calculation. By reducing the number of parameters, the detection speed of the model is improved, and spatial feature information of different sizes is extracted to achieve feature fusion, enriching the expression ability of the feature map. It can detect multiple targets, which is conducive to improving the robustness and detection accuracy of the model. Neck: It realizes the transmission and fusion of high-level feature information, so as to transmit the obtained more effective information to the prediction layer. Output: The prediction results with higher probability are retained through non-maximum suppression, which solves the problem of screening multiple target frames and achieves target frame deduplication. (2) Model training: The feature vectors formed by different behavior data are numbered to form a model set of different in-vehicle behaviors, and the corresponding three types of in-vehicle behavior models are obtained. According to the characteristics and requirements of the data set, the parameters of YOLOv5 are configured, including the input size, in-vehicle model, in-vehicle structure size, and number of training rounds. The YOLOv5 model is trained using the labeled data set. After continuous training, the model achieves the set effect on the validation set. (3) Model evaluation and optimization: Use the test set collected in the early stage to evaluate the performance of the model, including indicators such as accuracy, precision, recall, F1 score, ROC curve and AUC value. Accuracy: Accuracy = (TP + TN) / (TP + FP + TN + FN), the ratio of the number of correctly classified samples to the total number of samples. Precision: Precision = TP / (TP + FP), which measures the proportion of samples that are actually positive among the samples predicted by the model as positive samples. Recall: Recall = TP / (TP + FN), which measures the ability of the model to find all positive samples. F1 Score: F1 Score = 2 * (Precision * Recall) / (Precision + Recall). ROC curve and AUC value are AUC-ROC: which evaluates the effect of the classification model. The larger the AUC value, the better the model performance. Optimize the model according to the evaluation results. Optimization includes adjusting model parameters, using data enhancement technology, and trying different model architectures. (4) Real-time analysis and prediction: The real-time video stream inside the car is obtained through the camera. The information provided by the infrared sensor is used to supplement the camera data. Especially when the lighting conditions are poor or the driver wears glasses, the real-time video stream is preprocessed as necessary. The preprocessing includes adjusting the resolution and color space conversion. The trained YOLOv5 model is used to detect the target of the preprocessed video frame, identify the key objects of the driver's face, hand movements, mobile phones, and children, and finally determine whether the vehicle content behavior meets the expected three model standards by comparing with the predetermined standards. The YOLOv5 model continuously trains and optimizes itself: the adaptive enhancer of the CPA-Enhancer chain thinking network is introduced to improve the object detection performance, especially in complex environments; the collected data is divided into training sets in real time, and then the training process is started using the divided training sets through the background configuration script. During the training process, the model will continuously adjust the parameters through the back-propagation algorithm to minimize the loss function.

5. The method for detecting a driver in a cab according to claim 4, characterized in that: Combining different scene intentions, three different models are designed for driving behavior in the vehicle, including: the driver's eyes looking straight ahead model, the driver's hands on the steering wheel model, and the front and rear passengers interfering with the driver's behavior model. Video data containing various in-car scenes and driver behaviors are collected. The video data includes data on normal driving, fatigue driving, playing with mobile phones, and interference from children in the car. The labeling tool LabelImg is used to annotate key objects in the video frames, including the driver's face, mobile phone, and children. The annotations will be used to train the YOLOv5 model. Different car contents will form different model data, which will form different feature vectors after quantization encoding. By distinguishing the feature vectors, the distinction of in-car behaviors can be finally obtained.

6. The method for detecting a driver in a cab according to claim 2, characterized in that: The process of the assisted driving system reminding the driver to concentrate on driving: The analysis results calculated in real time by the algorithm model are sent to the on-board system. The on-board system plays a voice prompt to the driver based on the received signal instructions, which includes the current situation, such as the driver being distracted and not looking forward or playing with the mobile phone, and reminding the driver to concentrate on driving to avoid traffic accidents.

Images