System and method for detecting and quantifying driving behaviors

Abstract

The invention discloses a system and method for detecting and quantifying driving behaviors, the system comprises a system main body, the system main body comprises an automobile data recorder, an information capture module, a GPS, a speedometer, an accelerometer, an object detection module, an image classification module and a real-time deep learning module, the automobile data recorder is connected with the object detection module, and the image classification module is connected with the real-time deep learning module. The object detection module is connected with the image classification module, the image classification module is connected with the real-time deep learning module, and the real-time deep learning module is connected with the information capture module. According to the system and method for detecting and quantifying the driving behaviors, an instrument panel camera captures road videos, an information capture module collects GPS, speed and acceleration data, an object detection module recognizes and cuts images of traffic lights and road signs, an image classification module classifies the cut images, and a driving behavior detection module is used for detecting and quantifying the driving behaviors. The real-time deep learning module processes all the data to determine the type and severity of the driving behavior.

Description

Technical Field

[0001] The present invention relates to the technical field of real-time analysis of driving behavior using video capture, sensor data, object detection, image classification and deep learning, and in particular to a system and method for detecting and quantifying driving behavior. Background Art

[0002] The system for detecting and quantifying driving behavior is a driving intelligent assistance system. In recent years, the demand for advanced driver assistance systems (ADAS) and autonomous driving technology has increased significantly. These systems are designed to improve vehicle safety, improve driver performance, and reduce road traffic accidents; the integration of dashboard cameras (dash cams) has become increasingly popular because they can record video evidence of driving events. As technology continues to develop, people's requirements for systems for detecting and quantifying driving behavior are also increasing.

[0003] Existing systems for detecting and quantifying driving behavior have certain drawbacks when used. Traditional methods of monitoring driving behavior, such as manual observation and basic telemetry, lack the ability to provide comprehensive real-time insights into driver behavior and its impact. Therefore, a sophisticated system is needed that can use modern technology to accurately detect and quantify driving behavior in real time to promote safer driving habits and reduce the risks associated with dangerous driving behaviors. Traditional dashboard cameras are limited to passive recording and do not actively analyze driving behavior. On the other hand, information capture modules equipped with sensors such as GPS, speedometers, and accelerometers can provide valuable data about vehicle movement and speed. Although these systems are useful, they are often used in isolation and fail to correlate their data with visual input to generate meaningful insights about driving behavior; driving behavior is a key factor in road safety. Therefore, a system is needed that focuses primarily on providing alerts to drivers based on predefined conditions without fully leveraging the full potential of modern AI and sensor technologies. To this end, we propose a system and method for detecting and quantifying driving behavior. Summary of the invention

[0004] In view of the shortcomings of the prior art, the present invention provides a system and method for detecting and quantifying driving behavior. The dashboard camera captures road video, and the information capture module collects GPS, speed and acceleration data. The object detection module identifies and crops images of traffic lights and road signs, and the image classification module classifies these cropped images. The real-time deep learning module processes all data to determine the type and severity of driving behavior, which can effectively solve the problems in the background technology.

[0005] To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a system for detecting and quantifying driving behavior, including a system body, wherein the system body includes a driving recorder, an information capture module, a GPS, a speedometer, an accelerometer, an object detection module, an image classification module and a real-time deep learning module, wherein the driving recorder is connected to the object detection module, the object detection module is connected to the image classification module, the image classification module is connected to the real-time deep learning module, the real-time deep learning module is connected to the information capture module, the information capture module is connected to the GPS, the speedometer and the accelerometer.

[0006] As a preferred technical solution of the present application, the driving recorder is installed on the dashboard of the vehicle and is configured to capture road video when the driver is driving the vehicle, and the driving recorder captures internal and external views.

[0007] As a preferred technical solution of the present application, the information capture module is installed in the vehicle and is communicatively connected with the vehicle's GPS, speedometer and accelerometer. The information capture module is configured to collect one or more GPS, speed and acceleration data of the vehicle. The information capture module also collects the driver's physiological data. The information capture module wirelessly communicates with the central monitoring system.

[0008] As a preferred technical solution of the present application, the object detection module is communicatively connected to the driving recorder, and the object detection module is configured to identify and separate object images in the captured video to provide one or more cropped images of the object. The object detection module uses the YOLO-NAS computer vision technology developed by Deci AI for accurate identification, and the object detection module filters out irrelevant objects.

[0009] As a preferred technical solution of the present application, the image classification module is communicatively connected to the object detection module, and the image classification module is configured to receive the object cropped image provided by the object detection module, and classify the cropped image according to the object type to provide a classified image, and the image classification module uses the YOLOv8n-cls convolutional neural network developed by Ultralytics to perform object classification.

[0010] As a preferred technical solution of the present application, the real-time deep learning module is communicatively connected to the image classification module and the information capture module, and the real-time deep learning module is configured to determine the type and severity of driving behavior by processing real-time data of classified images, GPS, speed and vehicle acceleration data, and the real-time deep learning module uses a neural network for behavior analysis, and the real-time deep learning module adjusts the behavior analysis according to road conditions, and the real-time deep learning module provides feedback to the driver, and the real-time deep learning module evaluates aggressive driving, distraction, fatigue and compliance with traffic rules, and the real-time deep learning module adjusts the behavior analysis according to the driver's historical driving pattern.

[0011] As a preferred technical solution of the present application, the system body also includes a notification module, and the notification module issues an alarm to the driver according to the detected unsafe behavior, and a storage unit that stores historical behavior data for personalized analysis.

[0012] A method for detecting and quantifying driving behavior comprises the following steps: S1: Capturing one or more road videos during driving on a dashboard of a vehicle through a dashboard camera; S2: Capturing one or more GPS, speed and acceleration data of the vehicle from the GPS, speedometer and accelerometer of the vehicle through an information capture module; S3: Detecting one or more objects, such as traffic lights and road signs, from the captured video by an object detection module, and providing one or more cropped images of the objects; S4: classifying the cropped image according to the object type through an image classification module to provide a classified image; S5: Real-time data of classified images, GPS, speed and vehicle acceleration data are processed through a real-time deep learning module to determine the type and severity of the driver's driving behavior.

[0013] Compared with the prior art, the present invention provides a system and method for detecting and quantifying driving behavior, which has the following beneficial effects: the system and method for detecting and quantifying driving behavior, the dashboard camera captures road video, and the information capture module collects GPS, speed and acceleration data, the object detection module identifies and crops images of traffic lights and road signs, the image classification module classifies these cropped images, and the real-time deep learning module processes all data to determine the type and severity of driving behavior; the system includes a dashboard camera, an information capture module, a GPS, a speedometer, an accelerometer, an object detection module, an image classification module and a real-time deep learning module, the dashboard camera is installed on the dashboard of the vehicle, configured to capture road video when the driver is driving the vehicle, the dashboard camera captures internal and external views, the information capture module is installed in the vehicle and is connected to the vehicle's GPS, speedometer and accelerometer, and the information capture module is configured to collect one or more GPS, speed and acceleration data of the vehicle. The information capture module also collects driver physiological data (e.g., heart rate, eye movement). The information capture module communicates wirelessly with the central monitoring system, and the object detection module communicates with the dashboard camera. The object detection module is configured to identify and separate object images in the captured video, such as one or more traffic lights and road signs, to provide one or more cropped images of the object. The object detection module uses computer vision techniques such as YOLO-NAS (neural architecture search) developed by Deci AI for accurate recognition. The object detection module filters out irrelevant objects (e.g., pedestrians, animals), and the image classification module is in communication with the object detection module. The image classification module is configured to receive the object cropped images provided by the object detection module and classify the cropped images according to the object type to provide classified images. The image classification module uses convolutional neural networks (CNNs) such as YOLOv8n-cls developed by Ultralytics for object classification, and the real-time deep learning module is in communication with the image classification module and the information capture module. The real-time deep learning module is configured to determine the type and severity of driving behavior by processing real-time data of classified images, GPS, speed, and vehicle acceleration data. The real-time deep learning module uses neural networks for behavior analysis. The real-time deep learning module adjusts the behavior analysis based on road conditions (e.g., weather, traffic). The real-time deep learning module provides feedback to the driver (e.g., voice notification). The real-time deep learning module evaluates aggressive driving, distraction, fatigue, and compliance with traffic rules. The real-time deep learning module adjusts the behavior analysis based on the driver's historical driving patterns. The system also includes a notification module that alerts the driver based on the detected unsafe behavior, and a storage unit that stores historical behavior data for personalized analysis. The entire system for detecting and quantifying driving behavior has a simple structure, is easy to operate, and has better results than traditional methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram of a system for detecting and quantifying driving behavior according to the present invention.

[0015] Figure 2 The present invention is a flow chart of a method for detecting and quantifying driving behavior.

[0016] Figure 3 The present invention is a schematic diagram of a traffic signal violation tendency as a logic function in a system and method for detecting and quantifying driving behavior.

[0017] In the figure: 100, system body; 110, driving recorder; 120, information capture module; 122, GPS; 124, speedometer; 126, accelerometer; 130, object detection module; 140, image classification module; 150, real-time deep learning module. DETAILED DESCRIPTION

[0018] The technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific implementation methods.

[0019] like Figure 1-3 As shown, a system for detecting and quantifying driving behavior includes a system body 100, which includes a driving recorder 110, an information capture module 120, a GPS 122, a speedometer 124, an accelerometer 126, an object detection module 130, an image classification module 140 and a real-time deep learning module 150, wherein the driving recorder 110 is connected to the object detection module 130, the object detection module 130 is connected to the image classification module 140, the image classification module 140 is connected to the real-time deep learning module 150, the real-time deep learning module 150 is connected to the information capture module 120, the information capture module 120 is connected to the GPS 122, the speedometer 124 and the accelerometer 126, the dashboard camera captures road video, and the information capture module collects GPS, speed and acceleration data, the object detection module identifies and crops images of traffic lights and road signs, the image classification module classifies these cropped images, and the real-time deep learning module processes all data to determine the type and severity of driving behavior.

[0020] The driving recorder 110 is installed on a dashboard of a vehicle and is configured to capture a road video when a driver is driving the vehicle, and the driving recorder 110 captures interior and exterior views.

[0021] The information capture module 120 is installed in the vehicle and is in communication with the vehicle's GPS 122, speedometer 124 and accelerometer 126. The information capture module 120 is configured to collect one or more GPS, speed and acceleration data of the vehicle. The information capture module 120 also collects the driver's physiological data. The information capture module 120 wirelessly communicates with the central monitoring system.

[0022] The object detection module 130 is in communication with the driving recorder 110. The object detection module 130 is configured to identify and separate object images in the captured video to provide one or more cropped images of the object. The object detection module 130 uses the YOLO-NAS computer vision technology developed by Deci AI for accurate identification, and the object detection module 130 filters out irrelevant objects.

[0023] The image classification module 140 is communicatively connected to the object detection module 130. The image classification module 140 is configured to receive the object cropped image provided by the object detection module 130, and classify the cropped image according to the object type to provide a classified image. The image classification module 140 uses the YOLOv8n-cls convolutional neural network developed by Ultralytics to perform object classification.

[0024] The real-time deep learning module 150 is communicatively connected to the image classification module 140 and the information capture module 120. The real-time deep learning module 150 is configured to determine the type and severity of driving behavior by processing real-time data of classified images, GPS, speed and vehicle acceleration data. The real-time deep learning module 150 uses a neural network to perform behavior analysis, and the real-time deep learning module 150 adjusts the behavior analysis according to road conditions. The real-time deep learning module 150 provides feedback to the driver. The real-time deep learning module 150 evaluates aggressive driving, distraction, fatigue and compliance with traffic regulations, and the real-time deep learning module 150 adjusts the behavior analysis according to the driver's historical driving pattern.

[0025] The system body 100 also includes a notification module, and the notification module issues an alarm to the driver according to the detected unsafe behavior, and a storage unit that stores historical behavior data for personalized analysis.

[0026] A method for detecting and quantifying driving behavior comprises the following steps: S1: Capturing one or more road videos during driving on a dashboard of a vehicle through a dashboard camera; S2: Capturing one or more GPS, speed and acceleration data of the vehicle from the GPS, speedometer and accelerometer of the vehicle through an information capture module; S3: Detecting one or more objects, such as traffic lights and road signs, from the captured video by an object detection module, and providing one or more cropped images of the objects; S4: classifying the cropped image according to the object type through an image classification module to provide a classified image; S5: Real-time data of classified images, GPS, speed and vehicle acceleration data are processed through a real-time deep learning module to determine the type and severity of the driver's driving behavior.

[0027] The present invention aims to provide a comprehensive system that uses advanced computer vision and deep learning techniques to detect and quantify driving behavior.

[0028] In one aspect, Figure 1 As shown, the present invention provides a system 100 for detecting and quantifying the driving behavior of a driver when operating a vehicle. The system 100 includes a dashboard camera 110, an information capture module 120, a GPS 122, a speedometer 124, an accelerometer 126, an object detection module 130, an image classification module 140, and a real-time deep learning module 150.

[0029] The dashboard camera 110 is mounted on a dashboard of a vehicle and is configured to capture a road video while the driver is driving the vehicle. The dashboard camera 110 captures interior and exterior views.

[0030] The information capture module 120 is installed in the vehicle and is in communication with the vehicle's GPS 122, speedometer 124, and accelerometer 126. The information capture module 120 is configured to collect one or more GPS, speed, and acceleration data of the vehicle. The information capture module 120 also collects driver physiological data such as heart rate and eye movement. The information capture module 120 communicates wirelessly with the central monitoring system.

[0031] The object detection module 130 is in communication with the dashboard camera 110. The object detection module 130 is configured to identify and separate object images in the captured video, such as one or more traffic lights and road signs, to provide one or more cropped images of the object. The object detection module 130 uses computer vision techniques such as YOLO-NAS (Neural Architecture Search) developed by Deci AI for accurate recognition. The object detection module 130 filters out irrelevant objects (e.g., pedestrians, animals).

[0032] The image classification module 140 is in communication with the object detection module 130. The image classification module 140 is configured to receive the object cropped image provided by the object detection module 130, and classify the cropped image according to the object type to provide a classified image. The image classification module 140 uses convolutional neural networks (CNNs) such as YOLOv8n-cls developed by Ultralytics for object classification.

[0033] The real-time deep learning module 150 is in communication with the image classification module 140 and the information capture module 120. The real-time deep learning module 150 is configured to determine the type and severity of driving behavior by processing real-time data of classified images, GPS, speed, and vehicle acceleration data. The real-time deep learning module 150 uses a neural network to perform behavior analysis. The real-time deep learning module 150 adjusts the behavior analysis based on road conditions, such as weather, traffic. The real-time deep learning module 150 provides feedback to the driver, such as voice notifications. The real-time deep learning module 150 evaluates aggressive driving, distraction, fatigue, and compliance with traffic regulations. The real-time deep learning module 150 adjusts the behavior analysis based on the driver's historical driving pattern.

[0034] The system also includes a notification module that alerts the driver based on detected unsafe behavior and a storage unit that stores historical behavior data for personalized analysis.

[0035] In another aspect, Figure 2 As shown, the present invention provides a method for detecting and quantifying a driver's driving behavior when driving a vehicle. The method starts with the first step of capturing one or more road videos during driving on the dashboard of the vehicle through a dashboard camera 110. The second step of the method is to capture one or more GPS, speed and acceleration data of the vehicle from the GPS 122, speedometer 124 and accelerometer 126 of the vehicle through an information capture module 120. The third step of the method is to detect one or more objects, such as traffic lights and road signs, from the captured video through an object detection module 130, and provide one or more cropped images of the object. The fourth step of the method is to classify the cropped images according to the object type through an image classification module 140 to provide a classified image. The fifth step of the method is to process the real-time data of the classified images, GPS, speed and vehicle acceleration data through a real-time deep learning module 150 to determine the type and severity of the driver's driving behavior.

[0036] In an exemplary embodiment, after useful information is extracted from the information extraction module data and road object information, the data is processed by the real-time deep learning module to quantify driving behavior, including driving patterns at traffic lights and traffic signal violation tendencies. Driving patterns at traffic lights are quantified in two steps. The first step is to use the road object information and sensor statistics obtained from the signal processing step to determine whether the vehicle is approaching or crossing a traffic intersection or crosswalk. For example, if the vehicle speed is positive and the white line object in the image is approaching from the top to the bottom, the vehicle is crossing a traffic intersection. The second step is to use the characteristics and statistics of vehicle speed and acceleration to describe the driving behavior.

[0037] By utilizing the motion information of the traffic signal obtained in the signal processing step, we can detect whether a vehicle crosses the intersection when the light is yellow. Then, the acceleration data during the crossing process can be used to classify whether the vehicle accelerates when the light is yellow.

[0038] Using the road object information and the vehicle speed, we can determine the final speed of the vehicle when it approaches a red light traffic intersection. The tendency to violate a red light can be represented as a logistic function.

[0039] like Figure 3 As shown: p(v)=(1+e^(-k(v-v_0)) )^(-1) Where v is the vehicle speed, k and v_0 are the steepness and midpoint of the logistic function respectively. The intuition is that vehicles with speeds greater than v_0 are less likely to stop in time before the traffic intersection. To determine k and v_0, we can perform logistic regression on a set of sample data of red light tripping and normal driving. The values ​​of k and v_0 may be different for different types of vehicles. Datasets of different types of vehicles can be constructed to optimize these parameters.

[0040] The above shows and describes the basic principles and main features of the present invention and the advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A system for detecting and quantifying driving behavior, comprising a system body (100), characterized in that: The system body (100) comprises a driving recorder (110), an information capture module (120), a GPS (122), a speedometer (124), an accelerometer (126), an object detection module (130), an image classification module (140) and a real-time deep learning module (150), wherein the driving recorder (110) is connected to the object detection module (130), the object detection module (130) is connected to the image classification module (140), the image classification module (140) is connected to the real-time deep learning module (150), the real-time deep learning module (150) is connected to the information capture module (120), the information capture module (120) is connected to the GPS (122), the speedometer (124) and the accelerometer (126), the information capture module (120) is installed in a vehicle and is in communication connection with the GPS (122), the speedometer (124) and the accelerometer (126) of the vehicle, and the information capture module (120) is connected to the real-time deep learning module (150). The module (120) is configured to collect one or more GPS, speed and acceleration data of the vehicle, the information capture module (120) also collects driver physiological data, and the information capture module (120) wirelessly communicates with the central monitoring system; the real-time deep learning module (150) is communicatively connected to the image classification module (140) and the information capture module (120), the real-time deep learning module (150) is configured to determine the type and severity of driving behavior by processing real-time data of classified images, GPS, speed and vehicle acceleration data, the real-time deep learning module (150) uses a neural network to perform behavior analysis, and the real-time deep learning module (150) adjusts the behavior analysis according to road conditions, the real-time deep learning module (150) provides feedback to the driver, the real-time deep learning module (150) evaluates aggressive driving, distraction, fatigue and compliance with traffic rules, and the real-time deep learning module (150) adjusts the behavior analysis according to the driver's historical driving pattern.

2. A system for detecting and quantifying driving behavior according to claim 1, characterized in that: The driving recorder (110) is installed on a dashboard of a vehicle and is configured to capture road video when a driver is driving the vehicle, and the driving recorder (110) captures internal and external views.

3. A system for detecting and quantifying driving behavior according to claim 1, characterized in that: The object detection module (130) is in communication with the driving recorder (110), and the object detection module (130) is configured to identify and separate object images in the captured video to provide one or more cropped images of the object. The object detection module (130) uses the YOLO-NAS computer vision technology developed by Deci AI for accurate identification, and the object detection module (130) filters out irrelevant objects.

4. A system for detecting and quantifying driving behavior according to claim 1, characterized in that: The image classification module (140) is communicatively connected to the object detection module (130), and the image classification module (140) is configured to receive a cropped image of an object provided by the object detection module (130), and classify the cropped image according to the object type to provide a classified image, and the image classification module (140) uses a YOLOv8n-cls convolutional neural network developed by Ultralytics to perform object classification.

5. A system for detecting and quantifying driving behavior according to claim 1, characterized in that: A system for detecting and quantifying driving behavior according to claim 1, characterized in that: the system body (100) also includes a notification module, and the notification module issues an alarm to the driver according to the detected unsafe behavior, and a storage unit for storing historical behavior data for personalized analysis.

6. A method for detecting and quantifying driving behavior, characterized in that: The specific steps include the following: S1: Capturing one or more road videos during driving on a dashboard of a vehicle through a dashboard camera; S2: Capturing one or more GPS, speed and acceleration data of the vehicle from the GPS, speedometer and accelerometer of the vehicle through an information capture module; S3: Detecting one or more objects, such as traffic lights and road signs, from the captured video by an object detection module, and providing one or more cropped images of the objects; S4: classifying the cropped image according to the object type through an image classification module to provide a classified image; S5: Real-time data of classified images, GPS, speed and vehicle acceleration data are processed through a real-time deep learning module to determine the type and severity of the driver's driving behavior.

Images