Road condition early warning method and system based on vehicle event data recorder

Abstract

The application discloses a road condition early warning method and system based on a driving recorder, relates to the technical field of road condition early warning, and comprises the following steps: starting a vehicle, recording a driving road condition by using a driving recorder, collecting a vehicle driving speed, and detecting and identifying road condition obstacles from video data; calculating actual distances and speeds of the obstacles based on the identified road condition obstacles, predicting motion trajectories of the obstacles, and calculating a dynamic TTC by combining the predicted motion trajectories of the obstacles with the vehicle driving speed. The application can accurately determine future positions and relative trends of the obstacles by constructing a motion state vector of the obstacles and predicting the trajectories by combining an attention mechanism and an LSTM model, and can calculate a dynamic TTC value, so that early identification and graded early warning of potential collision risks are realized, and the accuracy of road condition early warning is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of road condition warning technology, in particular to a road condition warning method and system based on a driving recorder. BACKGROUND

[0002] With the continuous development of artificial intelligence, computer vision and Internet of Vehicles technology, the auxiliary driving system based on camera perception of the environment has gradually been popularized in medium and high-end vehicles. As a visual information acquisition device for vehicles, the driving recorder has been increasingly applied to intelligent driving assistance scenarios such as collision detection, accident analysis, path tracking, etc. in recent years due to its simple deployment, low cost, intuitive imaging and other characteristics. The traditional driving recorder is mainly used for passive recording, while the technology development in recent years is gradually upgrading it to an embedded intelligent device with partial active perception and judgment capabilities. Related technical research attempts to realize the functions of vehicle environment perception and potential risk warning by fusing target detection models (such as YOLO), path planning, sensor data processing and other algorithms without increasing the cost of additional hardware, and to empower existing vehicles with higher intelligent capabilities.

[0003] However, in the existing camera-based video analysis technology, there are still the following obvious deficiencies. First, most of the methods are based on fixed cameras or ADAS special modules, rather than using driving recorders, which have poor universality and adaptability, and most of the methods only stop at image-level target detection (i.e. identifying obstacles), without further modeling the motion state of the target, predicting its trajectory, or combining the motion parameters of the vehicle itself for dynamic prediction. Second, there is a lack of unified time sequence modeling and behavior prediction mechanism, which fails to realize continuous modeling of the future trend of obstacles, resulting in delayed response and insufficient reliability of the warning. In addition, the warning strategy is rough, usually based on simple threshold judgment, lacks time prediction indicators such as TTC (Time to Collision) as a basis for judgment, and cannot distinguish between high-risk and low-risk situations. Therefore, the existing road condition warning technology generally has the problems of single warning judgment dimension, insufficient risk assessment accuracy, poor real-time and accuracy of warning. SUMMARY

[0004] In view of the above existing problems, the present application is proposed.

[0005] Therefore, the present application provides a road condition warning method and system based on a driving recorder, which solves the problems of single warning judgment dimension, insufficient risk assessment accuracy, poor real-time and accuracy of warning.

[0006] To solve the above technical problems, the present application provides the following technical solutions:

[0007] In a first aspect, the present application provides a road condition warning method based on a driving recorder, which comprises,

[0008] The vehicle is started to record the driving road conditions by the driving recorder, collect the vehicle driving speed, and detect and identify the road obstacles from the video data;

[0009] The actual distance and speed of the obstacle are calculated based on the identified road obstacles, the motion trajectory of the obstacle is predicted, and the dynamic TTC is calculated by combining the predicted motion trajectory of the obstacle with the vehicle driving speed;

[0010] The warning analysis is carried out based on the dynamic TTC, the road warning is carried out according to the warning analysis result, and the warning event data is saved and synchronized to the cloud.

[0011] As a preferred scheme of the road warning method based on the driving recorder, wherein: the motion trajectory of the obstacle is predicted, the dynamic TTC is calculated by combining the predicted motion trajectory of the obstacle with the vehicle driving speed, which is based on the actual distance between the obstacle and the vehicle at the current time , the speed of the obstacle and the acceleration of the obstacle A three-tuple is constructed , that is, the motion state vector of the obstacle, and the motion state vectors of the obstacle in the historical time are collected by a sliding window to form an input sequence Z, the motion state vector of each time in the input sequence Z is input into an attention calculation unit, the attention score of each time step is calculated , and the attention scores of all time steps are normalized by using a Softmax function to obtain normalized attention weights The original input sequence Z is weighted to obtain a weighted input sequence Input into the LSTM model to obtain the position of the obstacle at the next time , and all trainable parameters in the LSTM network are flattened into a high-dimensional vector, the particle swarm is initialized, the LSTM parameters corresponding to all particles are substituted into the LSTM model for prediction, and the fitness function of each particle is calculated.

[0012] The fitness functions of all particles are sorted in descending order, and the particle with the smallest error in the particle swarm is selected as the global optimal solution of the current round , the particle position and speed are iteratively updated according to the APSO optimization algorithm, the position and speed of all particles are updated after each iteration, the global optimal solution is re-evaluated according to the fitness function, and the maximum iteration number is set When the continuous iteration round and the change amplitude of the fitness function converges, the iteration is stopped, and the optimal global optimal solution , that is, the optimal LSTM model parameter, is output, and the LSTM model is updated by the global optimal solution to predict the position of the obstacle , predict the obstacle position based on the updated LSTM model input into the updated LSTM model for prediction, predict the obstacle position at the next time, and repeat the prediction to obtain the future trajectory of the obstacle through the future trajectory of the obstacle the future speed of the obstacle is calculated between the two adjacent predicted positions , and the relative speed is calculated in combination with the vehicle speed v ;

[0013] When the relative speed is greater than 0, the vehicle is approaching the obstacle, and the TTC is calculated to determine the risk of collision, otherwise, it is normal, and the collision time TTC is calculated based on the relative speed and the predicted distance.

[0014] As a preferred scheme of the road condition warning method based on the driving recorder, wherein: the warning analysis is performed based on the dynamic TTC, and the road condition warning is performed according to the warning analysis result; according to the TTC calculation result, a threshold value is set through statistical analysis 、 , the threshold value is greater than the threshold value , when the TTC is less than the threshold value , the vehicle will collide with the obstacle, triggering a red warning, when the TTC is greater than or equal to the threshold value , less than the threshold value , a yellow warning will be triggered, when the TTC is greater than or equal to the threshold value , no warning is triggered, according to the risk level determined, the corresponding warning triggering mechanism is executed, when the red warning is triggered, the front obstacle is highlighted with a red frame on the vehicle display screen, and the voice prompt "collision risk ahead, please slow down!" is played, when the yellow warning is triggered, a yellow warning box is displayed on the vehicle display screen, reminding the driver to pay attention to the front obstacle and take measures to avoid collision.

[0015] As a preferred scheme of the road condition warning method based on the driving recorder, wherein: the warning event data is saved and synchronized to the cloud; when the red warning and the yellow warning are triggered, the vehicle appears a steering operation and an emergency braking behavior, the longitudinal acceleration is monitored in real time by the acceleration sensor, when the acceleration meets the condition, the emergency recording function is immediately started, the video before and after the emergency braking is automatically recorded, all emergency recording data are automatically set to "non-overwrite state", after the emergency recording is completed, the video file, the event image snapshot, the event timestamp, and the TTC data of this event are packaged and uploaded to the cloud server through the Wi-Fi network connected by the vehicle communication module.

[0016] As a preferred embodiment of the road condition warning method based on a dashcam described in this invention, the step of calculating the actual distance and speed of road obstacles based on obstacle identification refers to processing the camera focal length and the image height of the obstacle using camera calibration, calculating the actual distance D between the obstacle and the vehicle using a monocular visual ranging method, and calculating the speed of the obstacle through multiple consecutive frames of images. The change in obstacle velocity is calculated by subtracting the velocities of obstacles in adjacent frames. Based on the change in obstacle velocity Calculate the acceleration of the obstacle .

[0017] As a preferred embodiment of the road condition warning method based on a dashcam described in this invention, the step of detecting and identifying road obstacles in the recorded video data refers to inputting the real-time video stream image into a YOLOv5s model for obstacle target detection. The YOLOv5s model processes each frame of the image, generates a bounding box for each detected obstacle, outputs the bounding boxes of candidate obstacle targets, and applies a regression loss function. Regression optimization is performed on the center point coordinates and width and height parameters of the bounding box. The obstacle bounding box is obtained through the optimized parameters. Combined with the NMS algorithm, a cross-union threshold is set. When the intersection-union ratio (IU) of bounding box A and bounding box B is greater than the IU threshold... If the confidence score of bounding box A is greater than that of bounding box B, then bounding box B is removed. This process is repeated until all bounding boxes have been processed. For the processed bounding boxes, the obstacle category distribution is calculated using the softmax function, and the category classification is optimized using cross-entropy loss.

[0018] As a preferred embodiment of the road condition warning method based on a dashcam described in this invention, the vehicle starts recording the road conditions via the dashcam. The vehicle speed is collected when the vehicle's ACC is activated. The dashcam automatically starts recording the road conditions in real time and connects to the vehicle system or mobile APP via WIFI. The vehicle's actual speed is obtained via the CAN bus.

[0019] Secondly, the present invention provides a road condition warning system based on a driving recorder, comprising,

[0020] The road condition image acquisition module is used to automatically start the dashcam when the vehicle's auxiliary power ACC is turned on, record video in real time and collect vehicle speed data;

[0021] An obstacle recognition module is configured to use a YOLOv5s model to perform real-time detection on the video recording and recognize obstacles encountered during driving;

[0022] An obstacle distance and speed calculation module is configured to obtain the focal length of the camera and image data of the obstacle through camera calibration, calculate the actual distance between the obstacle and the vehicle in combination with the image height and the actual height, and calculate the speed and acceleration of the obstacle;

[0023] A dynamic TTC calculation module is configured to construct a motion state vector according to the distance, speed and acceleration of the obstacle, collect historical motion state vector data, predict the future motion trajectory of the obstacle through LSTM and attention mechanism, and calculate the TTC value.

[0024] A warning and data uploading module is configured to determine whether to trigger a warning according to the TTC calculation result, remind the driver through a display screen and voice according to the warning level, and upload data.

[0025] In a third aspect, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, any step of the road condition warning method based on a driving recorder according to the first aspect of the present application is implemented.

[0026] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, any step of the road condition warning method based on a driving recorder according to the first aspect of the present application is implemented.

[0027] The present application has the following advantages: the present application fuses vehicle speed information, image recognition, obstacle motion state analysis and dynamic TTC calculation to construct a multi-dimensional road condition warning mechanism, overcomes the problem of single judgment basis in the prior art, uses the YOLOv5s model to realize real-time video target detection, combines softmax classification and cross-entropy optimization to improve recognition accuracy, uses LSTM combined with attention mechanism to predict the trajectory of the obstacle, calculates the dynamic TTC value, grades the warning according to the collision risk, improves the risk assessment accuracy and the real-time warning, judges the sudden braking behavior combined with the acceleration threshold, realizes automatic saving and uploading of the key event video to the cloud, and enhances the accident tracing capability. BRIEF DESCRIPTION OF DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0029] Fig. 1 Flow chart of the road condition warning method based on the car DVR in embodiment 1.

[0030] Fig. 2 Structure diagram of the road condition warning system based on the car DVR in embodiment 1.

[0031] Fig. 3 Flow chart of the obstacle detection in embodiment 1. DETAILED DESCRIPTION

[0032] In order to make the above objectives, characteristics and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0033] Embodiment 1, refer to Figs. 1-3 , the first embodiment of the present application, the embodiment provides a road condition warning method based on the car DVR, comprising the following steps:

[0034] S1, starting the vehicle through the car DVR to record the driving road condition, collecting the vehicle driving speed, and detecting and identifying the road condition obstacles from the recorded video data;

[0035] Specifically, starting the vehicle through the car DVR to record the driving road condition, collecting the vehicle driving speed, that is, after the vehicle is connected to the ACC (auxiliary power supply), the car DVR is automatically started to begin real-time recording of the vehicle driving road condition, and is connected with the vehicle-mounted system or the mobile phone APP through WIFI (2.4 GHz) to ensure data synchronization and remote control, and the actual speed of the vehicle driving is obtained through the CAN bus (serial communication protocol), the vehicle speed is converted into meters per second (m / s) for subsequent path calculation and obstacle prediction, and the conversion formula is:

[0036]

[0037] Among them, is the real-time speed of the vehicle, is the speed of the vehicle.

[0038] The automatic starting of the driving recorder after the vehicle is connected to the ACC realizes the zero delay response and full recording basis of the system, ensures that the early warning system is activated synchronously with the vehicle, improves the integrity and continuity of road condition perception through real-time video recording of the driving road condition, avoids the risk of misrecognition caused by "frame interruption" or "image missing", expands the application boundary of the system through WIFI (2.4 GHz) connection with the vehicle-mounted system or APP, improves the operability and interactivity of the system, obtains the vehicle speed through the CAN bus and converts it into m / s, ensures the data consistency and precision of the whole motion state modeling and collision time prediction process, and effectively improves the response ability and judgment accuracy of the early warning system to dynamic risks.

[0039] Further, the video data is detected and identified as a road obstacle. The video stream image collected in real time is input into the YOLOv5s model for obstacle target detection. The YOLOv5s model processes each frame of image and generates a bounding box for each detected obstacle, outputs the bounding box of the candidate obstacle target, and frames the rectangular box of the target object. The bounding box is defined by the upper left corner , the lower right corner .

[0040]

[0041] wherein, is the bounding box coordinate, and the coordinate represents the position of the obstacle in the image;

[0042] To improve the positioning accuracy of the bounding box, mean square error (MSE) is used as the regression loss function during model training, and the center point coordinates and width and height parameters of the bounding box are optimized by the regression loss function . The optimized parameters are used to obtain the bounding box of the obstacle. To avoid the problem of multiple overlapping bounding boxes pointing to the same target, the NMS (non-maximum suppression) algorithm is combined to retain the bounding box with the highest confidence (the confidence of the existence of the object in the bounding box) as the final detection result of the target obstacle. The selection of the bounding box is based on the overlap degree between the bounding boxes, i.e. the intersection over union. The intersection over union threshold is set through cross-validation . When the intersection over union of the bounding box A and the bounding box B is greater than the intersection over union threshold , and the confidence score of the bounding box A is greater than that of the bounding box B, the bounding box B is removed to prevent repeated detection. The operation is repeated until all bounding boxes are processed. For the processed bounding boxes, the softmax function is used to calculate the obstacle class distribution (pedestrian, vehicle), and the cross-entropy loss is used to optimize the class classification, which measures the difference between the predicted class distribution of the model and the true class label.

[0043] The robustness and real-time performance of obstacle detection under different road conditions and light conditions are improved by inputting the real-time collected video stream image into the YOLOv5s model for target detection, providing a data basis for subsequent early warning, realizing the structured representation of the spatial position of the obstacle by generating a boundary box for each detected obstacle and outputting the position coordinates thereof, enhancing the data analyzability, significantly enhancing the accuracy and stability of obstacle positioning by introducing a mean square error (MSE) regression loss function in model training to optimize the boundary box prediction parameters, improving the recognition reliability of the system in complex scenes, reducing the early warning interference caused by false positives / misses, effectively suppressing redundant overlapping boxes by combining the non-maximum suppression (NMS) algorithm to filter candidate boundary boxes, and outputting a unique credible target box, and realizing high-credibility recognition and label refinement of the obstacle category (pedestrian, vehicle, etc.) by performing softmax classification and cross-entropy loss optimization on the finally retained boundary box.

[0044] S2, calculate the actual distance and speed of the obstacle based on the identified road obstacle, and predict the motion trajectory of the obstacle, calculate the dynamic TTC by combining the predicted motion trajectory of the obstacle with the vehicle driving speed;

[0045] Specifically, calculating the actual distance and speed of the obstacle based on the identified road obstacle refers to using camera calibration (a process used in computer vision and image processing to determine the internal parameters of the camera, which can convert pixel values into actual physical units) to process the focal length of the camera and the image height of the obstacle, and using monocular vision ranging method to calculate the actual distance between the obstacle and the vehicle at the current time by combining the processed focal length of the camera, the image height of the obstacle and the actual height of the obstacle:

[0046]

[0047] wherein, is the actual distance between the obstacle and the vehicle at the current time, f is the processed focal length of the camera, l is the image height of the obstacle processed by camera calibration, and H is the actual height of the obstacle;

[0048] The speed of the obstacle is calculated through consecutive multiple frames of images:

[0049]

[0050] wherein, is the speed of the obstacle, is the frame interval time, is the change of the position of the obstacle in adjacent frames;

[0051] The speed change of the obstacle is calculated by subtracting the speed of the obstacle in the adjacent two frames , based on the obstacle speed change amount The acceleration of the obstacle is calculated to describe the change in motion of the obstacle:

[0052]

[0053] wherein, is the acceleration of the obstacle.

[0054] Through the camera calibration step, the mapping of image pixels to actual physical units is achieved, and the basic spatial model for distance measurement is established. By combining the processed focal length, image height, and real height of the obstacle, the actual distance between the obstacle and the vehicle is calculated, realizing contactless monocular ranging. The change in the position of the obstacle is calculated through consecutive image frames, realizing the estimation of the relative speed of the obstacle, providing input for dynamic behavior modeling. By analyzing the speed change, the acceleration of the obstacle is further calculated, realizing sensitive recognition of the change in motion trend.

[0055] Further, the dynamic TTC is calculated based on the actual distance between the obstacle and the vehicle at the current time , the speed of the obstacle and the acceleration of the obstacle by predicting the motion trajectory of the obstacle and combining the vehicle driving speed. That is, the motion state vector of the obstacle, in order to support the dynamic perception and prediction of the subsequent model, the motion state vectors of the obstacle in the historical time are collected through a sliding window to form an input sequence wherein, is the motion state vector of the obstacle at time t. The motion state vector at each time in the input sequence Z is input into the attention calculation unit to calculate the attention score at each time step To ensure the comparability of the attention score, the Softmax function is used to normalize the attention scores of all time steps to obtain the normalized attention weight The normalized attention weight is used to weight process the original input sequence to obtain a new input sequence The new input sequence after weighting is input into the LSTM model in sequence to update the hidden state, and the hidden state at the last time step of the sequence is selected The next time obstacle position is output through the fully connected layer. In order to improve the ability of the LSTM model to handle complex problems, all trainable parameters in the LSTM network are flattened into a high-dimensional vector for unified optimization, and a particle swarm is initialized, with each particle representing a set of LSTM parameter vectors to obtain an initial particle swarm set ,in, Let be the LSTM parameters corresponding to the i-th particle. To evaluate the performance of the LSTM model for each particle, the LSTM parameters corresponding to all particles are substituted into the LSTM model for prediction, and the fitness function for each particle is calculated. The fitness function is defined as the root mean square error (RMSE) on the validation set. ;

[0056] Sort all particles by their fitness functions in descending order, and select the particle with the smallest error (i.e., the smallest fitness) as the global optimum for the current round. Based on the APSO optimization algorithm, the particle position and velocity are iteratively updated:

[0057]

[0058]

[0059] in, It is the updated particle velocity. This is the updated particle position. It is the inertial weight, which is set through experiments. It is the particle's current velocity. It is a random number. It is the best-performing global optimal solution in the current particle swarm. , These are the perturbation factor and the guiding factor, which are set through an adaptive mechanism;

[0060] After each iteration, the positions and velocities of all particles are updated, the global optimum is re-evaluated based on the fitness function, and the maximum number of iterations is set using fuzzy logic. When continuous iteration If the variation of the fitness function converges, then stop iterating and output the optimal global solution. That is, the optimal LSTM model parameters, obtained through the global optimal solution. Update the LSTM model to predict obstacle locations Predicting obstacle locations based on an LSTM model with updated parameters The LSTM model, after being updated with new parameters, makes predictions to determine the obstacle's position at the next time step. These predictions are repeated to obtain the obstacle's future trajectory. ,in, It is the future trajectory of the obstacle. It predicts the obstacle's position at time t+J;

[0061] Future movement trajectory through obstacles Calculate the future velocity of the obstacle between two adjacent predicted positions:

[0062]

[0063] wherein, is the future speed of the obstacle at t+1 time, is the sampling time interval;

[0064] based on the future speed of the obstacle at t+1 time combined with the vehicle speed v to calculate the relative speed for judging whether the vehicle and the obstacle have a close trend, when the relative speed is greater than 0, the vehicle is approaching the obstacle, and there is a collision risk to calculate TTC, otherwise, it is normal, based on the relative speed and the predicted distance to calculate the collision time (TTC):

[0065]

[0066]

[0067] wherein, TTC is the collision time, is the actual distance between the obstacle and the vehicle at the current time, is the predicted distance.

[0068] By constructing an obstacle motion state triple including actual distance, speed and acceleration, multi-dimensional dynamic behavior modeling is realized, providing comprehensive input for trajectory prediction, avoiding early warning lag caused by static identification, calculating the weight of historical state sequence through attention mechanism, strengthening the model's ability to capture complex behavior patterns, improving trajectory prediction accuracy, reducing false prediction caused by invalid data interference, realizing time series context modeling by inputting weighted sequence into LSTM network and continuously updating hidden state, enhancing the model's memory and reasoning ability, providing stable input for TTC calculation and early warning, and enhancing the stability and adaptability of the model under different road conditions and obstacle behavior patterns through APSO optimization algorithm for global search optimization of LSTM parameters, ensuring that trajectory prediction still has high reliability in complex environments, improving the prediction granularity and continuity of the motion trend of the obstacle by continuously calling the optimized LSTM model for position iterative prediction, calculating the relative speed between the future speed and the vehicle speed, and calculating the TTC value combined with the predicted distance of the obstacle, a more accurate and dynamic early warning mechanism is constructed, which can significantly reduce the false positive rate and false negative rate compared with the traditional fixed distance-based judgment method, and improve the reliability and practicality of road condition warning.

[0069] S3, based on dynamic TTC, road condition warning is performed according to the warning analysis result, and the warning event data is saved and synchronized to the cloud.

[0070] Specifically, the pre-warning analysis is performed based on the dynamic TTC, and the road condition pre-warning is performed according to the pre-warning analysis result. According to the calculation result of TTC (time to collision), a threshold value is set through statistical analysis 、 , the threshold value is greater than the threshold value , when the TTC is less than the threshold value , the vehicle will collide with the obstacle, triggering a red pre-warning, when the TTC is greater than or equal to the threshold value , less than the threshold value , then a yellow pre-warning will be triggered, when the TTC is greater than or equal to the threshold value , then no pre-warning is triggered, according to the risk level determined, the corresponding pre-warning triggering mechanism is executed, when the red pre-warning is triggered, the front obstacle is highlighted with a red frame on the vehicle display screen, helping the driver to quickly identify the collision risk area, and playing the voice prompt "collision risk ahead, please slow down!", to ensure that the driver can hear the warning in time, when the yellow pre-warning is triggered, a yellow warning box is displayed on the vehicle display screen, reminding the driver to pay attention to the front obstacle and take measures to avoid collision.

[0071] Through the threshold value judgment mechanism based on TTC (time to collision), a more progressive and refined risk assessment mechanism is realized, the response ability of the pre-warning system to different severity risk scenarios is improved, the practicality and fault tolerance in actual driving are improved, through the image visual enhancement way of marking the obstacle in the red pre-warning state, the intuitive perception and rapid response of the driver to the high-risk target are realized, through the low-interference way of reminding the driver to pay attention to the front target in the yellow pre-warning state, the timely reminder of the moderate risk is realized instead of intervention.

[0072] Further, the pre-warning event data is saved and synchronized to the cloud. When the red pre-warning and the yellow pre-warning are triggered, the vehicle appears a steering operation and an emergency braking behavior, the longitudinal acceleration (i.e. the acceleration in the front-rear direction of the vehicle) is monitored in real time by the acceleration sensor, when the acceleration satisfies:

[0073]

[0074] wherein, is the longitudinal acceleration, is a threshold value set by a typical emergency braking behavior of the vehicle, to distinguish normal braking from emergency braking;

[0075] When the condition is met, the emergency recording function is immediately started, and the video before and after the emergency braking is automatically recorded. To ensure that the key data before and after the accident is not lost or covered, all emergency recording data is automatically set to "non-overwrite state". After completing the emergency recording, the video file of this event, event image snapshot, event timestamp, TTC data are packaged and uploaded to the cloud server through the Wi-Fi network connected by the vehicle communication module.

[0076] By combining vehicle operation behavior (such as turning and emergency braking) to trigger acceleration monitoring in the red-yellow warning state, accurate identification of key driving dynamics is achieved, improving the accuracy of identifying high-risk events. Ensure that data recording and processing only occur when there is a real safety hazard. Enhance the intelligence and credibility of the warning system. Through the emergency braking acceleration threshold judgment mechanism, the distinction between normal braking and dangerous braking is achieved. A scientific trigger criterion is established. By automatically recording the video content before and after the emergency braking, the time closed-loop recording of the entire event process is achieved. By setting the emergency recording data to "non-overwrite state", it is ensured that the key data is not overwritten by new data, and the priority data saving mechanism is achieved. Through the Wi-Fi network, the event-related data is packaged and uploaded to the cloud, realizing remote accident synchronization and background recording management.

[0077] The embodiment also provides a road condition warning system based on a driving recorder, which comprises:

[0078] A road condition image acquisition module is configured to automatically start the driving recorder by connecting the auxiliary power supply ACC of the vehicle, record in real time, and acquire vehicle speed data.

[0079] An obstacle recognition module is configured to use a YOLOv5s model to detect the recorded video in real time and identify obstacles encountered during driving.

[0080] An obstacle distance and speed calculation module is configured to obtain the focal length of the camera and the image data of the obstacle by camera calibration, calculate the actual distance between the obstacle and the vehicle in combination with the image height and the actual height, and calculate the speed and acceleration of the obstacle.

[0081] A dynamic TTC calculation module is configured to construct a motion state vector according to the distance, speed and acceleration of the obstacle, collect historical motion state vector data, predict the future motion trajectory of the obstacle through LSTM and attention mechanism, and calculate the TTC value.

[0082] A warning and data uploading module is configured to determine whether to trigger a warning according to the TTC calculation result, remind the driver through the display screen and voice according to the warning level, and perform data uploading.

[0083] The embodiment also provides a computer device suitable for the driving recorders based road condition early warning method, including a memory and a processor; the memory is used for storing computer executable instructions, and the processor is used for executing the computer executable instructions to realize the driving recorders based road condition early warning method provided in the above embodiment.

[0084] The computer device can be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. The processor of the computer device is used for providing computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for wired or wireless communication with external terminals. The wireless communication can be realized through WIFI, an operator network, NFC (Near Field Communication) or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0085] The embodiment also provides a storage medium having a computer program stored thereon, the program being executed by a processor to realize the driving recorders based road condition early warning method provided in the above embodiment. The storage medium can be realized by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.

Claims

1. A method for road condition warning based on a driving recorder, characterized in that: The application relates to a road condition warning method based on a vehicle-mounted DVR (digital video recorder). The DVR is started to record a driving road condition, vehicle speed is collected, and road condition obstacles are detected and recognized from video data. The road condition obstacles include pedestrians and vehicles. Actual distances and speeds of the obstacles are calculated based on the recognized road condition obstacles, and obstacle motion trajectories are predicted. A dynamic TTC (time to collision) is calculated based on the obstacle motion trajectories and the vehicle speed. The TTC refers to a collision time. The predicting the motion trajectory of the obstacle is based on the actual distance between the obstacle and the vehicle at the current time , the speed of the obstacle , and the acceleration of the obstacle Constructing a triple , i.e. the motion state vector of the obstacle, and constructing an input sequence Z by collecting the motion state vectors of the obstacle in the historical time through a sliding window, inputting the motion state vector of each time in the input sequence Z into an attention calculation unit, calculating the attention score of each time step , and normalizing the attention scores of all time steps by using a Softmax function to obtain normalized attention weights The original input sequence Z is weighted to obtain a weighted input sequence Input into the LSTM model to obtain the position of the obstacle at the next time , and all trainable parameters in the LSTM network are flattened into a high-dimensional vector, the particle swarm is initialized, the LSTM parameters corresponding to all particles are substituted into the LSTM model for prediction, and the fitness function of each particle is calculated; Sort the fitness function of all particles in descending order, select the particle with the smallest error in the particle swarm as the global optimal solution of the current round , according to the iterative update of particle position and speed of APSO optimization algorithm, after each iteration, update the position and speed of all particles, and reevaluate the global optimal solution according to the fitness function, and set the maximum iteration number , when continuous iteration Round and the change range of fitness function converges, stop iteration and output the optimal global optimal solution , that is, the optimal LSTM model parameter, update the LSTM model to predict the position of the obstacle , based on the updated parameter of the LSTM model, predict the position of the obstacle , based on the updated parameter of the LSTM model, predict the position of the obstacle , input into the updated parameter of the LSTM model for prediction, predict the position of the next moment of the obstacle, and repeat the prediction to obtain the future motion trajectory of the obstacle . 2.The dashcam-based road condition warning method of claim 1, wherein: The dynamic TTC passing through the future trajectory of the obstacle is calculated by combining the predicted trajectory of the obstacle with the speed of the vehicle The future speed of the obstacle is calculated in the adjacent two predicted positions in the future trajectory of the obstacle The relative speed is calculated in combination with the speed of the vehicle v ; When the relative speed is greater than 0, then the vehicle is approaching the obstacle, there is a risk of collision, the TTC is calculated, otherwise, normal, the collision time TTC is calculated based on the relative speed and the predicted distance. 3.The dashcam-based road condition warning method of claim 2, wherein: The pre-warning analysis is based on the dynamic TTC, and the road condition pre-warning is performed according to the pre-warning analysis result , , the threshold is greater than the threshold , when the TTC is less than the threshold , the imminent collision between the vehicle and the obstacle is triggered, and the red pre-warning is triggered, when the TTC is greater than or equal to the threshold , less than the threshold , the yellow pre-warning is triggered, when the TTC is greater than or equal to the threshold , no pre-warning is triggered, according to the risk level determined, the corresponding pre-warning triggering mechanism is executed, when the red pre-warning is triggered, the front obstacle is highlighted with a red frame on the vehicle display screen, and the voice prompt "collision risk ahead, please slow down!" is played, when the yellow pre-warning is triggered, the yellow warning frame is displayed on the vehicle display screen, reminding the driver to pay attention to the front obstacle and take measures to avoid collision. 4.The dashcam-based road condition warning method of claim 3, wherein: Warning analysis is performed based on the dynamic TTC, road condition warnings are given according to the warning analysis results, and warning event data are saved and synchronously uploaded to a cloud. 5.The dashcam-based road condition warning method of claim 4, wherein: The calculating actual distance and speed of the obstacle based on identifying road obstacles comprises: processing camera focal length and image height of the obstacle by camera calibration; calculating actual distance D between the obstacle and the vehicle by monocular vision ranging method; calculating speed of the obstacle through continuous multiple images , calculating speed change amount of the obstacle by subtracting speed of the adjacent two frames of the obstacle , calculating acceleration of the obstacle based on the speed change amount of the obstacle .​ 6.The dashcam-based road condition warning method of claim 5, wherein: The detecting and identifying road obstacles from the video data is to input the video stream image collected by real-time video recording into a YOLOv5s model for obstacle target detection, the YOLOv5s model processes each frame of image, generates a bounding box for each detected obstacle, outputs the bounding box of the candidate obstacle target, and through a regression loss function The center point coordinates and width and height parameters of the bounding box are optimized by regression, the obstacle bounding box is obtained through the optimized parameters, the NMS algorithm is combined, and an intersection-over-union threshold is set When the intersection-over-union of the bounding box A and the bounding box B is greater than the intersection-over-union threshold , and the confidence score of the bounding box A is greater than the bounding box B, then the bounding box B is removed, and the operation is repeated until all the bounding boxes are processed, the obstacle class distribution is calculated through a softmax function for the processed bounding boxes, and the class classification is optimized through a cross-entropy loss. 7.The dashcam-based road condition warning method of claim 6, wherein: When a red warning or a yellow warning is triggered, a vehicle steering operation or an emergency braking behavior occurs, longitudinal acceleration is monitored in real time by an acceleration sensor, when the acceleration meets a condition, an emergency recording function is immediately started, video before and after the emergency braking is automatically recorded, all emergency recording data are automatically set as "non-overwrite state", and after the emergency recording is completed, video files, event image snapshots, event time stamps and TTC data of the event are packaged and uploaded to the cloud server through a Wi-Fi network connected by a vehicle-mounted communication module.

8. A road condition warning system based on a car video recorder, the road condition warning method based on the car video recorder according to any one of claims 1 to 7, characterized in that: When the vehicle is connected to the ACC, the DVR is automatically started to record the driving road condition in real time, and is connected to a vehicle-mounted system or a mobile phone APP through WIFI, and the actual speed of the vehicle is acquired through a CAN bus. The application relates to a road condition warning method based on a vehicle-mounted DVR (digital video recorder). A road condition image acquisition module is used to automatically start the DVR through an auxiliary power supply ACC of a vehicle, record in real time and acquire vehicle speed data. An obstacle recognition module is used to use a YOLOv5s model to detect a video in real time and recognize obstacles encountered in a driving process. An obstacle distance and speed calculation module is used to acquire camera focal length and image data of the obstacles through camera calibration, calculate actual distances between the obstacles and the vehicle in combination with image height and actual height, and calculate speeds and accelerations of the obstacles. A dynamic TTC calculation module is used to acquire historical motion state vector data according to distances, speeds and accelerations of the obstacles, predict future motion trajectories of the obstacles through LSTM and an attention mechanism, and calculate TTC values. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is characterized in that: A warning and data uploading module is used to judge whether a warning is triggered according to the TTC calculation results, remind a driver through a display screen and voice according to a warning level, and perform data uploading.

10. A computer readable storage medium having stored thereon a computer program, characterized in that: The processor executes the computer program to realize the steps of the road condition warning method based on the DVR. The computer program is executed by the processor to realize the steps of the road condition warning method based on the DVR.

Images