Vehicle collision risk early warning method, system, equipment and medium

Abstract

The invention provides a vehicle collision risk early warning method, system and device and a medium, and belongs to the technical field of vehicle safety control. The method comprises the steps that surrounding environment information of a vehicle is collected in real time through a vehicle built-in sensor and a GPS module; a multi-sensor fusion device technology is used to carry out comprehensive processing on surrounding environment information of a vehicle, and an environment perception model is generated after data integration; based on the environment perception model, target identification is carried out by using a computer vision technology based on a convolutional neural network so as to identify travelers, vehicles, traffic signs and obstacles; performing behavior prediction based on a deep learning model to predict a behavior track of a front vehicle or pedestrian; evaluating a potential collision risk in the current scene in combination with the target recognition result and the behavior prediction result; when a potential collision risk is detected, an early warning signal is sent to a driver, and when the driver does not take corresponding measures, active safety measures are automatically triggered to avoid collision or reduce accident damage.

Description

Technical Field

[0001] The present invention belongs to the technical field of vehicle safety control, and more specifically relates to a vehicle collision risk warning method, system, equipment and medium. Background Art

[0002] With the continuous development of the global economy and the acceleration of urbanization, the number of vehicles has shown an explosive growth trend. While this trend brings traffic convenience, it has also caused increasingly serious traffic safety problems, especially the frequent occurrence of traffic accidents, which has become an important issue that needs to be solved urgently worldwide. Traffic accidents will not only cause huge economic losses, but also may cause serious casualties, posing a serious threat to social stability and harmony.

[0003] Traditional vehicle collision avoidance systems mainly rely on the driver's reaction speed and simple sensor alarm mechanisms installed on the vehicle. These systems usually detect the distance and speed of obstacles in front or adjacent vehicles through sensors such as radar and cameras. Once a potential collision risk is detected, an alarm is issued to the driver through sound, light, etc., reminding him to take braking or evasive measures. However, such systems have exposed a series of limitations in practical applications: 1. Reliance on driver response: The effectiveness of traditional collision avoidance systems is highly dependent on the driver’s reaction speed, judgment and driving experience. In an emergency, the driver may fail to respond correctly in time due to panic, fatigue or misjudgment, resulting in the collision avoidance system failing to play its intended role.

[0004] 2. Insufficient prediction capabilities: Due to the limitations of sensor performance and data processing technology, traditional systems can only detect obstacles that have already appeared in real time, but cannot conduct in-depth analysis and prediction of complex traffic environments. This means that the system cannot identify potential collision risks in advance, such as sudden braking or lane change intentions of vehicles ahead, thus missing the best time to take preventive measures.

[0005] 3. Insufficient sensor fusion: Traditional systems usually use only a single or a few types of sensors, resulting in a single information acquisition channel and a low degree of data fusion. This not only limits the system's ability to fully perceive the environment, but may also lead to false alarms and missed alarms, reducing the reliability and accuracy of the collision avoidance system.

[0006] 4. Lack of intelligent decision-making capabilities: When faced with complex and ever-changing traffic scenarios, traditional systems lack sufficient intelligence and flexibility to formulate optimal collision avoidance strategies. They can often only provide simple warning information, but cannot take proactive preventive measures based on the current situation, such as automatic deceleration, emergency braking, or path planning. Summary of the invention

[0007] In response to the above problems, the purpose of the present invention is to provide a vehicle collision risk warning method, system, equipment and medium. By comprehensively using artificial intelligence, deep learning and sensor fusion technology, deep perception, accurate prediction and intelligent decision-making of the traffic environment are achieved, overcoming the defects of traditional vehicle collision avoidance systems that rely on driver reaction, insufficient prediction ability, insufficient sensor fusion and lack of intelligent decision-making ability, and significantly improving the safety and reliability of vehicle driving.

[0008] In order to achieve the above object, the present invention is implemented through the following technical solutions: In a first aspect, an embodiment of the present application provides a vehicle collision risk warning method, comprising: The vehicle's built-in sensors and GPS module collect the vehicle's surrounding environment information in real time; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environment data, real-time traffic flow and V2X data; Use multi-sensor fusion technology to comprehensively process the vehicle's surrounding environment information and generate a unified environmental perception model after data integration; Based on the environmental perception model, computer vision technology based on convolutional neural networks is used for target recognition to identify pedestrians, vehicles, traffic signs and obstacles; behavior prediction is performed based on deep learning models to predict the possible behavior trajectory of the vehicle or pedestrian in front; the target recognition results and behavior prediction results are combined to evaluate the potential collision risk in the current scene; When a potential collision risk is detected, a warning signal is sent to the driver, and if the driver does not take corresponding measures, active safety measures are automatically triggered to avoid collision or reduce accident damage.

[0009] In an optional embodiment, the real-time collection of the vehicle's surrounding environment information through the vehicle's built-in sensors and GPS module includes: Collect road images, obstacle detection data, object depth data, and lane line data as environmental perception data through cameras, millimeter-wave radars, and lidars; The vehicle speed, acceleration, steering wheel angle, and vehicle posture data are collected as vehicle motion state data through the GPS model, vehicle-mounted speed sensor, steering wheel angle sensor, and inertial measurement unit; The distance to the vehicle in front, the side distance, and the distance to pedestrians are collected through cameras, millimeter-wave radars, lidars, and ultrasonic sensors as relative position and distance data; Traffic sign information, traffic light status information and road status information are collected as traffic signal and road information data through cameras and road status sensors; Collect light intensity, temperature, humidity and rainfall information as weather and environmental data through cameras, light sensors, and temperature and humidity sensors; The acceleration, speed and direction change information of the vehicle in front is collected through V2V communication, and the dynamic information of pedestrians and vehicles is collected through V2I as real-time traffic flow and V2X data.

[0010] In an optional implementation, the method of using multi-sensor fusion technology to comprehensively process the vehicle's surrounding environment information and generate a unified environment perception model after data integration includes: Perform filtering and data synchronization on the vehicle's surrounding environment information; The Kalman filter algorithm is used to fuse the relative position and distance data; The EKF algorithm is used to fuse the vehicle motion state data; The particle filter algorithm is used to track the movement trajectory of pedestrians or vehicles based on the surrounding environment information of the vehicle; Convolutional neural network is used to extract features from road images and fuse them with relative position and distance data to obtain the environmental structure information around the vehicle. The Bayesian fusion algorithm is used to perform probabilistic inference and fusion of weather and environmental data to assess the impact of current road conditions and weather on driving safety; Feature extraction and correlation analysis are performed based on the comprehensively processed data to generate a unified environmental perception model.

[0011] In an optional implementation, the unified environment perception model includes: a road model, an object detection and tracking model, a vehicle's own state model, a dynamic traffic environment model, a road and weather model, and a traffic signal and road sign recognition model.

[0012] In an optional embodiment, the target recognition using computer vision technology based on convolutional neural networks to identify pedestrians, vehicles, traffic signs and obstacles includes: Extracting real-time image data of the front and surrounding environment from the vehicle's surrounding environment information; Use Gaussian filtering algorithm or bilateral filtering algorithm to remove noise from real-time image data, enhance real-time image data according to weather and environmental data, perform image cropping and resolution scaling, and generate target data; Based on the target data, the edge information is extracted using a convolutional neural network, the target is detected using the YOLO algorithm, and the target position is selected; the target includes but is not limited to: vehicles, pedestrians, and road signs; The targets are further classified through the ResNet network to identify the types and attributes of the targets; Extracting attributes of the target, wherein the attributes of the target include but are not limited to: type and color of the vehicle, and posture of the pedestrian; Based on the target data, the depth information of the target object is estimated through the depth estimation network.

[0013] In an optional embodiment, the use of a deep learning model to perform behavior prediction to predict possible behavior trajectories of the vehicle or pedestrian ahead includes: Generate model training set and model validation set based on the environment perception model; Build a behavior prediction model through the long short-term memory network LSTM, use the model training set to train the model, and optimize the model parameters through the back propagation algorithm; Use the model validation set to evaluate the behavior prediction model, calculate the accuracy, recall rate, and F1 score to evaluate the prediction performance of the behavior prediction model, and optimize the model parameters and model structure based on the evaluation results; The environment perception model is input into the behavior prediction model to perform real-time prediction, obtain the possible behavior trajectory of the vehicle or pedestrian in front, and generate the prediction result.

[0014] In an optional implementation, the combining of the target recognition result and the behavior prediction result to evaluate the potential collision risk in the current scene includes: Integrate the target recognition results with the prediction results to form a comprehensive scene graph containing all key dynamic information, and analyze the current motion state of each target object in the scene to generate analysis results; Based on the comprehensive scene graph and analysis results, the collision risk assessment algorithm is used to quantify each potential collision event and obtain a specific collision risk value or probability; According to the collision risk value or probability, the collision risk is divided into different levels; According to the collision risk level, corresponding emergency measures are queried from the emergency measures information database; the emergency measures include but are not limited to deceleration, steering avoidance, emergency braking and automatic obstacle avoidance.

[0015] In a second aspect, the embodiment of the present application further provides a vehicle collision risk warning system, including: A multi-sensor perception module is used to collect the vehicle's surrounding environment information in real time through the vehicle's built-in sensors and GPS module; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environment data, real-time traffic flow and V2X data; The data processing and fusion module is used to comprehensively process the vehicle's surrounding environment information using multi-sensor fusion technology and generate a unified environmental perception model after data integration; Intelligent assessment module, which is used to identify pedestrians, vehicles, traffic signs and obstacles based on the environment perception model and computer vision technology based on convolutional neural networks; to predict the possible behavior trajectory of the vehicle or pedestrian ahead by using the behavior prediction based on the deep learning model; and to evaluate the potential collision risk in the current scene by combining the target recognition results and the behavior prediction results; The warning and active safety module is used to send a warning signal to the driver when a potential collision risk is detected, and automatically trigger active safety measures to avoid collision or reduce accident damage if the driver does not take corresponding measures.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the collision risk warning method as described in any one of the above items are implemented.

[0017] In a fourth aspect, an embodiment of the present application further provides a storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the collision risk warning method as described in any one of the above items are implemented.

[0018] It can be seen from the above technical solutions that the present invention has the following advantages: In the collision risk warning method provided by this application, through intelligent analysis and processing of sensor data, the vehicle driving environment is monitored in real time, possible collision risks are predicted, warnings are issued in time, and risk avoidance measures are automatically taken when necessary. This application adopts multiple advanced technical means such as multi-sensor fusion, deep learning algorithms, V2X (vehicle-to-everything) technology, which can effectively cope with complex road transportation scenarios and improve vehicle safety.

[0019] This application uses real-time data collection from the vehicle's built-in sensors and GPS module, as well as comprehensive processing using multi-sensor fusion technology. This method can fully and accurately acquire and integrate the vehicle's surrounding environment information and generate a unified environmental perception model. This greatly improves the vehicle's ability to perceive the surrounding environment and provides a reliable data basis for subsequent target recognition and behavior prediction.

[0020] This application uses computer vision technology based on convolutional neural networks and behavior prediction technology based on deep learning models. This method can accurately identify key targets such as pedestrians, vehicles, traffic signs and obstacles, and predict the possible behavior trajectory of the vehicle or pedestrian in front. This not only enhances the vehicle's ability to identify potential dangers, but also improves the accuracy of predicting collision risks, providing drivers with timely warning signals.

[0021] This application can evaluate the potential collision risk in the current scenario based on the results of target recognition and behavior prediction, and take corresponding emergency measures according to the risk level. This can not only effectively avoid or reduce the occurrence of vehicle collision accidents, but also improve the safety and efficiency of road traffic, and provide drivers and passengers with a safer and more comfortable travel experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A flow chart of the vehicle collision risk warning method provided in this application.

[0024] Figure 2 This is a schematic diagram of the structure of the vehicle collision risk warning system provided in this application.

[0025] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0026] In the specific steps of the vehicle collision risk warning method, which will be described in detail below, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0027] Hereinafter, the terms "include" or "may include" used in various embodiments of the present disclosure indicate the presence of disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. In addition, as used in various embodiments of the present disclosure, the terms "include", "have", and their cognates are intended only to indicate specific features, numbers, steps, operations, elements, components, or a combination of the foregoing, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, elements, components, or a combination of the foregoing or the possibility of adding one or more features, numbers, steps, operations, elements, components, or a combination of the foregoing.

[0028] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0029] See also Figure 1 The figure is a flow chart of a method for early warning of vehicle collision risk in a specific embodiment, the method comprising: S1: Real-time collection of the vehicle's surrounding environment information through the vehicle's built-in sensors and GPS module; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environment data, real-time traffic flow and V2X data.

[0030] For example, a variety of sensors can be used to collect information, including cameras, millimeter-wave radars, laser radars, ultrasonic sensors, and GPS modules. The camera is responsible for collecting real-time visual information in front of and to the sides of the vehicle; the millimeter-wave radar and laser radar are used to measure the distance, speed, and angle to the vehicle or obstacle in front; and the GPS provides real-time location information of the vehicle.

[0031] S2: Use multi-sensor fusion technology to comprehensively process the vehicle's surrounding environment information and generate a unified environmental perception model after data integration.

[0032] For example, multi-sensor fusion technology is used to comprehensively process data from different sensors, and a unified environmental perception model is generated after data processing and fusion processing and analysis. Based on this model, obstacles, road signs, traffic signals, pedestrians and other traffic participants around the vehicle can be detected. S3: Based on the environmental perception model, the convolutional neural network-based computer vision technology is used for target recognition to identify pedestrians, vehicles, traffic signs and obstacles; the behavior prediction based on the deep learning model is used to predict the possible behavior trajectory of the vehicle or pedestrian in front; the target recognition results and behavior prediction results are combined to evaluate the potential collision risk in the current scenario.

[0033] S4: When a potential collision risk is detected, a warning signal is sent to the driver, and if the driver does not take corresponding measures, active safety measures are automatically triggered to avoid collision or reduce accident damage.

[0034] For example, when a potential collision risk is detected, the driver is promptly reminded through sound and light signals. If the driver fails to take action in time, active safety measures will be automatically triggered, including emergency braking, steering or deceleration, to avoid collision or reduce accident damage.

[0035] In this embodiment, by real-time collection and comprehensive processing of vehicle surrounding environment information, convolutional neural networks and deep learning models are used to achieve accurate target recognition and behavior prediction, effectively assess potential collision risks, and send warning signals to the driver or automatically trigger safety measures when necessary, thereby significantly improving road traffic safety and accident prevention capabilities.

[0036] In an embodiment of the present invention, based on step S1, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0037] In step S1, the vehicle's built-in sensors and GPS module are used to collect the vehicle's surrounding environment information in real time, including: Road images, obstacle detection data, object depth data, and lane line data are collected by cameras, millimeter-wave radars, and lidars as environmental perception data. Specifically, road images are collected by cameras and used to identify information such as lanes, traffic signs, pedestrians, and vehicles, and are suitable for image segmentation, object detection, and recognition. Obstacle detection data is collected by millimeter-wave radars and lidars to detect the position, size, and movement direction of surrounding static and dynamic obstacles. Object depth information is collected by lidars or binocular cameras to estimate the distance and relative speed of objects in front (such as vehicles and pedestrians) to determine whether there is a risk of collision. Lane line data: collected by cameras, through image processing and edge detection algorithms, the type and position of lane lines are identified for vehicle positioning and lane keeping assistance.

[0038] The vehicle speed, acceleration, steering wheel angle, and vehicle posture data are collected as vehicle motion state data through the GPS model, vehicle-mounted speed sensor, steering wheel angle sensor, and inertial measurement unit. Specifically, the vehicle speed is collected by the vehicle-mounted GPS and speed sensor to monitor the vehicle's driving speed in real time, and combined with the distance information of the preceding vehicle for rear-end collision prevention analysis. The acceleration is collected by the acceleration sensor to detect the acceleration and deceleration state of the vehicle, and to assist in judging dangerous situations such as sudden braking or sudden acceleration. The steering wheel angle is collected by the steering wheel angle sensor to reflect the vehicle's steering state and help the system predict the vehicle's driving trajectory. The vehicle posture (yaw angle, roll angle, etc.) is collected by the inertial measurement unit (IMU) to analyze the vehicle's dynamic posture, especially in curves, lane changes, or bumpy sections, to help the system make dynamic collision avoidance adjustments.

[0039] The distance to the vehicle in front, the lateral distance and the distance to pedestrians are collected through cameras, millimeter-wave radars, lidars and ultrasonic sensors as relative position and distance data. Specifically, the distance to the vehicle in front is collected by millimeter-wave radars or lidars, which are used to monitor the distance to the vehicle in front in real time and assess potential rear-end collision risks. The lateral distance is collected by lateral ultrasonic sensors or side millimeter-wave radars to detect obstacles or vehicles in adjacent lanes to avoid collisions caused by lane changes in blind spots. The distance to pedestrians is collected jointly by cameras and lidars to detect the distance between pedestrians and vehicles, ensuring that anti-collision measures are taken at close ranges.

[0040] Traffic sign information, traffic light status information and road status information are collected through cameras and road status sensors as traffic signal and road information data. Specifically, traffic sign information is collected by cameras to identify traffic information such as speed limit signs, stop signs, and no-entry signs, and assist the system in reasonable speed control and path planning. Traffic light status is collected by on-board cameras or V2I (vehicle-to-infrastructure communication) to identify traffic light signal status and ensure that vehicles make correct decisions at traffic lights. Road conditions are collected through cameras or road status sensors to identify water, ice, snow or potholes on the road that are not conducive to safe driving, and help the system make dynamic adjustments.

[0041] Light intensity, temperature, humidity and rainfall information are collected through cameras, light sensors, and temperature and humidity sensors as weather and environmental data. Specifically, light intensity is collected by light sensors to determine current lighting conditions to adapt to image processing algorithms in nighttime, foggy and other environments. Humidity and temperature are collected by humidity sensors and temperature sensors to determine whether there is rain or snow, and the auxiliary system adjusts driving control, such as reducing vehicle speed and increasing braking distance. The amount of rain or snow is detected and collected by rain sensors or cameras to determine the degree of rain or snowfall, so as to help the system adjust driving strategies and avoid accidents caused by slippery roads.

[0042] The acceleration, speed and direction change information of the vehicle in front is collected through V2V communication, and the dynamic information of pedestrians and vehicles is collected through V2I as real-time traffic flow and V2X data. Specifically, the driving status of the vehicle in front is obtained through V2V (vehicle-to-vehicle communication), and the acceleration, speed and direction change of the vehicle in front are collected in real time. The auxiliary system predicts the behavior of the vehicle in front to ensure that a safe distance is maintained. The dynamic information of pedestrians and vehicles is collected through V2I (vehicle-to-infrastructure communication), including dynamic information released by pedestrian detection equipment and signal control systems, which helps the system avoid pedestrians and adjust the dynamic path.

[0043] In an embodiment of the present invention, based on step S2, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0044] This embodiment discloses a data fusion processing method, which specifically includes the following steps: S201: Filtering and data synchronization processing are performed on the vehicle's surrounding environment information.

[0045] Among them, Gaussian filtering can be used to process image noise, and Kalman filtering can be applied to remove errors in radar data. Since different sensors have different sampling frequencies and timestamps, time synchronization methods are used to align the data of each sensor. For example, radar, camera, and GPS data are synchronized using timestamps to ensure real-time and consistency of data.

[0046] S202: Using Kalman filter algorithm to fuse the relative position and distance data.

[0047] For example, the distance information of millimeter wave radar is fused with GPS position data to obtain a more accurate relative distance estimate. Kalman filtering is suitable for the fusion of sensor data in linear dynamic systems.

[0048] S203: Using the EKF algorithm to perform fusion processing on the vehicle motion state data.

[0049] For example, the EKF algorithm is used to fuse the data of the acceleration sensor and the steering wheel angle sensor for vehicle posture estimation.

[0050] S204: Using a particle filter algorithm to track the movement trajectory of a pedestrian or a vehicle according to the surrounding environment information of the vehicle.

[0051] Among them, the particle filter algorithm is used for data fusion in complex nonlinear and multimodal scenarios, such as tracking the motion trajectories of multiple pedestrians or vehicles around.

[0052] S205: Use a convolutional neural network to extract features from the road image, and fuse them with the relative position and distance data to obtain environmental structure information around the vehicle.

[0053] For example, a convolutional neural network (CNN) is used to extract features from image data, and features such as lanes and pedestrians recognized by the camera are integrated with the distance information detected by the radar to obtain the environmental structure around the vehicle.

[0054] S206: Use the Bayesian fusion algorithm to perform probabilistic inference and fusion of weather and environmental data to evaluate the impact of current road conditions and weather on driving safety.

[0055] For example, probabilistic inference and fusion of heterogeneous data (such as temperature, humidity and road conditions) from different sensors can be performed to evaluate the impact of current road conditions and weather on driving safety.

[0056] S207: Perform feature extraction and correlation analysis based on the comprehensively processed data to generate a unified environmental perception model.

[0057] It should be noted that the unified environmental perception model is a comprehensive model based on multi-sensor data fusion and deep learning algorithms, which is used to monitor and understand the vehicle's surrounding environment in real time. The specific content of the model is as follows: 1. Road model: Lane line recognition: Use the camera to perform image segmentation and edge detection to identify the location, type (solid line / dashed line), width and direction of lane lines. Lane line data is used for vehicle positioning and maintaining stable lane driving.

[0058] Lane width and curvature: Calculate the width and curvature of the lane based on the identified lane lines to help the vehicle maintain stability on special sections such as curves.

[0059] 2. Object detection and tracking model: Vehicle detection and classification: Use YOLO or SSD to detect vehicles in the image, identify the type of vehicle (sedan, truck, bus, etc.) and location, and estimate its speed by combining radar data.

[0060] Pedestrian recognition and trajectory prediction: Use convolutional neural networks and LSTM models to detect pedestrians in images and predict their movement trajectories so that the system can make avoidance decisions in time when pedestrians suddenly cross the road.

[0061] Obstacle recognition: LiDAR and millimeter-wave radar are used to detect static obstacles (such as guardrails and road bollards) and dynamic obstacles (such as animals and roadblocks that suddenly appear) in the environment to ensure that the vehicle avoids them in time.

[0062] 3. Vehicle status model: Speed ​​and acceleration status: The speed sensor and acceleration sensor are used to obtain the vehicle's driving speed and acceleration status in real time to determine the vehicle's acceleration or deceleration intention.

[0063] Steering status: The steering wheel angle sensor detects the current vehicle's steering status and combines it with lane information to ensure the vehicle's safety when changing lanes or turning.

[0064] Yaw angle and roll angle detection: The IMU is used to detect the vehicle's posture, especially when driving at high speed or making sharp turns, to ensure the stability of the vehicle and prevent dangerous situations such as rollover.

[0065] 4. Dynamic traffic environment model: Driving status of the vehicle in front: The speed, acceleration and direction changes of the vehicle in front are obtained through radar and V2V communication to help the system analyze the behavior of the vehicle in front (such as sudden deceleration) to avoid the risk of rear-end collision.

[0066] Side Vehicle and Blind Spot Detection: Monitors vehicles in adjacent lanes through side millimeter-wave radar or ultrasonic sensors and identifies potential risks in blind spots to ensure that no collision occurs when changing lanes.

[0067] 5. Road and weather model: Road condition recognition: Use cameras and road condition sensors to detect road conditions (such as slippery, icy, potholes, etc.), and combine humidity and temperature data to assess the danger of the current road conditions.

[0068] Weather condition judgment: Use data from light sensors, temperature sensors, humidity sensors, etc. to judge current weather conditions (such as light intensity, rainfall, snowfall), and adjust the vehicle's driving mode to adapt to complex weather.

[0069] 6. Traffic signal and road sign recognition: Traffic light signal recognition: Use cameras and V2I communication to obtain traffic light status, and combine it with traffic signs (such as stop signs, speed limit signs, etc.) to assist the system in making accurate decisions at traffic intersections.

[0070] Speed ​​limit and warning sign recognition: The system uses image recognition algorithms to identify speed limit signs, curve warning signs, etc. The system adjusts the vehicle speed in a timely manner according to road signs to avoid speeding or danger due to failure to slow down in time.

[0071] In an embodiment of the present invention, based on step S3, a possible embodiment is given below to illustrate its specific implementation scheme in a non-limiting manner.

[0072] In this embodiment, the specific process of step S3 is as follows: S301: Extracting real-time image data of the front and surrounding environment from the surrounding environment information of the vehicle.

[0073] S302: Use a Gaussian filtering algorithm or a bilateral filtering algorithm to remove noise from the real-time image data, enhance the real-time image data according to weather and environmental data, and perform image cropping and resolution scaling to generate target data.

[0074] S303: Based on the target data, edge information is extracted using a convolutional neural network, the target is detected using a YOLO algorithm, and the target position is selected; the target includes but is not limited to: vehicles, pedestrians, and road signs.

[0075] S304: Further classify the target through the ResNet network to identify the type and attribute of the target.

[0076] S305: Extracting attributes of the target, where the attributes of the target include but are not limited to: type and color of the vehicle, and posture of the pedestrian.

[0077] S306: Estimate the depth information of the target object through a depth estimation network based on the target data.

[0078] S307: Generate a model training set and a model verification set based on the environment perception model.

[0079] S308: Build a behavior prediction model through the long short-term memory network LSTM, use the model training set to train the model, and optimize the model parameters through the back propagation algorithm.

[0080] S309: Evaluate the behavior prediction model using the model validation set, evaluate the prediction performance of the behavior prediction model by calculating the accuracy, recall rate, and F1 score, and optimize the model parameters and model structure according to the evaluation results.

[0081] S310: Input the environment perception model into the behavior prediction model, perform real-time prediction, obtain the behavior trajectory of the vehicle or pedestrian in front, and generate a prediction result.

[0082] S311: Integrate the target recognition result and the prediction result to form a comprehensive scene graph containing all key dynamic information, and analyze the current motion state of each target object in the scene to generate analysis results.

[0083] S312: Based on the comprehensive scenario graph and analysis results, a collision risk assessment algorithm is used to quantify each potential collision event and obtain a specific collision risk value or probability.

[0084] S313: Classify the collision risk into different levels according to the collision risk value or probability.

[0085] S314: According to the collision risk level, query the emergency measures from the emergency measures information database; the emergency measures include but are not limited to deceleration, steering avoidance, emergency braking and automatic obstacle avoidance.

[0086] like Figure 2 As shown, the following is an embodiment of the vehicle collision risk warning system provided by the embodiment of the present disclosure. The system and the vehicle collision risk warning methods of the above-mentioned embodiments belong to the same inventive concept. For details not described in detail in the embodiment of the vehicle collision risk warning system, please refer to the embodiment of the above-mentioned vehicle collision risk warning method.

[0087] A vehicle collision risk warning system comprises: a multi-sensor perception module 1, a data processing and fusion module 2, an intelligent evaluation module 3 and a warning and active safety module 4.

[0088] The multi-sensor perception module 1 is used to collect the vehicle's surrounding environment information in real time through the vehicle's built-in sensors and GPS module; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environmental data, real-time traffic flow and V2X data.

[0089] The data processing and fusion module 2 is used to comprehensively process the vehicle's surrounding environment information using multi-sensor fusion technology, and generate a unified environmental perception model after data integration.

[0090] Intelligent assessment module 3 is used to perform target recognition based on the environment perception model using computer vision technology based on convolutional neural networks to identify pedestrians, vehicles, traffic signs and obstacles; perform behavior prediction based on a deep learning model to predict the possible behavior trajectory of the vehicle or pedestrian in front; and combine the target recognition results and behavior prediction results to evaluate the potential collision risk in the current scene.

[0091] The warning and active safety module 4 is used to send a warning signal to the driver when a potential collision risk is detected, and automatically trigger active safety measures to avoid collision or reduce accident damage if the driver fails to take corresponding measures.

[0092] The vehicle collision risk warning system provided in this embodiment achieves deep perception, accurate prediction and intelligent decision-making of the traffic environment by comprehensively using artificial intelligence, deep learning and sensor fusion technology, overcomes the defects of traditional vehicle collision avoidance systems that rely on driver reaction, insufficient prediction ability, insufficient sensor fusion and lack of intelligent decision-making ability, and significantly improves the safety and reliability of vehicle driving.

[0093] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0094] The vehicle collision risk warning method provided in the embodiment of the present application can be applied to electronic devices. Those skilled in the art will appreciate that the electronic device structure involved in the embodiment of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or less components than shown, or combine certain components, or arrange different components. In the embodiment of the present invention, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the embodiments of the present application described and / or required herein.

[0095] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, buttons, a camera, a display, and a SIM card interface, etc.

[0096] The processor may include one or more processing units, for example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors.

[0097] The processor can be the nerve center and command center of the electronic device. The controller can generate an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0098] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or is cyclically used. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves system efficiency.

[0099] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to implement data storage functions. For example, files such as music and videos can be saved in the external memory card.

[0100] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The internal memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0101] The wireless communication function of an electronic device can be realized through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0102] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0103] Electronic devices can implement audio functions, etc. through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0104] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0105] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0106] The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0107] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0108] The above-mentioned electronic device realizes the vehicle collision risk warning method of the present application by real-time collection of the vehicle's surrounding environment information, including multi-dimensional perception data and dynamic traffic information, and uses multi-sensor fusion technology for precise comprehensive processing, as well as adopting advanced computer vision technology and deep learning models for target recognition and behavior prediction. The method realizes effective assessment and timely warning of potential collision risks, thereby significantly improving the safety and efficiency of road traffic, providing drivers with safer and more reliable driving assistance, and achieving the beneficial effect of improving travel experience.

[0109] The storage medium provided in the present application stores a program product that can implement a vehicle collision risk warning method.

[0110] The vehicle collision risk warning method includes: collecting the vehicle's surrounding environment information in real time through the vehicle's built-in sensors and GPS module; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environmental data, real-time traffic flow and V2X data; using multi-sensor fusion technology to comprehensively process the vehicle's surrounding environment information, and generating a unified environmental perception model after data integration; based on the environmental perception model, using computer vision technology based on convolutional neural networks to perform target recognition to identify pedestrians, vehicles, traffic signs and obstacles; using a deep learning model to perform behavior prediction to predict the possible behavior trajectory of the vehicle or pedestrian in front; combining the target recognition results and the behavior prediction results to evaluate the potential collision risk in the current scenario; when a potential collision risk is detected, a warning signal is sent to the driver, and when the driver does not take corresponding measures, active safety measures are automatically triggered to avoid collision or reduce accident damage.

[0111] In some possible implementations, the vehicle collision risk warning method disclosed herein can be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above “Exemplary Method” section of this specification according to various exemplary implementations of the present disclosure.

[0112] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0113] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle collision risk warning method, characterized in that: include: Collect the vehicle's surrounding environment information in real time through the vehicle's built-in sensors and GPS module; The surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environmental data, real-time traffic flow and V2X data; Use multi-sensor fusion technology to comprehensively process the vehicle's surrounding environment information and generate a unified environmental perception model after data integration; Based on the environmental perception model, the convolutional neural network-based computer vision technology is used to perform target recognition to identify pedestrians, vehicles, traffic signs and obstacles; the deep learning model is used to perform behavior prediction to predict the behavior trajectory of the vehicle or pedestrian in front; the target recognition results and behavior prediction results are combined to evaluate the potential collision risk in the current scene; When a potential collision risk is detected, a warning signal is sent to the driver, and if the driver does not take corresponding measures, active safety measures are automatically triggered to avoid collision or reduce accident damage.

2. The vehicle collision risk warning method according to claim 1, characterized in that: The real-time collection of the vehicle's surrounding environment information through the vehicle's built-in sensors and GPS module includes: Collect road images, obstacle detection data, object depth data, and lane line data as environmental perception data through cameras, millimeter-wave radars, and lidars; The vehicle speed, acceleration, steering wheel angle, and vehicle posture data are collected as vehicle motion state data through the GPS model, vehicle-mounted speed sensor, steering wheel angle sensor, and inertial measurement unit; The distance to the vehicle in front, the side distance, and the distance to pedestrians are collected through cameras, millimeter-wave radars, lidars, and ultrasonic sensors as relative position and distance data; Traffic sign information, traffic light status information and road status information are collected as traffic signal and road information data through cameras and road status sensors; Collect light intensity, temperature, humidity and rainfall information as weather and environmental data through cameras, light sensors, and temperature and humidity sensors; The acceleration, speed and direction change information of the vehicle in front is collected through V2V communication, and the dynamic information of pedestrians and vehicles is collected through V2I as real-time traffic flow and V2X data.

3. The vehicle collision risk warning method according to claim 2, characterized in that: The multi-sensor fusion technology is used to comprehensively process the vehicle's surrounding environment information, and a unified environment perception model is generated after data integration, including: Perform filtering and data synchronization on the vehicle's surrounding environment information; The Kalman filter algorithm is used to fuse the relative position and distance data; The EKF algorithm is used to fuse the vehicle motion state data; The particle filter algorithm is used to track the movement trajectory of pedestrians or vehicles based on the surrounding environment information of the vehicle; Convolutional neural network is used to extract features from road images and fuse them with relative position and distance data to obtain the environmental structure information around the vehicle. The Bayesian fusion algorithm is used to perform probabilistic inference and fusion of weather and environmental data to assess the impact of current road conditions and weather on driving safety; Feature extraction and correlation analysis are performed based on the comprehensively processed data to generate a unified environmental perception model.

4. The vehicle collision risk warning method according to claim 3, characterized in that: The unified environment perception model includes: a road model, an object detection and tracking model, a vehicle state model, a dynamic traffic environment model, a road and weather model, and a traffic signal and road sign recognition model.

5. The vehicle collision risk warning method according to claim 1, characterized in that: The method of using computer vision technology based on convolutional neural networks to perform target recognition to identify pedestrians, vehicles, traffic signs and obstacles includes: Extracting real-time image data of the front and surrounding environment from the vehicle's surrounding environment information; Use Gaussian filtering algorithm or bilateral filtering algorithm to remove noise from real-time image data, enhance real-time image data according to weather and environmental data, perform image cropping and resolution scaling, and generate target data; Based on the target data, the edge information is extracted using a convolutional neural network, the target is detected using the YOLO algorithm, and the target position is selected; the target includes but is not limited to: vehicles, pedestrians, and road signs; The targets are further classified through the ResNet network to identify the types and attributes of the targets; Extracting attributes of the target, wherein the attributes of the target include but are not limited to: type and color of the vehicle, and posture of the pedestrian; Based on the target data, the depth information of the target object is estimated through the depth estimation network.

6. The vehicle collision risk warning method according to claim 5, characterized in that: The method of using a deep learning model to predict the behavior trajectory of a vehicle or pedestrian ahead includes: Generate model training set and model validation set based on the environment perception model; Build a behavior prediction model through the long short-term memory network LSTM, use the model training set to train the model, and optimize the model parameters through the back propagation algorithm; Use the model validation set to evaluate the behavior prediction model, calculate the accuracy, recall rate, and F1 score to evaluate the prediction performance of the behavior prediction model, and optimize the model parameters and model structure based on the evaluation results; The environment perception model is input into the behavior prediction model to perform real-time prediction, obtain the behavior trajectory of the vehicle or pedestrian in front, and generate the prediction result.

7. The vehicle collision risk warning method according to claim 6, characterized in that: The target recognition result and the behavior prediction result are combined to evaluate the potential collision risk in the current scene, including: Integrate the target recognition results with the prediction results to form a comprehensive scene graph containing all key dynamic information, and analyze the current motion state of each target object in the scene to generate analysis results; Based on the comprehensive scene graph and analysis results, the collision risk assessment algorithm is used to quantify each potential collision event and obtain a specific collision risk value or probability; According to the collision risk value or probability, the collision risk is divided into different levels; According to the collision risk level, corresponding emergency measures are queried from the emergency measures information database; the emergency measures include but are not limited to deceleration, steering avoidance, emergency braking and automatic obstacle avoidance.

8. A vehicle collision risk warning system, characterized in that: The system adopts the vehicle collision risk warning method as claimed in any one of claims 1 to 7; The system comprises: A multi-sensor perception module is used to collect the vehicle's surrounding environment information in real time through the vehicle's built-in sensors and GPS module; the surrounding environment information includes: environmental perception data, vehicle motion status data, relative position and distance data, traffic signal and road information data, weather and environment data, real-time traffic flow and V2X data; The data processing and fusion module is used to comprehensively process the vehicle's surrounding environment information using multi-sensor fusion technology and generate a unified environmental perception model after data integration; Intelligent assessment module, which is used to identify pedestrians, vehicles, traffic signs and obstacles based on the environment perception model and computer vision technology based on convolutional neural networks; to predict the possible behavior trajectory of the vehicle or pedestrian ahead by using the behavior prediction based on the deep learning model; and to evaluate the potential collision risk in the current scene by combining the target recognition results and the behavior prediction results; The warning and active safety module is used to send a warning signal to the driver when a potential collision risk is detected, and automatically trigger active safety measures to avoid collision or reduce accident damage if the driver does not take corresponding measures.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the vehicle collision risk warning method as described in any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the vehicle collision risk warning method according to any one of claims 1 to 7 are implemented.

Images