Vehicle safety auxiliary method and system and vehicle

Abstract

The invention provides a vehicle safety assisting method and system and a vehicle. The method comprises the steps that scene information of the vehicle is acquired; obstacle identification and obstacle behavior prediction are carried out on the scene information to obtain space state information of the target obstacle in future time; according to the spatial state information, a target area is determined, and the target area comprises a drivable area and / or a non-drivable area; and controlling the vehicle to execute corresponding safety auxiliary operation according to the target area and the predicted driving track of the vehicle. According to the invention, through obstacle identification and obstacle behavior prediction, the accuracy of the spatial state information of the obstacle in the future time is improved, the accuracy of target area determination is further improved, and on the basis of the target area and the predicted driving track of the vehicle, the driving risk of the vehicle can be effectively reduced under the condition that the vehicle faces the driving risk. The vehicle is timely controlled to execute the corresponding safety auxiliary operation, and the reliability of the safety auxiliary operation and the safety of vehicle driving are improved.

Description

Technical Field

[0001] The present application relates to the field of vehicle control, and in particular to a vehicle safety assistance method, system and vehicle. Background Art

[0002] During vehicle driving, the driver has blind spots in his vision and cannot accurately grasp the obstacles around the vehicle with his eyes, which poses certain safety hazards.

[0003] In the related art, a radar sensor is installed on a vehicle. During the driving process of the vehicle, the radar sensor detects obstacles around the vehicle in real time. When an obstacle is detected, an alarm is issued to the driver of the vehicle.

[0004] However, the above methods are not very effective in improving vehicle safety. Summary of the invention

[0005] Based on the above technical status, the present application provides a vehicle safety assistance method, system and vehicle, which can improve the execution accuracy of vehicle safety assistance operations and thus improve driving safety.

[0006] In order to achieve the above technical objectives, this application specifically proposes the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a vehicle safety assistance method is provided, comprising: acquiring scene information of a vehicle; performing obstacle recognition and obstacle behavior prediction on the scene information to obtain spatial state information of a target obstacle at a future time; determining a target area based on the spatial state information, the target area including a drivable area and / or a non-drivable area; and controlling the vehicle to perform corresponding safety assistance operations based on the target area and the predicted driving trajectory of the vehicle at the future time.

[0008] In some implementations, the scene information includes a scene video, and the performing obstacle identification and obstacle behavior prediction on the scene information to obtain spatial state information of the target obstacle in the future includes: performing obstacle identification on the scene video through a video target detection model to obtain identification information of the target obstacle; and performing obstacle behavior prediction on the target obstacle in combination with the scene video and the identification information to obtain the spatial state information.

[0009] In some implementations, the target obstacle includes a dynamic obstacle, the spatial state information includes motion trajectory information of the dynamic obstacle at the future time, and combining the scene video and the identification information to predict the obstacle behavior of the target obstacle to obtain the spatial state information includes: determining the dynamic obstacle among the target obstacles, and obtaining the identification information of the dynamic obstacle from the identification information of the target obstacle; using the identification information of the dynamic obstacle as enhanced data of the scene video, and inputting the identification information of the dynamic obstacle and the scene video into a behavior prediction model; in the behavior prediction model, based on the identification information of the dynamic obstacle and the scene video, predicting the motion trajectory of the dynamic obstacle at the future time to obtain the motion trajectory information.

[0010] In some implementations, the target obstacle includes a static obstacle, the spatial state information includes contour position information of the static obstacle, and combining the scene video and the identification information to predict the obstacle behavior of the target obstacle to obtain the spatial state information includes: determining the static obstacle in the target obstacle, and obtaining the identification information of the static obstacle from the identification information of the target obstacle; and performing contour position detection on the static obstacle in the scene video according to the identification information of the static obstacle to obtain the contour position information.

[0011] In some implementations, determining the target area based on the spatial state information includes: determining position relationship information of the target obstacle at a future time based on the spatial state information, the position relationship information including at least one of the following: relative position information between the target obstacles, relative position information between the target obstacle and a road boundary, and relative position information between the target obstacle and the vehicle; determining the target area based on the position relationship information.

[0012] In some implementations, when the target area includes a drivable area, determining the target area based on the position relationship information includes: determining a candidate area where the target obstacle does not exist based on the position relationship information; and screening out the drivable area from the candidate areas based on vehicle traffic conditions.

[0013] In some implementations, the controlling the vehicle to perform corresponding safety assistance operations based on the target area and the predicted driving trajectory of the vehicle at the future time includes: predicting the collision risk of the vehicle based on the target area and the predicted driving trajectory to obtain a prediction result; if the prediction result satisfies a safety assistance condition, controlling the vehicle to perform corresponding safety assistance operations; wherein the safety assistance conditions include warning conditions and / or braking conditions, and the safety assistance operations include warning prompt operations and / or braking operations.

[0014] In some implementations, the prediction result includes a predicted duration of a collision, and when the prediction result meets a safety assistance condition, the vehicle is controlled to perform corresponding safety assistance operations, including: when the predicted duration of a collision is less than or equal to a first duration threshold, the vehicle is controlled to perform the early warning prompt operation; and / or, when the predicted duration of a collision is less than or equal to a second duration threshold, the vehicle is controlled to perform the braking operation; wherein the second duration threshold is less than the first duration threshold.

[0015] According to a second aspect of an embodiment of the present application, a vehicle safety assistance system is provided, comprising: a safety assistance device, the safety assistance device comprising a memory and a processor, the memory storing a safety assistance program, and the safety assistance program, when executed by the processor, is used to implement the vehicle safety assistance method according to the first aspect or any implementation method of the first aspect.

[0016] According to a third aspect of an embodiment of the present application, a vehicle is provided, wherein the vehicle is equipped with the vehicle safety assistance system described in the second aspect.

[0017] A vehicle safety assistance method, system and vehicle provided in the embodiments of the present application obtain spatial state information of a target obstacle at a future time by performing obstacle recognition and obstacle behavior prediction on scene information of the vehicle, thereby improving the accuracy of the spatial state information of the target obstacle at the future time; based on the spatial state information of the target obstacle at the future time, a prediction of a target area (a drivable area and / or a non-drivable area) of the vehicle is determined, thereby improving the accuracy of the target area; the predicted driving trajectory of the target area and the vehicle can accurately reflect the driving risk of the vehicle, and combined with the predicted driving trajectory of the target area and the vehicle, the vehicle can be controlled in time to perform safety assistance operations when the vehicle faces driving risks, thereby improving the reliability of the execution of safety assistance operations and the safety of vehicle driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0019] Figure 1 A process of a vehicle safety assistance method provided in an embodiment of the present application Figure 1 ;

[0020] Figure 2 A process of a vehicle safety assistance method provided in an embodiment of the present application Figure 2 ;

[0021] Figure 3 is a structural schematic diagram of a vehicle safety auxiliary device provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of the structure of a safety assistance system provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of the structure of a vehicle provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The technical solution proposed in the embodiment of this application is applicable to vehicle driving scenarios, and aims to significantly improve the safety of vehicle driving scenarios by improving the vehicle safety assistance method. The technical solution described in the embodiment of this application can enhance the accuracy of risk perception during vehicle driving and improve the timeliness of vehicle safety assistance operations.

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] Before introducing this application solution, the relevant technologies are first introduced:

[0027] The driver has a blind spot in the vehicle, which leads to certain safety hazards when driving. Take the low-speed driving scenario as an example: when starting, the driver's vision is partially blocked by the hood and cannot see the low objects in front of the front of the vehicle; when reversing, the driver's vision is blocked by the vehicle body and cannot see obstacles on the side of the vehicle or behind the vehicle.

[0028] In the related technology, in order to improve the driving safety of the vehicle, the real-time conditions around the vehicle are detected by the sensors equipped on the vehicle. According to the real-time conditions detected by the sensors, the driver adjusts the driving strategy according to the detected conditions. Method 1: When the vehicle is driving at a low speed, the ultrasonic radar is used to detect obstacles around the vehicle, and an alarm is issued when an obstacle is detected to remind the driver to adjust the driving strategy; Method 2: When the vehicle is parked, four cameras are used to collect the surrounding environment of the vehicle, and the images collected by the four cameras are synthesized into a panoramic image, and the display module is controlled to display the panoramic image.

[0029] However, method one cannot identify obstacles and may result in false alarms. The accuracy of risk identification during vehicle driving is low, and the effect of improving vehicle safety is not good. Method two provides the driver with real-time images of the surrounding area. Whether the vehicle currently faces risks depends on the driver's experience and judgment, and the accuracy cannot be guaranteed, and the effect of improving vehicle safety is not good.

[0030] In view of this, the embodiments of the present application are committed to providing a vehicle safety assistance method, system and vehicle, which obtains the spatial state information of the target obstacle in the future by performing obstacle recognition and obstacle behavior prediction on the vehicle's scene information, thereby improving the prediction accuracy of the spatial state information; based on the spatial state information, a target area is determined, and the target area includes a drivable area and / or a non-drivable area, thereby improving the prediction accuracy of the target area; the target area and the predicted driving trajectory of the vehicle in the future can reflect the risk situation of the vehicle in the future, and based on the target area and the predicted driving trajectory, when the vehicle faces driving risks, the vehicle can be promptly controlled to perform corresponding safety assistance operations, thereby improving the reliability of the safety assistance operations and the safety of vehicle driving.

[0031] Exemplary Methods

[0032] Figure 1 A process of a vehicle safety assistance method provided in an embodiment of the present application Figure 1 .like Figure 1 As shown, the vehicle safety assistance method provided in this embodiment includes the following steps S101 to S104:

[0033] S101. Obtain scene information of the vehicle.

[0034] Among them, the current scene of the vehicle is the driving scene.

[0035] In one example, the current scene of the vehicle is a low-speed driving scene. In the low-speed driving scene, the driving environment of the vehicle is more complex (for example, the traffic is more congested, the probability of obstacles appearing is higher, there are more blind spots in the field of vision due to the obstruction of facilities or other objects, and the distance between the obstacle and the vehicle is closer, leaving the driver with less reaction time). The vehicle safety assistance method provided in this embodiment can improve the safety of vehicle driving in the low-speed driving scene.

[0036] Optionally, the low-speed driving scenarios include at least one of the following: vehicle starting scenarios, vehicle reversing scenarios, low-speed meeting scenarios, traffic intersection scenarios, and scenes with dense pedestrian flow (such as roads around schools, roads around scenic spots, etc.). The vehicle safety assistance method provided in this embodiment can improve the safety of vehicle driving in these low-speed driving scenarios.

[0037] In this embodiment, the scene information of the vehicle is collected in real time through sensors installed on the vehicle.

[0038] In one example, the sensor installed on the vehicle includes a camera device, which can collect scene information of the vehicle in real time, and the scene information includes a scene video of the current scene of the vehicle. Compared with the scene point cloud data collected by the radar sensor, the scene video collected by the camera device can more accurately reflect the type of obstacles and reduce the probability of misidentification of obstacles.

[0039] Among them, the camera device can be a traditional camera device, or it can be a three-dimensional vision camera device (a camera device including three-dimensional vision sensors such as structured light sensors and binocular vision sensors), an intelligent camera device (a camera device including a processor and an intelligent algorithm, and capable of running an intelligent algorithm through a processor), a far-infrared imaging camera device, etc.

[0040] Optionally, the camera device is calibrated in advance. After the scene video is obtained through the calibrated camera device, the video frames in the scene video can be corrected according to the internal and external parameters of the camera device (such as camera intrinsic parameters and camera extrinsic parameters) to improve the video quality of the scene video.

[0041] Optionally, the camera device may include multiple cameras corresponding to multiple shooting directions to obtain scene videos in multiple directions, thereby improving the comprehensiveness of scene information.

[0042] In this optional method, the multi-channel camera may be at least four-channel cameras, so as to achieve full coverage of the current scene where the vehicle is located through at least four-channel cameras.

[0043] Furthermore, the sensors installed on the vehicle may also include radar sensors, which can collect scene information of the vehicle in real time through cameras and radar sensors. The scene information includes scene point cloud data and scene video of the vehicle's current scene, so as to improve the accuracy of subsequent obstacle recognition and obstacle behavior prediction in combination with the scene point cloud data and scene video.

[0044] S102: Perform obstacle recognition and obstacle behavior prediction on the scene information to obtain spatial state information of the target obstacle in the future.

[0045] Among them, target obstacles refer to obstacles that require emergency avoidance during vehicle driving, such as pedestrians, animals, road facilities, and other large foreign objects.

[0046] The future time is a time period after the current moment, and the starting time of the future time can be the current moment or a future moment after the current moment.

[0047] Among them, the spatial state information of the target obstacle at the future time indicates the spatial state of the target obstacle at the future time, and the spatial state may include one or more of the obstacle position, obstacle shape, obstacle size, obstacle posture and / or obstacle motion state.

[0048] In this embodiment, the scene information includes obstacle information. For example, when the scene information includes a scene video, the video frame of the scene video includes image information of the obstacle; when the scene information includes scene point cloud data, the scene point cloud data includes outline information of the obstacle. Obstacle information can be extracted from the scene information, and the target obstacle can be identified based on the extracted obstacle information. The behavior of the identified target obstacle in the future can be predicted to obtain spatial state information of the target obstacle in the future.

[0049] S103: Determine the target area according to the spatial state information.

[0050] The target area includes a drivable area and / or a non-drivable area. When the vehicle drives to the drivable area at a future time, there is no safety hazard of colliding with the target obstacle; when the vehicle drives to the non-drivable area at a future time, there is a safety hazard of colliding with the target obstacle.

[0051] In this embodiment, based on the spatial state information of the target obstacle at the future time, a local area where there is no target obstacle in the future time can be identified to obtain a drivable area; alternatively, based on the spatial state information of the target obstacle at the future time, a local area where there is a target obstacle in the future time can be identified to obtain a non-drivable area.

[0052] S104: Control the vehicle to perform corresponding safety assistance operations according to the target area and the predicted driving trajectory of the vehicle in the future.

[0053] Among them, safety assistance operation refers to functions or operations used to assist in improving vehicle driving safety.

[0054] In this embodiment, the predicted driving trajectory of the vehicle in the future can be obtained, and the collision risk between the vehicle and the target obstacle in the future can be predicted based on the target area and the predicted driving trajectory. According to the collision risk prediction result, the vehicle is controlled to perform corresponding safety assistance operations.

[0055] In one example, the real-time vehicle body state information can be obtained, and the vehicle's driving trajectory in the future can be predicted based on the vehicle body state information to obtain the predicted driving trajectory of the vehicle in the future. The vehicle body state information may include one or more of the following: vehicle power state, vehicle driving gear state, vehicle steering wheel steering angle, vehicle driving speed, vehicle turn signal state, vehicle accelerator pedal state, vehicle brake pedal state, and vehicle actuator state. Vehicle actuators are used to control functional components of the vehicle, such as actuators in the electronic stability control (ESC) system and actuators for controlling vehicle body indicator lights.

[0056] In this example, a vehicle dynamics model can be constructed, and the vehicle's driving trajectory in the future can be predicted based on the vehicle body state information and the vehicle dynamics model to obtain the predicted driving trajectory of the vehicle in the future. In the process of constructing the vehicle dynamics model, multiple factors such as the vehicle's driving direction, the vehicle's weight, the friction between the vehicle's tires and the ground, and the vehicle's engine power can be considered. The vehicle dynamics model is constructed by combining kinematics and these multiple factors. The construction of the vehicle dynamics model is not specifically limited here. Alternatively, the vehicle body state information can be input into the vehicle trajectory prediction model, and the vehicle's driving trajectory in the future can be predicted by the vehicle trajectory prediction model to obtain the predicted driving trajectory of the vehicle in the future. The vehicle trajectory prediction model is a trained deep learning model.

[0057] In another example, the predicted behavior trajectory of the vehicle in the future generated by other tasks (such as navigation tasks, driving status analysis tasks, etc.) can be obtained.

[0058] In the embodiment of the present application, obstacle recognition and obstacle behavior prediction are performed on the vehicle's scene information to obtain the spatial state information of the target obstacle in the future. Based on the spatial state information, the target area is determined, thereby improving the prediction accuracy of the target area. The target area and the predicted driving trajectory of the vehicle in the future are combined to identify the collision risk between the vehicle and the target obstacle in the future. Based on the collision risk between the vehicle and the target obstacle in the future, the vehicle can be controlled in time to perform corresponding safety assistance operations, thereby improving the reliability of the safety assistance operations and the safety of vehicle driving.

[0059] Figure 2 A process of a vehicle safety assistance method provided in an embodiment of the present application Figure 2 .like Figure 2 As shown, the vehicle safety assistance method provided in this embodiment includes the following steps S201 to S205:

[0060] S201. Obtain scene information of the vehicle, where the scene information includes a scene video.

[0061] Among them, the implementation principle and technical effects of S201 can be referred to the aforementioned embodiments and will not be described in detail.

[0062] S202: Perform obstacle recognition on the scene video through a video target detection model to obtain recognition information of the target obstacle.

[0063] The video target detection model is a deep learning model used to detect and identify the target of interest in the input video. In this embodiment, the target of interest is the target obstacle. The video target detection model can be trained in advance based on the training data related to the target obstacle recognition (such as the scene video with the target obstacle marked), so that the video target detection model learns the obstacle features of the target obstacle and improves the recognition ability of the video target detection model for the target obstacle.

[0064] Among them, the identification information of the target obstacle may include one or more of the following: type information of the target obstacle, real-time position information of the target obstacle, real-time shape information of the target obstacle, real-time posture information of the target obstacle, and real-time motion state information of the target obstacle.

[0065] The real-time position information of the target obstacle may include: the real-time position coordinates of the center point of the target obstacle and / or the vertex coordinates of the real-time detection frame of the target obstacle. The real-time motion state information of the target obstacle may include one or more of the following: the real-time motion direction of the target obstacle, the real-time motion speed of the target obstacle, and the real-time motion displacement of the target obstacle.

[0066] In this embodiment, the scene video can be input into the video target detection model, in which obstacle features are extracted from the scene video, and target obstacles appearing in the scene video are identified based on the extracted obstacle features to obtain identification information of the target obstacles. Thus, the recognition accuracy of the target obstacles is improved through the deep learning model.

[0067] S203: combining the scene video and the recognition information, predicting the target obstacle's behavior, and obtaining the spatial state information of the target obstacle in the future.

[0068] In this embodiment, the scene video shows the scene space where the target obstacle is located. After obtaining the identification information of the target obstacle, the obstacle behavior of the target obstacle in the future can be predicted in the scene space shown by the scene video based on the identification information of the target obstacle, so as to obtain the spatial state information of the target obstacle in the future.

[0069] In one example, the target obstacle includes a dynamic obstacle, and the spatial state information of the target obstacle at a future time includes the motion trajectory information of the dynamic obstacle at a future time. A dynamic obstacle refers to a target obstacle whose position changes dynamically. For a target obstacle whose position changes dynamically, the motion trajectory information can more accurately reflect its spatial state information.

[0070] In one possible implementation, Figure 2 As shown, S203 includes: S2031, determining the dynamic obstacle among the target obstacles, and obtaining the identification information of the dynamic obstacle from the identification information of the target obstacle; S2032, using the identification information of the dynamic obstacle as the enhanced data of the scene video, inputting the identification information of the dynamic obstacle and the scene video into the behavior prediction model; S2033, in the behavior prediction model, based on the identification information of the dynamic obstacle and the scene video, predicting the motion trajectory of the dynamic obstacle in the future time, and obtaining the motion trajectory information. Thus, the prediction accuracy of the obstacle motion trajectory is improved through the behavior prediction model and data enhancement.

[0071] The motion trajectory information of the dynamic obstacle in the future may include position information of multiple time points of the dynamic obstacle in the future.

[0072] In S2031, dynamic obstacles may be searched for in the target obstacles according to the identification information of the target obstacle to improve the accuracy of dynamic obstacle search. For example, the identification information of the target obstacle includes the type information of the target obstacle. Among the target obstacles, obstacles whose type information is a set dynamic object type (such as pedestrians, animals, and vehicles) may be determined to be dynamic obstacles. For another example, the identification information of the target obstacle includes the real-time position information of the target obstacle, and obstacles whose real-time position information has changed may be determined to be dynamic obstacles among the target obstacles. For another example, the identification information of the target obstacle includes the real-time motion state information of the target obstacle, and obstacles whose real-time motion state information contains a real-time motion direction, a real-time motion speed that is not zero, or a real-time motion displacement that is not zero may be determined to be dynamic obstacles among the target obstacles.

[0073] In S2032, the recognition information of the dynamic obstacles is used as the enhanced data of the scene video, and the recognition information of the dynamic obstacles and the scene video are input into the behavior prediction model: in one method, the recognition information of the moving obstacles can be used as the annotation information of the scene video to obtain the annotated scene video, and the annotated scene video is input into the behavior prediction model; in another method, the recognition information of the moving obstacles and the scene video are input into the behavior prediction model together as two independent data.

[0074] In S2033, in the behavior prediction model, feature information related to the behavior of the obstacle and feature information related to the driving scene can be extracted from the identification information of the dynamic obstacle and the scene video, and the motion trajectory of the dynamic obstacle in the future time is predicted based on the feature information related to the behavior of the obstacle and the feature information related to the driving scene, so as to obtain the motion trajectory information of the dynamic obstacle in the future time. Among them, the behavior prediction model is a deep learning model, and the behavior prediction model can be trained in advance based on training data related to the behavior prediction of the dynamic obstacle (such as a scene video containing a dynamic obstacle) to improve the accuracy of the behavior prediction model in predicting the behavior of the obstacle.

[0075] In another example, the target obstacle includes a static obstacle, and the spatial state information includes the contour position information of the static obstacle. The static obstacle refers to a target obstacle whose position remains unchanged, and the contour position information of the static obstacle can more accurately reflect the spatial state information of the static obstacle.

[0076] In a possible implementation, S203 includes: S2034, determining a static obstacle in the target obstacle, and obtaining identification information of the static obstacle from the identification information of the target obstacle; S2035, performing contour position detection on the static obstacle in the scene video according to the identification information of the static obstacle, and obtaining contour position information of the static obstacle.

[0077] The contour position information of the static obstacle may include position information corresponding to a plurality of contour points of the static obstacle.

[0078] In S2034, static obstacles may be searched for in the target obstacles according to the identification information of the target obstacle, so as to improve the accuracy of the static obstacle search. For example, the identification information of the target obstacle includes the type information of the target obstacle. In the target obstacles, obstacles whose type information is a set static object type (such as a flower bed, a tree, a house) may be determined as static obstacles. In another example, the identification information of the target obstacle includes the real-time position information of the target obstacle, and obstacles whose real-time position information remains unchanged may be determined as static obstacles in the target obstacles. In another example, the identification information of the target obstacle includes the real-time motion state information of the target obstacle, and obstacles whose real-time motion direction does not exist in the real-time motion state information, whose real-time motion speed is zero, or whose real-time motion displacement is zero may be determined as static obstacles in the target obstacles. Alternatively, after determining the dynamic obstacles, obstacles other than the dynamic obstacles in the target obstacles may be determined as static obstacles.

[0079] In S2035, after obtaining the identification information of the static obstacle, the contour position detection of the static obstacle can be performed in the scene video with the aid of the identification information of the static obstacle to obtain the contour position information of the static obstacle. For example, the identification information of the static obstacle includes the type information of the static obstacle, and the contour shape of the static obstacle can be determined based on the type information of the static obstacle. According to the contour shape of the static obstacle, the contour position detection of the static obstacle can be performed in the scene video to obtain the contour position information of the static obstacle. For another example, the identification information of the static obstacle includes the real-time position information of the static obstacle, and the image position of the static obstacle in the video frame included in the target video can be determined based on the real-time position information of the static obstacle, and the contour position detection of the static obstacle can be performed around the image position to obtain the contour position information of the static obstacle.

[0080] It should be noted that the execution order of steps S2034 and S2035 is irrelevant to the execution order of steps S2031 to S2033. Figure 2 Take S2034 and S2035 being executed after S2031 to S2033 as an example.

[0081] S204: Determine a target area according to the space status information, where the target area includes a drivable area and / or a non-drivable area.

[0082] S205: Control the vehicle to perform corresponding safety assistance operations according to the target area and the predicted driving trajectory of the vehicle in the future.

[0083] The implementation principles and technical effects of S204 to S205 refer to the aforementioned embodiments and will not be described in detail.

[0084] In the embodiment of the present application, obstacle recognition is performed on the scene information of the vehicle to obtain recognition information of the target obstacle. The recognition information and the scene video are combined to predict the obstacle behavior of the target obstacle to obtain the spatial state information of the target obstacle in the future, thereby improving the prediction accuracy of the spatial state information, and further improving the prediction accuracy of the target area. Combined with the target area and the predicted driving trajectory of the vehicle in the future, the vehicle can be more accurately controlled to perform corresponding safety assistance operations in a timely manner, thereby improving the reliability of the safety assistance operations and the safety of vehicle driving.

[0085] Below, based on any of the foregoing embodiments, more possible embodiments are provided for implementing some steps in the vehicle safety assistance method.

[0086] In some embodiments, determining the target area according to the spatial state information of the target obstacle at a future time includes: determining the position relationship information of the target obstacle at a future time according to the spatial state information of the target obstacle at a future time, the position relationship information including at least one of the following: relative position information between target obstacles, relative position information between the target obstacle and the road boundary, and relative position information between the target obstacle and the vehicle; determining the target area according to the position relationship information of the target obstacle at a future time. Thus, by analyzing the position relationship of the target obstacle at a future time, the accuracy of determining the target area for the vehicle is improved, that is, the accuracy of determining the drivable area and / or non-drivable area for the vehicle is improved.

[0087] Among them, the relative position information between target obstacles may include the relative distance and relative orientation between the target obstacles, the relative position information between the target obstacle and the road boundary may include the relative distance and relative orientation between the target obstacle and the road boundary, and the relative position information between the target obstacle and the vehicle may include the relative distance and relative orientation between the target obstacle and the vehicle.

[0088] In this embodiment, the position relationship information of the target obstacle in the future time may be determined according to the motion trajectory information of the dynamic obstacle in the future time and / or the contour position information of the static obstacle.

[0089] In the process of determining the relative position information between target obstacles, the position information corresponding to multiple target obstacles (dynamic obstacles and / or static obstacles) in the same time and space can be obtained from the motion trajectory information of the dynamic obstacle in the future and / or the contour position information of the static obstacle. According to the position information corresponding to the multiple target obstacles in the same time and space, the relative distance and relative orientation between any two of the multiple target obstacles are determined.

[0090] In the process of determining the relative position information between the target obstacle and the road boundary, the position information of the target obstacle (dynamic obstacle and / or static obstacle) can be obtained from the motion trajectory information of the dynamic obstacle in the future time and / or the contour position information of the static obstacle, and the relative distance and relative orientation between the target obstacle and the road boundary can be determined according to the position information of the target obstacle and the position information of the road boundary. The relative position information between the target obstacle and the vehicle can refer to the determination process of the relative position information between the target obstacle and the road boundary, which will not be repeated here.

[0091] In this embodiment, when the target area includes a drivable area, a local area without a target obstacle can be determined in a driving scene corresponding to the future time according to the positional relationship information of the target obstacle at the future time, and the drivable area can be determined as the local area without the target obstacle. When the target area includes a non-drivable area, a local area with a target obstacle can be determined in a driving scene corresponding to the future time according to the positional relationship information of the target obstacle at the future time, and the non-drivable area can be determined as the local area with the target obstacle.

[0092] Optionally, when the target area includes a drivable area, a candidate area without target obstacles is determined based on the position relationship information of the target obstacle at a future time; and a drivable area is screened out from the candidate area based on the vehicle traffic conditions. Thus, a drivable area without target obstacles and satisfying the vehicle traffic conditions is obtained, thereby improving the accuracy of the drivable area.

[0093] The vehicle passage condition may include a set vehicle passage width and / or a set vehicle passage height. The allowed passage width of the candidate area may be compared with the set vehicle passage width, and / or the allowed passage height of the candidate area may be compared with the set vehicle passage height, and according to the comparison result, a drivable area that meets the vehicle passage condition may be screened out from the candidate area.

[0094] In some embodiments, controlling the vehicle to perform corresponding safety assistance operations based on the target area and the predicted driving trajectory of the vehicle in the future may include: predicting the collision risk of the vehicle based on the target area and the predicted driving trajectory of the vehicle to obtain a prediction result; and controlling the vehicle to perform corresponding safety assistance operations when the prediction result meets the safety assistance conditions.

[0095] In one example, the target area includes a drivable area. Based on the separation between the drivable area and the predicted driving trajectory of the vehicle in the future, the behavior of the vehicle leaving the drivable area in the future can be detected to obtain the exit behavior detection result; according to the exit behavior detection result, the prediction result of the collision risk is determined. Thus, by detecting the exit behavior of the vehicle in the drivable area, the prediction accuracy of the collision risk between the vehicle and the target obstacle in the future is improved.

[0096] Among them, the predicted driving trajectory of the vehicle at a future time leaves the drivable area, indicating that the vehicle has left the drivable area at a future time.

[0097] Optionally, the exit behavior detection result is that the vehicle will exit the drivable area at a future time, and the collision risk prediction result is that there is a collision risk between the vehicle and the target obstacle at a future time. Alternatively, the exit behavior detection result is that the vehicle does not exit the drivable area at a future time, and the collision risk prediction result is that there is no collision risk between the vehicle and the target obstacle at a future time.

[0098] Optionally, the exit detection result includes that the vehicle will exit the drivable area at a future time and the exit time when the vehicle exits the drivable area at a future time; the collision risk prediction result includes the risk of collision between the vehicle and the target obstacle at a future time and the collision time between the vehicle and the target obstacle at a future moment, and the collision time is the above-mentioned exit time.

[0099] In another example, the target area includes a non-drivable area. Based on the intersection of the non-drivable area and the predicted driving trajectory of the vehicle in the future, the behavior of the vehicle entering the non-drivable area in the future can be detected to obtain the driving behavior detection result; according to the driving behavior detection result, the prediction result of the collision risk is determined. Thus, by detecting the driving behavior of the vehicle in the non-drivable area, the prediction accuracy of the collision risk between the vehicle and the target obstacle in the future is improved.

[0100] Among them, the predicted driving trajectory of the vehicle in the future time intersects with the non-drivable area, indicating that the vehicle has entered the non-drivable area in the future time.

[0101] Optionally, the entry behavior detection result is that the vehicle will enter the non-drivable area in the future, and the collision risk prediction result is that there is a collision risk between the vehicle and the target obstacle in the future. Alternatively, the entry behavior detection result is that the vehicle will not enter the non-drivable area in the future, and the collision risk prediction result is that there is no collision risk between the vehicle and the target obstacle in the future.

[0102] Optionally, the entry detection result includes that the vehicle will enter the non-drivable area at a future time and the entry time of the vehicle into the non-drivable area at a future time; the prediction result of the collision risk includes the risk of collision between the vehicle and the target obstacle at a future time and the collision time between the vehicle and the target obstacle at a future moment, and the collision time is the above-mentioned entry time.

[0103] The safety assistance conditions include warning conditions and / or braking conditions, and the safety assistance operations include warning prompt operations and / or braking operations. Thus, the driving safety of the vehicle when facing a collision risk is improved through the warning prompt operations and / or active braking operations.

[0104] In one example, the prediction result includes a predicted duration of a collision. When the prediction result meets the safety assistance conditions, the vehicle is controlled to perform corresponding safety assistance operations, including: when the predicted duration of a collision is less than or equal to a first duration threshold, the vehicle is controlled to perform a warning prompt operation; and / or, when the predicted duration of a collision is less than or equal to a second duration threshold, the vehicle is controlled to perform a braking operation; wherein the second duration threshold is less than the first duration threshold.

[0105] The predicted collision time refers to the time from the current time to the collision time. The collision time can be obtained from the prediction result of the collision risk. For details, please refer to the above embodiment.

[0106] In this example, when the predicted collision time is less than or equal to the first time threshold, it means that if the vehicle continues to drive according to the current driving strategy, it will collide with the target obstacle after the time interval of the first time threshold. To avoid the collision, the vehicle can be controlled to perform a warning prompt operation, such as outputting a warning prompt sound, and the driver can change the driving strategy after hearing the warning prompt sound; when the predicted collision time is less than or equal to the second time threshold, it means that if the vehicle continues to drive according to the current driving strategy, it will collide with the target obstacle after the time interval of the second time threshold. The second time threshold is less than the first time threshold, which means that the collision time is shorter. To avoid the collision, the vehicle can be controlled to perform a braking operation, such as controlling the vehicle to brake or decelerate, to ensure the safety of vehicle driving.

[0107] In some embodiments, the target area and / or target obstacle may be marked in the scene video of the vehicle according to the identification information of the target area and / or target obstacle, and the marked scene video may be obtained, and the vehicle may be controlled to display the marked scene video. Thus, by displaying the marked scene video, accurate and reliable safety auxiliary information is provided to the driver, thereby improving the driving safety of the vehicle.

[0108] Exemplary Devices

[0109] Corresponding to the above-mentioned vehicle safety assistance method, an embodiment of the present application also provides a vehicle safety assistance device. Figure 3 Schematic diagram of the structure of a vehicle safety auxiliary device provided in an embodiment of the present application. Figure 3 As shown, the vehicle safety auxiliary device 30 provided in the embodiment of the present application includes: an acquisition unit 31, a prediction unit 32, a determination unit 33 and a control unit 34:

[0110] The acquisition unit 31 is used to acquire scene information of the vehicle; the prediction unit 32 is used to perform obstacle recognition and obstacle behavior prediction on the scene information to obtain spatial state information of the target obstacle in the future; the determination unit 33 is used to determine the target area according to the spatial state information, and the target area includes a drivable area and / or a non-drivable area; the control unit 34 is used to control the vehicle to perform corresponding safety assistance operations according to the target area and the predicted driving trajectory of the vehicle.

[0111] In some embodiments, the scene information includes a scene video, and the prediction unit 32 is specifically used to: identify obstacles in the scene video through a video target detection model to obtain identification information of the target obstacle; and predict the obstacle behavior of the target obstacle by combining the scene video and the identification information to obtain spatial state information.

[0112] In some embodiments, the target obstacle includes a dynamic obstacle, and the spatial state information includes the motion trajectory information of the dynamic obstacle in the future. The prediction unit 32 is specifically used to: determine the dynamic obstacle among the target obstacles, and obtain the identification information of the dynamic obstacle from the identification information of the target obstacle; use the identification information of the dynamic obstacle as the enhanced data of the scene video, and input the identification information of the dynamic obstacle and the scene video into the behavior prediction model; in the behavior prediction model, based on the identification information of the dynamic obstacle and the scene video, predict the motion trajectory of the dynamic obstacle in the future to obtain the motion trajectory information.

[0113] In some embodiments, the target obstacle includes a static obstacle, the spatial state information includes contour position information of the static obstacle, and the prediction unit 32 is specifically used to: determine the static obstacle in the target obstacle, and obtain identification information of the static obstacle from the identification information of the target obstacle; and perform contour position detection on the static obstacle in the scene video according to the identification information of the static obstacle to obtain contour position information.

[0114] In some embodiments, the determination unit 33 is specifically used to: determine the position relationship information of the target obstacle at a future time based on the spatial state information, the position relationship information including at least one of the following: relative position information between target obstacles, relative position information between the target obstacle and the road boundary, and relative position information between the target obstacle and the vehicle; determine the target area based on the position relationship information.

[0115] In some embodiments, when the target area includes a drivable area, the determination unit 33 is specifically used to: determine a candidate area where no target obstacle exists according to the position relationship information; and filter out the drivable area from the candidate area according to the vehicle traffic conditions.

[0116] In some embodiments, the control unit 34 is specifically used to: predict the collision risk of the vehicle based on the target area and the predicted driving trajectory of the vehicle to obtain a prediction result; when the prediction result meets the safety assistance condition, control the vehicle to perform corresponding safety assistance operations; wherein, the safety assistance conditions include warning conditions and / or braking conditions, and the safety assistance operations include warning prompt operations and / or braking operations.

[0117] In some embodiments, the prediction result includes a predicted duration of a collision, and the control unit 34 is specifically used to: when the predicted duration of a collision is less than or equal to a first duration threshold, control the vehicle to perform a warning prompt operation; and / or, when the predicted duration of a collision is less than or equal to a second duration threshold, control the vehicle to perform a braking operation; wherein the second duration threshold is less than the first duration threshold.

[0118] The vehicle safety assistance device provided in this embodiment belongs to the same application concept as the vehicle safety assistance method provided in the above embodiments of this application, and can execute the vehicle safety assistance method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the vehicle safety assistance method. For technical details not fully described in this embodiment, please refer to the specific processing content of the vehicle safety assistance method provided in the above embodiments of this application, and will not be repeated here.

[0119] The functions implemented by the above acquisition unit 31, prediction unit 32, determination unit 33 and control unit 34 may be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0120] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0121] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0122] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0123] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0124] Exemplary Systems

[0125] The present application embodiment provides a safety assistance system, see Figure 4 As shown, the safety assistance system includes: a safety assistance device, which includes a memory 400 and a processor 410; wherein the memory 400 is connected to the processor 410 and is used to store programs; the processor 410 is used to implement the vehicle safety assistance method disclosed in any of the above embodiments by running the program stored in the memory 400.

[0126] Specifically, the above system may further include: a bus, a communication interface 420 , an input device 430 and an output device 440 .

[0127] The processor 410, the memory 400, the communication interface 420, the input device 430 and the output device 440 are connected to each other via a bus.

[0128] A bus may include a pathway that transfers information between components of a computer system.

[0129] Processor 410 may be a general purpose processor, such as a general purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0130] The processor 410 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0131] The memory 400 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 400 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0132] The input device 430 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0133] Output device 440 may include a device that allows information to be output to a user, such as a display screen, a printer, a speaker, etc.

[0134] The communication interface 420 may include any transceiver or the like to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0135] The processor 410 executes the program stored in the memory 400 and calls other devices, which can be used to implement each step of any vehicle safety assistance method provided in the above embodiments of the present application.

[0136] An embodiment of the present application also proposes a chip, which includes a processor and a data interface. The processor reads and runs a program stored in a memory through the data interface to execute the vehicle safety assistance method introduced in any of the above embodiments. The specific processing process and its beneficial effects can be found in the introduction to the embodiment of the above-mentioned vehicle safety assistance method.

[0137] An embodiment of the present application also proposes a vehicle, which is equipped with the above-mentioned safety assistance system and is used to execute the steps of the above-mentioned vehicle safety assistance method.

[0138] For an example, see Figure 5 The vehicle includes a video acquisition unit 51, a vehicle body information acquisition unit 52, a vehicle-side processing unit 53, an alarm unit 54 and a vehicle braking unit 55:

[0139] The video acquisition unit 51 is used to acquire the scene video of the vehicle, and transmit the scene video to the vehicle-side processing unit 53 through a bus (such as a network bus); the body information acquisition unit 52 is used to acquire the body status information of the vehicle, and transmit the body status information to the vehicle-side processing unit 53 through a bus; the vehicle-side processing unit 53 is deployed with a safety assistance system, which executes the steps of the vehicle safety assistance method provided in the above embodiment through the safety assistance system based on the scene video and the body status information, generates corresponding safety assistance operation execution instructions, and sends the execution instructions to the alarm unit 54 and / or the vehicle braking unit 55; the alarm unit 54 is used to respond to the received execution instruction and perform a warning prompt operation; the vehicle braking unit 55 is used to respond to the received execution instruction and perform a braking operation.

[0140] Optionally, the vehicle also includes a human-computer interaction unit 56. In the safety assistance system, the target obstacle and / or target area can be marked in the scene video, and the scene video marked with the target obstacle and / or target area is sent to the human-computer interaction unit 56. The human-computer interaction unit 56 is used to display the scene video marked with the target obstacle and / or target area.

[0141] Exemplary computer program products and storage media

[0142] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the vehicle safety assistance method according to various embodiments of the present application described in any of the above embodiments of this specification.

[0143] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0144] In addition, an embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the vehicle safety assistance method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically can implement the steps of the vehicle safety assistance method as described above.

[0145] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0146] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0147] The steps in the methods of each embodiment of the present application can be adjusted in order, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0148] The units of the device in each embodiment of the present application can be combined, divided and deleted according to actual needs.

[0149] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0150] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0151] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0152] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0153] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0154] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0155] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A vehicle safety assistance method, characterized in that: include: Obtain vehicle scene information; Perform obstacle recognition and obstacle behavior prediction on the scene information to obtain spatial state information of the target obstacle in the future; Determine a target area according to the spatial state information, wherein the target area includes a drivable area and / or a non-drivable area; The vehicle is controlled to perform a corresponding safety assistance operation according to the target area and the predicted driving trajectory of the vehicle at the future time.

2. The vehicle safety assistance method according to claim 1, characterized in that: The scene information includes a scene video, and the obstacle identification and obstacle behavior prediction are performed on the scene information to obtain spatial state information of the target obstacle at a future time, including: Obstacle identification is performed on the scene video through a video target detection model to obtain identification information of the target obstacle; The obstacle behavior of the target obstacle is predicted by combining the scene video and the recognition information to obtain the space state information.

3. The vehicle safety assistance method according to claim 2, characterized in that: The target obstacle includes a dynamic obstacle, the spatial state information includes motion trajectory information of the dynamic obstacle at the future time, and the obstacle behavior prediction of the target obstacle is performed by combining the scene video and the recognition information to obtain the spatial state information, including: Determine the dynamic obstacle among the target obstacles, and obtain identification information of the dynamic obstacle from identification information of the target obstacle; Using the identification information of the dynamic obstacle as enhanced data of the scene video, the identification information of the dynamic obstacle and the scene video are input into a behavior prediction model; In the behavior prediction model, based on the identification information of the dynamic obstacle and the scene video, the motion trajectory of the dynamic obstacle at the future time is predicted to obtain the motion trajectory information.

4. The vehicle safety assistance method according to claim 2, characterized in that: The target obstacle includes a static obstacle, the spatial state information includes contour position information of the static obstacle, and the combining the scene video and the recognition information to perform obstacle behavior prediction on the target obstacle to obtain the spatial state information includes: Determine the static obstacle among the target obstacles, and obtain identification information of the static obstacle from the identification information of the target obstacle; According to the identification information of the static obstacle, contour position detection is performed on the static obstacle in the scene video to obtain the contour position information.

5. The vehicle safety assistance method according to any one of claims 1 to 4, characterized in that: The step of determining a target area according to the spatial state information includes: Determine, according to the spatial state information, position relationship information of the target obstacle at the future time, the position relationship information including at least one of the following: relative position information between the target obstacles, relative position information between the target obstacle and a road boundary, and relative position information between the target obstacle and the vehicle; The target area is determined according to the position relationship information.

6. The vehicle safety assistance method according to claim 5, characterized in that: In the case where the target area includes a drivable area, determining the target area according to the position relationship information includes: Determining, based on the position relationship information, a candidate area where the target obstacle does not exist; The drivable area is screened out from the candidate areas according to vehicle traffic conditions.

7. The vehicle safety assistance method according to any one of claims 1 to 4, characterized in that: The controlling the vehicle to perform a corresponding safety assistance operation according to the target area and the predicted driving trajectory of the vehicle at the future time includes: According to the target area and the predicted driving trajectory, a collision risk prediction is performed on the vehicle to obtain a prediction result; When the prediction result satisfies the safety assistance condition, controlling the vehicle to perform a corresponding safety assistance operation; Among them, the safety auxiliary conditions include warning conditions and / or braking conditions, and the safety auxiliary operations include warning prompt operations and / or braking operations.

8. The vehicle safety assistance method according to claim 7, characterized in that: The prediction result includes a predicted duration of a collision, and when the prediction result satisfies a safety assistance condition, controlling the vehicle to perform a corresponding safety assistance operation includes: When the predicted collision duration is less than or equal to a first duration threshold, controlling the vehicle to perform the early warning prompt operation; and / or, when the predicted collision duration is less than or equal to a second duration threshold, controlling the vehicle to perform the braking operation; The second duration threshold is smaller than the first duration threshold.

9. A vehicle safety assistance system, characterized in that: include: A safety assistance device, comprising a memory and a processor, wherein a safety assistance program is stored in the memory, and when the safety assistance program is executed by the processor, it is used to implement the vehicle safety assistance method according to any one of claims 1 to 8.

10. A vehicle, characterized in that: The vehicle is equipped with the vehicle safety assistance system as claimed in claim 9.

Images