Driving assistance system and method based on visual recognition

Abstract

The present invention relates to the field of driving assistance technology, and specifically discloses a driving assistance system and method based on visual recognition, wherein the system includes a first image acquisition module for acquiring image data outside the vehicle; a driving control module for obtaining vehicle driving data and accelerator pedal force data; an auxiliary module for determining the environment outside the vehicle based on the image data outside the vehicle; judging the current state of the vehicle based on the driving data, and if the vehicle is in the state of moving or parking, judging whether there is an obstacle on the driving track based on the vehicle's driving track and the environment outside the vehicle, and if there is an obstacle, calculating the distance between the vehicle and the obstacle; and further for calculating the estimated force of the accelerator pedal based on a preset warning value and the distance between the vehicle and the obstacle, judging whether the throttle force data exceeds the estimated force, and if so, generating an alarm message. The technical solution of the present invention can ensure the safety of the vehicle and the driver in parking or moving scenes, and reduce the probability of accidents.

Description

Technical Field

[0001] The present invention relates to the field of driving assistance technology, and in particular to a driving assistance system and method based on visual recognition. Background Art

[0002] With the development of modern society, the number of cars is increasing. This requires drivers to be alert at all times while driving. They should not only avoid collisions with other vehicles, pedestrians, and obstacles, but also try to avoid the impact of other vehicles on themselves. However, since the driver's attention and observation range are limited during driving, sometimes they cannot detect the situation ahead in time and avoid the impact of other vehicles, which may lead to traffic accidents.

[0003] To this end, driving assistance systems have emerged. They use a variety of sensors installed on the car to sense the surrounding environment at any time while the car is driving, collect data, identify, detect and track static and dynamic objects, and combine navigation map data to perform systematic calculations and analysis, so that the driver can be aware of possible dangers in advance and increase the comfort and safety of car driving.

[0004] However, such driving assistance systems are usually used while driving on the road and require the driver to actively turn them on. For parking or moving the car, if the driver chooses manual operation, they often only provide external images or radar warnings, and cannot intervene in the first time when the driver makes an operational error.

[0005] Therefore, a vision-based driving assistance system and method are needed that can ensure the safety of the vehicle and the driver in parking or maneuvering scenarios. Summary of the Invention

[0006] One of the objectives of the present invention is to provide a driving assistance system based on visual recognition, which can ensure the safety of the vehicle and the driver in parking or maneuvering scenarios and reduce the probability of accidents.

[0007] In order to solve the above technical problems, this application provides the following technical solutions:

[0008] Driver assistance systems based on visual recognition include:

[0009] A first image acquisition module, used for acquiring image data outside the vehicle;

[0010] The driving control module is used to obtain the vehicle's current driving data and accelerator pedal force data. The driving data includes steering angle data, gear data and vehicle speed data;

[0011] The auxiliary module is used to determine the external environment based on the image data outside the vehicle. It is also used to determine the current state of the vehicle based on the driving data. If the vehicle is in the state of moving or parking, it estimates the vehicle's driving trajectory based on the driving data. It is also used to determine whether there are obstacles on the driving trajectory based on the vehicle's driving trajectory and the external environment. If there are obstacles, it calculates the distance between the vehicle and the obstacle.

[0012] It is also used to calculate the estimated force of the accelerator pedal based on the preset warning value and the distance between the vehicle and the obstacle, and to determine whether the throttle force data exceeds the estimated force. If not, the accelerator pedal instruction is executed; if exceeded, an alarm message is generated and the accelerator pedal instruction is not executed.

[0013] The basic scheme principles and beneficial effects are as follows:

[0014] In this solution, after the vehicle is started, image data of the parking space is collected. After visual recognition-related analysis, the environment outside the vehicle can be determined. Based on the current driving data, it can be determined whether the vehicle is in the state of moving or parking. If so, the vehicle's driving trajectory is estimated to determine whether there are any obstacles on the driving trajectory. In the case of obstacles, if the driver has problems controlling the accelerator pedal, it is easy to cause an accident. For example, the force of controlling the accelerator pedal is too large, causing the vehicle to accelerate suddenly, or the accelerator pedal is used as a brake pedal. This solution pre-sets a warning value, which is the value of the reaction time. The estimated force of the accelerator pedal is calculated based on the preset warning value and the distance between the vehicle and the obstacle to ensure that the driver has time to react before the vehicle contacts the obstacle. If the throttle force data is determined to exceed the estimated force, an alarm message is generated and the accelerator pedal instruction is not executed to avoid the driver's untimely reaction, which causes the vehicle to collide with the obstacle.

[0015] In summary, this solution ensures vehicle and driver safety during parking or maneuvering scenarios, reducing the probability of accidents. This solution performs pre-analysis before the driver applies pressure to the accelerator pedal. When the driver applies pressure, the throttle force data is compared with the estimated force. This reduces computational effort, reduces system latency, and provides a better driving experience. Furthermore, this solution does not modify the accelerator pedal calibration, preventing it from affecting the driver's muscle memory.

[0016] Furthermore, the auxiliary module is used to identify the external environment of the vehicle including the driving area and the non-driving area based on the image data outside the vehicle, and to determine whether the vehicle's driving trajectory passes through the non-driving area. If it passes through the non-driving area, it is determined that there is an obstacle on the driving trajectory.

[0017] In this priority scheme, the definition of obstacles is broader. For example, non-driving areas with height differences are also considered obstacles. This can prevent vehicles from entering non-driving areas with height differences, providing higher safety.

[0018] Furthermore, the auxiliary module is further configured to obtain historical driving data and accelerator pedal force data, classify the driver's driving style according to the historical driving data and accelerator pedal force data, and select different alert values ​​according to the classification.

[0019] For example, for drivers with aggressive acceleration styles, a shorter warning value is used to achieve personalized settings for different drivers and different driving styles.

[0020] Furthermore, it also includes a second image acquisition module for acquiring image data inside the vehicle;

[0021] The auxiliary module is also used to identify the driver based on the image data inside the vehicle and determine the driver's viewing direction; it is also used to determine the direction of obstacles outside the vehicle and judge whether the driver's viewing direction is consistent with the direction of obstacles outside the vehicle. If they are inconsistent, the current warning value is maintained; if they are consistent, the current warning value is increased.

[0022] When the driver's viewing direction is consistent with the direction of an obstacle outside the vehicle, the driver knows that there is a high possibility of an obstacle, increases the current warning value, and puts more control of the vehicle in the hands of the driver, reducing the possibility of interference to the driver.

[0023] Furthermore, the auxiliary module is also used to estimate the driving direction of the vehicle based on the driving area and the non-driving area, and determine whether the gear is consistent with the driving direction based on the driving direction and gear data. If not, a reminder message is generated.

[0024] It can remind the driver when he / she shifts into the wrong gear.

[0025] Furthermore, it also includes a positioning module for obtaining satellite positioning data of the vehicle;

[0026] The auxiliary module is also used to record the vehicle's satellite positioning data, and determine whether the current location is a common parking location based on the vehicle's historical satellite positioning data and current satellite positioning data. If it is a common parking location, the current warning value is increased.

[0027] In the driver's usual parking location, they are usually more aware of the surrounding situation and are familiar with the vehicle's entry and exit routes, which increases the current alert value and reduces the possibility of interference to the driver.

[0028] Furthermore, the driving data also includes vehicle inclination data;

[0029] The auxiliary module is also used to calculate the estimated force of the accelerator pedal based on the preset warning value, the distance between the vehicle and the obstacle, the vehicle speed data and the vehicle inclination data.

[0030] Avoid situations where the vehicle does not move when the accelerator pedal is pressed in scenarios such as on a slope.

[0031] A second object of the present invention is to provide a driving assistance method based on visual recognition, comprising the following contents:

[0032] S1, collecting image data outside the vehicle;

[0033] S2. Acquire the vehicle's current driving data and accelerator pedal force data, where the driving data includes steering angle data, gear position data, and vehicle speed data;

[0034] S3, determining the environment outside the vehicle based on the image data outside the vehicle; the identified environment outside the vehicle includes a driving area and a non-driving area;

[0035] S4. Determine the current state of the vehicle based on the driving data; determine whether the vehicle is in the state of moving or parking; if not, jump to S4; if yes, jump to S5;

[0036] S5. Estimate the vehicle's driving trajectory based on the driving data; determine whether there are any obstacles on the driving trajectory based on the vehicle's driving trajectory and the environment outside the vehicle. If there are obstacles, jump to S6; if there are no obstacles, jump to S5;

[0037] S6, calculating the distance between the vehicle and the obstacle; calculating the estimated accelerator pedal force based on the preset warning value and the distance between the vehicle and the obstacle, and determining whether the accelerator force data exceeds the estimated force. If not, jump to step S7; if so, jump to step S8;

[0038] S7, executing the accelerator pedal command;

[0039] S8. Generate an alarm message and do not execute the accelerator pedal command. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a logic block diagram of a driving assistance system based on visual recognition in Example 1. DETAILED DESCRIPTION

[0041] The following is further described in detail through specific implementation methods:

[0042] Example 1

[0043] like Figure 1 As shown, the visual recognition-based driving assistance system of this embodiment includes a first image acquisition module, a driving control module and an auxiliary module.

[0044] The first image acquisition module is used to acquire image data outside the vehicle. In this embodiment, the first image acquisition module uses a camera arranged outside the vehicle.

[0045] The driving control module is used to obtain the vehicle's current driving data and accelerator pedal force data. Driving data includes steering angle data, gear position data, and vehicle speed data. Specifically, the driving control module obtains vehicle speed data from the vehicle's speed sensor, gear position data from the transmission, steering angle data from the steering mechanism, and inclination data from the gyroscope.

[0046] The auxiliary modules include an image analysis unit, a path planning unit, and a data analysis unit;

[0047] The image analysis unit is used to determine the external environment based on the image data outside the vehicle. In this embodiment, the image analysis unit uses a pre-trained neural network model to recognize the external image data. The recognized external environment includes driving areas and non-driving areas. In this embodiment, road surfaces with depressions, protrusions, objects placed on them, other vehicles, and pedestrians are considered non-driving areas.

[0048] The data analysis unit is used to determine the current state of the vehicle based on the driving data. If the vehicle is in the state of maneuvering or parking, the path planning unit is used to estimate the vehicle's driving trajectory based on the driving data. The data analysis unit is also used to determine whether there are obstacles along the driving trajectory based on the vehicle's driving trajectory and the external environment, and if so, calculate the distance between the vehicle and the obstacle. In this embodiment, the data analysis unit determines whether the vehicle's driving trajectory passes through a non-driving area. If so, it determines that there is an obstacle along the driving trajectory. In this embodiment, if the vehicle speed is below a threshold and the gear is in reverse, it is considered parking; if the vehicle is started, the speed is zero, and the vehicle is engaged in forward or reverse gear, it is considered maneuvering.

[0049] The data analysis unit is further configured to calculate an estimated accelerator pedal force based on a preset warning value and the distance between the vehicle and the obstacle, and to determine whether the accelerator pedal force data exceeds the estimated force. If not, the accelerator pedal command is executed; if so, an alarm is generated and the accelerator pedal command is not executed. In this embodiment, the preset warning value is time. Given the distance and time, the average speed can be calculated. The increase in speed required to achieve the increase in accelerator pedal force is lower than the average speed.

[0050] Based on the above system, this embodiment also provides a driving assistance method based on visual recognition, including the following contents:

[0051] S1, collecting image data outside the vehicle;

[0052] S2. Obtaining the vehicle's current driving data and accelerator pedal force data. The driving data includes steering angle data, gear position data, and vehicle speed data. Specifically, the driving control module obtains vehicle speed data from the vehicle's speed sensor, gear position data from the transmission, steering angle data from the steering mechanism, and inclination data from the gyroscope.

[0053] S3. Determine the environment outside the vehicle based on the image data outside the vehicle; the identified environment outside the vehicle includes a driving area and a non-driving area. In this embodiment, a road surface with depressions, protrusions, objects, other vehicles, and pedestrians is considered a non-driving area.

[0054] S4. Determine the current state of the vehicle based on the driving data; determine whether the vehicle is in the state of moving or parking; if not, jump to S4; if yes, jump to S5;

[0055] S5. Estimate the vehicle's driving trajectory based on the driving data; determine whether there are any obstacles on the driving trajectory based on the vehicle's driving trajectory and the environment outside the vehicle. If there are obstacles, jump to S6; if there are no obstacles, jump to S5. In this embodiment, determine whether the vehicle's driving trajectory passes through a non-driving area. If so, determine that there are obstacles on the driving trajectory.

[0056] S6. Calculate the distance between the vehicle and the obstacle. Calculate the estimated accelerator pedal force based on the preset warning value and the distance between the vehicle and the obstacle. Determine whether the accelerator pedal force exceeds the estimated force. If not, skip to step S7. If so, skip to step S8.

[0057] S7, executing the accelerator pedal command;

[0058] S8. Generate an alarm message and do not execute the accelerator pedal command.

[0059] Example 2

[0060] The difference between this embodiment and the real-time example 1 is that the system of this embodiment further includes a second image acquisition module.

[0061] The second image acquisition module is used to acquire image data inside the vehicle. In this embodiment, the second image acquisition module adopts a camera arranged inside the vehicle.

[0062] The data analysis unit is further configured to obtain historical driving data and accelerator pedal force data, classify the driver's driving style based on the historical driving data and accelerator pedal force data, and select different alert values ​​based on the classification. In this embodiment, the driver's driving style is categorized as cautious, normal, and aggressive, and the corresponding alert values ​​decrease gradually. In other words, the cautious driving style has the longest duration of alert value.

[0063] The image analysis unit is also used to identify the driver based on the image data inside the vehicle and determine the driver's viewing direction; the data analysis unit is also used to determine the direction of obstacles outside the vehicle and judge whether the driver's viewing direction is consistent with the direction of obstacles outside the vehicle. If they are inconsistent, the current alert value is maintained; if they are consistent, the current alert value is increased.

[0064] The path planning unit is also used to estimate the vehicle's driving direction based on the driving area and non-driving area, and to determine whether the gear is consistent with the driving direction based on the driving direction and gear data. If not, a reminder message is generated.

[0065] Example 3

[0066] The difference between this embodiment and the second embodiment is that the system of this embodiment further includes a positioning module, which is used to obtain satellite positioning data of the vehicle.

[0067] The data analysis unit is further configured to record the vehicle's satellite positioning data and determine whether the current location is a frequently used parking location based on the vehicle's historical and current satellite positioning data. If so, the current alert value is increased. In this embodiment, a frequently used parking location refers to a location where the vehicle has been moved or parked more than five times.

[0068] The driving data also includes inclination data. The data analysis unit is further configured to calculate an estimated accelerator pedal force based on a preset warning value, the distance between the vehicle and the obstacle, the vehicle speed data, and the vehicle inclination data.

[0069] The above are only embodiments of the present invention. The invention is not limited to the fields involved in this implementation case. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field are aware of all common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the inspiration given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several variations and improvements can be made, which should also be regarded as the scope of protection of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. A driving assistance system based on visual recognition, characterized in that: include: A first image acquisition module, used for acquiring image data outside the vehicle; The driving control module is used to obtain the vehicle's current driving data and accelerator pedal force data. The driving data includes steering angle data, gear data and vehicle speed data; The auxiliary module is used to determine the external environment based on the image data outside the vehicle. It is also used to determine the current state of the vehicle based on the driving data. If the vehicle is in the state of moving or parking, it estimates the vehicle's driving trajectory based on the driving data. It is also used to determine whether there are obstacles on the driving trajectory based on the vehicle's driving trajectory and the external environment. If there are obstacles, it calculates the distance between the vehicle and the obstacle. It is also used to calculate the estimated force of the accelerator pedal based on the preset warning value and the distance between the vehicle and the obstacle, and determine whether the throttle force data exceeds the estimated force. If it does not exceed, the accelerator pedal instruction is executed; if it exceeds, an alarm message is generated and the accelerator pedal instruction is not executed; The auxiliary module is also used to obtain historical driving data and accelerator pedal force data, classify the driver's driving style based on the historical driving data and accelerator pedal force data, and select different warning values ​​according to the classification; The driver's driving style is divided into cautious, normal and aggressive, and the corresponding alert values ​​decrease gradually; The auxiliary module is used to identify the vehicle's external environment, including a driving area and a non-driving area, based on the image data outside the vehicle, and determine whether the vehicle's driving trajectory passes through the non-driving area. If the vehicle passes through the non-driving area, it is determined that there is an obstacle on the driving trajectory; Also included is a second image acquisition module for acquiring image data inside the vehicle; The auxiliary module is also used to identify the driver based on the image data inside the vehicle and determine the driver's viewing direction; it is also used to determine the direction of obstacles outside the vehicle and judge whether the driver's viewing direction is consistent with the direction of the obstacles outside the vehicle. If they are inconsistent, the current alert value is maintained; if they are consistent, the current alert value is increased; The auxiliary module is further configured to estimate the vehicle's driving direction based on the driving area and the non-driving area, determine whether the gear position is consistent with the driving direction based on the driving direction and the gear position data, and generate a warning message if they are inconsistent; It also includes a positioning module for obtaining satellite positioning data of the vehicle; The auxiliary module is also used to record the vehicle's satellite positioning data, and determine whether the current location is a common parking location based on the vehicle's historical satellite positioning data and current satellite positioning data. If it is a common parking location, the current warning value is increased.

2. The visual recognition-based driving assistance system according to claim 1, characterized in that: The driving data also includes vehicle inclination data; The auxiliary module is also used to calculate the estimated force of the accelerator pedal based on the preset warning value, the distance between the vehicle and the obstacle, the vehicle speed data and the vehicle inclination data.

3. A driving assistance method based on visual recognition, using the system according to any one of claims 1 to 2, characterized in that: Includes the following: S1, collecting image data outside the vehicle; S2. Acquire the vehicle's current driving data and accelerator pedal force data, where the driving data includes steering angle data, gear position data, and vehicle speed data; S3, determining the environment outside the vehicle based on the image data outside the vehicle; the identified environment outside the vehicle includes a driving area and a non-driving area; S4. Determine the current state of the vehicle based on the driving data; determine whether the vehicle is in the state of moving or parking; if not, jump to S4; if yes, jump to S5; S5. estimating the vehicle's driving trajectory based on the driving data; According to the vehicle's driving trajectory and the environment outside the vehicle, determine whether there are obstacles on the driving trajectory. If there are obstacles, jump to S6; if there are no obstacles, jump to S5; S6, calculating the distance between the vehicle and the obstacle; calculating the estimated accelerator pedal force based on the preset warning value and the distance between the vehicle and the obstacle, and determining whether the accelerator force data exceeds the estimated force. If not, jump to step S7; if so, jump to step S8; S7, executing the accelerator pedal command; S8. Generate an alarm message and do not execute the accelerator pedal command.

Images