Blind area obstacle early warning method and system, vehicle and storage medium

Abstract

The application relates to the technical field of vehicles, in particular to a blind area obstacle early warning method and system, a vehicle and a storage medium, wherein the method comprises the following steps: monitoring a blind area image around the vehicle; identifying whether an obstacle exists in the blind area image; if the obstacle exists, early warning is given to a driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle. Thus, the problems that the obstacle monitoring accuracy of the related art is low and the driving safety is low are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicles, in particular to a blind area obstacle early warning method and system, a vehicle and a storage medium. BACKGROUND

[0002] Due to the limitation of the structure of the vehicle, there is a problem of a side rear visual blind area. Especially when the driver is driving at a high speed, the line of sight is mainly focused on the front. The driving track of the side rear vehicle is usually observed by relying on the rearview mirror or the honk of the rear vehicle to attract attention. However, due to the limited field of view of the rearview mirror, the field of view blind area still exists, and especially when other factors block the rear vehicle or the driver fails to concentrate on observing the side rear, potential safety hazards may be caused.

[0003] In the related art, radar, camera, ultrasonic technology and the like are mainly used for blind area detection to reduce driving safety problems caused by blind area problems. However, in the related art, a single means is mainly used for blind area detection, resulting in low accuracy of blind area obstacle monitoring. SUMMARY

[0004] The present application provides a blind area obstacle early warning method, system, vehicle and storage medium to solve the problem of low accuracy of blind area obstacle monitoring in the related art, resulting in low driving safety.

[0005] The first aspect of the present application provides a blind area obstacle early warning method, comprising the following steps: monitoring a blind area image around the vehicle; identifying whether there is an obstacle in the blind area image; if there is an obstacle, warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle.

[0006] Optionally, in an embodiment of the present application, warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle comprises: determining a warning level based on the distance between the obstacle and the vehicle; and warning the driver of the vehicle and / or the obstacle based on the warning level.

[0007] Optionally, in an embodiment of the present application, determining a warning level based on the distance between the obstacle and the vehicle comprises: if the distance is greater than a first preset value and less than or equal to a second preset value, the first warning level is determined; if the distance is greater than a third preset value and less than or equal to the first preset value, the second warning level is determined; and if the distance is less than or equal to the third preset value, the third warning level is determined, wherein the third preset value is less than the first preset value, and the first preset value is less than the second preset value.

[0008] Optionally, in an embodiment of the present application, the pre-warning mode comprises at least one of displaying an obstacle image, a text and / or graphical reminder, a voice reminder, a sound reminder, a light reminder, and a steering wheel vibration reminder, and the driver of the vehicle and the obstacle are pre-warned based on the pre-warning level, including: if it is the first pre-warning level, displaying an obstacle image to the driver and performing a voice reminder and a light reminder, and performing a text and / or graphical reminder to the obstacle; if it is the second pre-warning level or the third pre-warning level, displaying an obstacle image to the driver and performing a voice reminder, a light reminder, and a steering wheel vibration reminder, and performing a text and / or graphical reminder, a sound reminder, and a light reminder to the obstacle.

[0009] Optionally, in an embodiment of the present application, after pre-warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle, if it is the third pre-warning level, the driving intention of the driver is identified, and the steering torque of the vehicle is adjusted based on the driving intention to adjust the motion trajectory of the vehicle to avoid the obstacle.

[0010] Optionally, in an embodiment of the present application, the driving intention comprises lane keeping, left lane changing, right lane changing, and emergency avoidance, and the steering torque of the vehicle is adjusted based on the driving intention, including: if the driving intention is lane keeping, the change rate of the steering torque is adjusted; if the driving intention is left lane changing or right lane changing, the steering torque is linearly increased to a saturation state; and if the driving intention is emergency avoidance, the steering torque is directly adjusted to a saturation state.

[0011] Optionally, in an embodiment of the present application, the calculation formula of the steering torque is:

[0012] τ guide = f (Δθ) ;

[0013] wherein τ guide is the steering torque guide, Δθ is the difference between the real steering wheel rotation angle θ real and the target steering wheel rotation angle θ target , Δθ = θ real - θ target , K is a peak value coefficient of the guide torque, a is a linear difference curvature of the rotation angle, and b is a linear compensation of the rotation angle difference.

[0014] The second aspect embodiment of the present application provides a blind area obstacle early warning system, comprising: a monitoring subsystem, configured to monitor a blind area around the vehicle and acquire a distance between an obstacle and the vehicle; an information display subsystem, configured to display an obstacle image in a blind area image; an image display subsystem, configured to remind the obstacle by text and / or graphics; a steering wheel control subsystem, configured to remind a driver of the vehicle by voice, light and steering wheel vibration; an audio and light feedback subsystem, configured to remind the obstacle by sound and light; a steering torque guiding subsystem, configured to identify a driving intention of the driver and adjust a steering torque of the vehicle based on the driving intention; and a controller, configured to control the monitoring subsystem to monitor, control at least one of the distance control information subsystem, the image display subsystem, the steering wheel control subsystem and the audio and light feedback subsystem to perform early warning, and control the steering torque guiding subsystem to adjust the steering torque of the vehicle.

[0015] The third aspect embodiment of the present application provides a vehicle comprising the blind area obstacle early warning system in the above embodiments.

[0016] The fourth aspect embodiment of the present application provides a computer readable storage medium, which stores a computer program or instructions, and the computer program or instructions are executed by a processor to perform the blind area obstacle early warning method in the above embodiments.

[0017] Therefore, the present application has at least the following beneficial effects:

[0018] The embodiments of the present application can monitor a blind area image around the vehicle, and when an obstacle is identified in the blind area image, the driver and the obstacle are warned based on the distance between the vehicle and the obstacle, the accuracy of obstacle monitoring is improved, the safety of driving is improved, and the potential danger caused by the blind area is effectively reduced. Therefore, the technical problems of low accuracy of obstacle detection in the blind area in the related art and low driving safety are solved.

[0019] Additional aspects and advantages of the present application will be in part apparent and in part pointed out hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and be readily appreciated from the following description, including the accompanying drawings, in which:

[0021] Figure 1 A schematic diagram for classification of blind areas of a vehicle;

[0022] Figure 2 A schematic diagram for blind areas of a vehicle;

[0023] Figure 3A flowchart of a blind area obstacle early warning method according to an embodiment of the present application is provided.

[0024] Figure 4 A schematic diagram of a blind area obstacle early warning system according to an embodiment of the present application is provided.

[0025] Figure 5 A display diagram of vehicle-mounted hardware and visual range according to an embodiment of the present application is provided.

[0026] Figure 6 A flowchart of selection of a main target obstacle according to an embodiment of the present application is provided.

[0027] Figure 7 A flowchart of a working process of a blind area obstacle early warning system according to an embodiment of the present application is provided.

[0028] Figure 8 A flowchart of a process performed based on an early warning level system according to an embodiment of the present application is provided.

[0029] Figure 9 A flowchart of image preprocessing according to an embodiment of the present application is provided.

[0030] Figure 10 A schematic diagram of road head type custom emoticons and text according to an embodiment of the present application is provided.

[0031] Figure 11 A module composition diagram of an information display subsystem according to an embodiment of the present application is provided.

[0032] Figure 12 A running schematic diagram of an information display subsystem according to an embodiment of the present application is provided. DETAILED DESCRIPTION

[0033] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by reference to the accompanying drawings are exemplary and are intended to explain the present application, and cannot be understood as limiting the present application.

[0034] Before describing the scheme of the present application, the blind area of a vehicle is introduced first, so as to understand the scheme of the present application subsequently.

[0035] Due to the limitations of the structure of the vehicle itself, there is a problem of a blind area in the side rear view. Especially when the driver is driving at a high speed, the line of sight is mainly focused on the front. The observation of the driving track of the side rear vehicle usually depends on the rearview mirror or the honk of the rear vehicle to attract attention. However, due to the limited field of view of the rearview mirror, the field of view blind area still exists, especially when other factors block the rear vehicle or the driver fails to concentrate on observing the side rear, which may cause potential safety hazards, as shown in the blind area classification Figure 1 With the increase of the popularity of cars, investigations show that the side rear collision caused by the field of view blind area of the left and right rearview mirrors accounts for 30.5% of the total traffic accidents, as shown in the blind area display Figure 2 The side rear collision accident is usually caused by the failure of the driver to realize the overtaking of the rear vehicle and to change lanes, thereby causing the collision. Therefore, in order to ensure driving safety, it is not only necessary to widen the side rear field of view by using safety auxiliary equipment to avoid the generation of the visual blind area, but also necessary to rely on the equipment to provide the driver with road condition reminders or even danger warning prompts. Studies have shown that if the driver can react to the dangerous situation 1-2 seconds in advance, it is possible to avoid 90% of traffic accidents.

[0036] The blind area obstacle warning method, system, vehicle and storage medium of the embodiments of the present application are described below with reference to the accompanying drawings. In view of the problem that the current blind area detection mainly adopts a single means, resulting in low accuracy of blind area obstacle monitoring, the present application provides a blind area obstacle warning method. In the method, the blind area image around the vehicle can be monitored, and when an obstacle is identified in the blind area image, the driver and the obstacle can be warned based on the distance between the vehicle and the obstacle, thereby improving the accuracy of obstacle monitoring. Thus, the problems of low accuracy of blind area obstacle monitoring and low driving safety in the related art are solved.

[0037] Specifically, Figure 3 A flowchart of a blind area obstacle warning method provided by an embodiment of the present application is shown.

[0038] As shown in Figure 3 , the blind area obstacle warning method comprises the following steps:

[0039] In step S101, the blind area image around the vehicle is monitored.

[0040] It should be noted that in order to monitor the blind area image around the vehicle, a laser radar can be installed on the vehicle. Specifically, one RS-Ruby laser radar can be provided on the roof of the vehicle, and four blind fill laser radars can be provided on the body of the vehicle. As can be seen from the figure, the maximum visual range is achieved, and the field of view blind area around the vehicle is avoided.

[0041] In step S102, it is identified whether there is an obstacle in the blind area image.

[0042] The obstacle includes a pedestrian, a vehicle, an object, etc.

[0043] It can be understood that the embodiment of the present application can identify whether there is an obstacle in the blind area image, so as to subsequently perform early warning.

[0044] In step S103, if there is an obstacle, the driver of the vehicle and / or the obstacle are warned based on the distance between the obstacle and the vehicle.

[0045] It can be understood that the embodiment of the present application can warn the driver of the vehicle and the obstacle based on the distance between the obstacle and the vehicle when there is an obstacle in the blind area image, so as to ensure driving safety and effectively reduce potential dangers caused by the blind area.

[0046] In the embodiment of the present application, warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle includes determining a warning level based on the distance between the obstacle and the vehicle, and warning the driver of the vehicle and / or the obstacle based on the warning level.

[0047] It can be understood that the embodiment of the present application can determine a warning level based on the distance between the obstacle and the vehicle, and warn the driver of the vehicle and the obstacle based on the warning level.

[0048] In the embodiment of the present application, determining a warning level based on the distance between the obstacle and the vehicle includes: if the distance is greater than a first preset value and less than or equal to a second preset value, the first warning level is determined; if the distance is greater than a third preset value and less than or equal to the first preset value, the second warning level is determined; and if the distance is less than or equal to the third preset value, the third warning level is determined, wherein the third preset value is less than the first preset value, and the first preset value is less than the second preset value.

[0049] The first preset value, the second preset value, and the third preset value can be set according to specific conditions, and are not limited specifically, the third preset value is less than the first preset value, and the first preset value is less than the second preset value, for example, the first preset value can be set to 60 cm, the second preset value can be set to 100 cm, and the third preset value can be set to 30 cm.

[0050] It can be understood that the embodiment of the present application can determine a warning level according to the distance, for example, taking the first preset value of 60 cm, the second preset value of 100 cm, and the third preset value of 30 cm as an example,

[0051] If the distance is greater than 60 cm and less than or equal to 100 cm, the first warning level is determined;

[0052] If the distance is greater than or equal to 30 cm and less than or equal to 60 cm, the second warning level is determined;

[0053] If the distance is less than 30 cm, the third warning level is determined.

[0054] In the embodiments of the present application, the warning mode includes at least one of displaying an obstacle image, text and / or graphical reminder, voice reminder, sound reminder, light reminder, steering wheel vibration reminder.

[0055] The text and image can be set in advance according to the user's situation and displayed when the warning occurs. The voice reminder or sound reminder can also be a user-set or original system default prompt to provide a more intelligent and emotional interactive experience while ensuring the safety of the driver and the pedestrian.

[0056] In the embodiments of the present application, the driver and / or obstacle of the vehicle are warned based on the warning level, including: if the first warning level is determined, the driver is displayed with an obstacle image and reminded with voice and light; the obstacle is reminded with text and / or graphics.

[0057] It can be understood that the embodiments of the present application can determine how to remind the driver and the obstacle according to the different warning levels, as follows:

[0058] If the first warning level is determined, the driver is displayed with an obstacle image and reminded with voice and light, such as displaying the obstacle image on the HUD (Head-Up Display, head-up display system), the voice reminder can be "please pay attention to the blind area, and the first level warning will be started", and the light strip in the vehicle can be made to flash, and the obstacle can be reminded with text and graphics, which can be projected on the obstacle position.

[0059] If the second or third warning level is determined, the driver is displayed with an obstacle image and reminded with voice, light and steering wheel vibration, and the obstacle is reminded with text, graphics, sound and light, such as the sound reminder can be a vehicle alarm, and the lights on both sides or the rear of the vehicle can flash.

[0060] It should be noted that the embodiments of the present application can also determine the warning mode of the obstacle according to the type of the obstacle, such as text and / or graphical reminder can only be for pedestrians or pedestrians riding bicycles, electric vehicles and other vehicles, sound and light reminders can be for pedestrians, pedestrians riding bicycles, electric vehicles and other vehicles, other small animals, or other vehicles in progress, and for other fixed obstacles such as pillars, fences and other fixed objects, no warning is needed.

[0061] In the embodiment of the present application, after the driver of the vehicle and / or the obstacle are warned based on the distance between the obstacle and the vehicle, if it is the third warning level, the driving intention of the driver is identified, and the steering torque of the vehicle is adjusted based on the driving intention to adjust the motion trajectory of the vehicle to avoid the obstacle.

[0062] The driving intention includes lane keeping, left lane changing, right lane changing, and emergency avoidance.

[0063] It can be understood that in the third warning level of the embodiment of the present application, that is, in a more dangerous situation, the driving intention of the driver can be identified, and the steering torque of the vehicle is adjusted based on the driving intention to adjust the motion trajectory of the vehicle to achieve the purpose of safely avoiding the obstacle, as follows.

[0064] In the embodiment of the present application, the steering torque of the vehicle is adjusted based on the driving intention, including: if the driving intention is lane keeping, the change rate of the steering torque is adjusted; if the driving intention is left lane changing or right lane changing, the steering torque is linearly increased to a saturation state; and if the driving intention is emergency avoidance, the steering torque is directly adjusted to a saturation state.

[0065] The calculation formula of the steering torque is:

[0066] τ guide =f(Δθ);

[0067] Wherein, τ guide is the steering torque guide, Δθ is the difference between the actual steering wheel angle θ real and the target steering wheel angle θ target , Δθ=θ real -θ target , K is the peak value coefficient of the guide torque, a is the linear difference curvature of the angle, and b is the linear compensation of the angle difference.

[0068] It can be understood that in the embodiment of the present application, when the driving intention is lane keeping, the change rate of the steering torque can be adjusted, when the driving intention is left lane changing or right lane changing, the steering torque is linearly increased to a saturation state, and when the driving intention is emergency avoidance, the steering torque is directly adjusted to a saturation state.

[0069] According to the blind area obstacle warning method provided in the embodiment of the present application, the blind area image around the vehicle can be monitored, and when it is identified that there is an obstacle in the blind area image, the driver and the obstacle are warned based on the distance between the vehicle and the obstacle, the accuracy of obstacle monitoring is improved, the safety of driving is improved, and the potential danger caused by the blind area is effectively reduced.

[0070] Secondly, the blind area obstacle early warning system according to the embodiment of the application is described with reference to the accompanying drawings.

[0071] Figure 4 is a schematic diagram of the blind area obstacle early warning system according to the embodiment of the application.

[0072] As Figure 4 shown, the blind area obstacle early warning system 10 comprises a monitoring subsystem 11, an information display subsystem 12, an image display subsystem 13, a steering wheel control subsystem 14, an audible and light feedback subsystem 15, a steering torque guiding subsystem 16 and a controller 17.

[0073] The monitoring subsystem 11 is configured to monitor the blind area around the vehicle and obtain the distance between the obstacle and the vehicle; the information display subsystem 12 is configured to display the obstacle image in the blind area image; the image display subsystem 13 is configured to provide a text and / or graphical reminder for the obstacle; the steering wheel control subsystem 14 is configured to provide a voice reminder, a light reminder and a steering wheel vibration reminder for the driver of the vehicle; the audible and light feedback subsystem 15 is configured to provide a sound reminder and a light reminder for the obstacle; the steering torque guiding subsystem 16 is configured to identify the driving intention of the driver and adjust the steering torque of the vehicle based on the driving intention; and the controller 17 is configured to control the monitoring subsystem 11 to perform monitoring, control at least one of the information display subsystem 12, the image display subsystem 13, the steering wheel control subsystem 14 and the audible and light feedback subsystem 15 to perform early warning based on the distance, and control the steering torque guiding subsystem 16 to adjust the steering torque of the vehicle.

[0074] It can be understood that the blind area obstacle early warning system 10 according to the embodiment of the application comprises a plurality of subsystems for performing different functions, and the controller is configured to control the plurality of subsystems to work, thereby realizing early warning for the blind area obstacle and improving the safety of driving.

[0075] Specifically, the blind area obstacle early warning system according to the embodiment of the application comprises:

[0076] 1. A vehicle blind area detection subsystem (equivalent to the monitoring subsystem described above)

[0077] Firstly, a main laser radar (RS-Ruby) and four 20Hz short-range blind-filling laser radars (RS-Bpearl) are added to collect the speed and acceleration of the vehicle, and the data is transmitted to the central control data processor through a wireless communication device to analyze the safety distance between the current vehicle and the pedestrian in the blind area of the vehicle, upload the analyzed target pedestrian speed and safety distance to the safety route early warning module, and plan the route according to the calculation result.

[0078] Vehicle is equipped with a car angle sensor, keep the vehicle in emergency steering stability in the event of sudden, while monitoring the vehicle and pedestrian eye angle to ensure that in different angles, pedestrians can see the image display.

[0079] Vehicle is equipped with a temperature sensor, using infrared technology to monitor whether there is a biological outside the car, and determine whether in the blind area, if the detection of surface temperature alarm.

[0080] Vehicle is equipped with a visibility sensor, in the event of heavy fog, thunderstorm weather, detect the visibility of the environment outside the car and adjust the brightness and color of the HUD display inside the car and the display brightness outside the car according to the brightness.

[0081] 2、AR-HUD(Augmented Reality Head Up Display, front window augmented reality head-up display) information display subsystem

[0082] AR-HUD is a kind of visual auxiliary equipment of vehicle, using augmented reality technology to superimpose information such as Bluetooth phone, ADAS(Advanced Driver Assistance System, advanced driver assistance system), navigation state and surrounding environment on the auxiliary equipment in the driver's field of view, A column blind area is the main reason for 80% of the accidents now, by installing two cameras on the A column of the vehicle, the driver's face is captured, and the face orientation and line of sight height are analyzed, the blind area is identified and the blind area picture is displayed on the windshield in real time, so that the driver can perceive the blind area obstacles in advance and feedback.

[0083] 3、image display subsystem

[0084] When the blind area vehicle or pedestrian reaches the collision warning condition, the driver can customize the display image or text content, and project the information to the vehicle blind area to remind the pedestrians outside the vehicle to avoid, the graphics can be changed in real time according to the vehicle and pedestrian motion trajectory, and the animation effect and display color can be displayed according to the individual customization.

[0085] 4、steering wheel control subsystem

[0086] The steering wheel is more intelligent, integrating voice, light strip flicker, vibration reminder and other functions. By installing steering wheel rotation sensor and combining with front wheel deflection angle, the driver's driving intention is analyzed, when the system detects that the vehicle and the blind area pedestrian and vehicle are in the alarm distance, the voice will remind "please pay attention to the A column blind area environment, slow down" according to the blind area position, the light strip is divided into left and right layout, according to the driver's intention to remind left turn or right turn, adjust the flicker frequency according to the alarm level, and then the alarm steering wheel vibration intensity also changes with the light strip.

[0087] 5, Acousto-optic feedback subsystem

[0088] The vehicle is provided with a voice buzzer and a matrix lamp on both sides of the driver and front passenger, which performs sound and light two-way early warning and behavior guidance when detecting a vehicle and a pedestrian in the blind area. The sound and frequency are also adaptively adjusted to match the rearview mirror frequency flash lamp. Compared with the traditional alarm device, the photoelectric form is faster, the feature is more obvious, and the guidance and early warning are stronger.

[0089] 6, Steering torque guidance subsystem

[0090] The steering wheel angle sensor monitors the vehicle speed, heading angle, and body tilt angle data at all times, actively adjusts the steering angle and the front wheel angle ratio, i.e., the size of the steering force guidance, according to the driver's driving intention and safety distance. When the driver's intention is to turn left or right, the corresponding torque is linearly increased to gradually reach a saturated state. When the driver's intention is to avoid in an emergency, the torque directly reaches the saturation stage, which cooperates with the heading angle and the body tilt angle to ensure the stability of the vehicle body, and changes the vehicle trajectory according to the size of the torque to achieve the purpose of safe obstacle avoidance.

[0091] It should be noted that the foregoing explanation and description of the embodiment of the blind area obstacle warning method also applies to the blind area obstacle warning system of this embodiment, which will not be described here.

[0092] The blind area obstacle warning system according to the embodiments of the present application can monitor the blind area image around the vehicle and control multiple subsystems of the blind area obstacle warning system to work, and use multiple ways to warn obstacles, thereby improving the safety of driving and effectively reducing potential dangers caused by blind areas.

[0093] The implementation process of the blind area obstacle warning according to the embodiments of the present application will be described below through a specific embodiment, and the specific implementation process will be described in conjunction with the blind area obstacle warning system.

[0094] First, the vehicle-mounted hardware and visual range display diagram involved in the implementation of the blind area obstacle warning method according to the present application are described, as shown in Figure 5

[0095] At present, due to the limitations of the vertical field of view (FOV, hereinafter referred to as FOV) monitoring range of the laser radar and the top-mounted installation method, the vertical FOV has a viewing angle range of 25° and a fan shape; the horizontal FOV has a viewing angle range of 360° and is commonly arranged on the roof. A common vehicle-mounted semi-solid laser radar can reach a range of 120°, but due to the limitations, there will be a perception blind area around the horizontal vehicle body that is difficult to monitor. This area hides many unmonitored corner cases, which is the direction that the present application focuses on and solves.

[0096] ​RS-Ruby adopts a hybrid solid-state laser radar mode, the measurement distance can reach 250 meters, the measurement accuracy is up to + / - 3cm, the horizontal angle of measurement is 360°, and the vertical angle of measurement is -25°~15°. RS-Ruby continuously scans the external environment by emitting high-frequency laser beams while rapidly rotating 128 laser emitters, and provides three-dimensional spatial point cloud coordinates and object reflection intensity through ranging algorithms, thereby providing strong support for positioning, navigation, obstacle avoidance, etc.

[0097] RS-Bpearl is a near-range blind-filling laser radar specially designed to eliminate the near-field blind area, which can detect objects within a few centimeters, has an ultra-wide field of view of nearly 360°x90°, and can effectively monitor the blind area range of the vehicle body, effectively solving the Corner Case of passing through narrow lanes, parallel dense traffic, pedestrians or pets approaching the vehicle body, etc.

[0098] By setting one RS-Ruby laser radar on the roof and four blind-filling laser radars on the vehicle body, the maximum visual range can be achieved, and the visual blind area around the vehicle body is avoided.

[0099] Two A-pillar cameras are arranged on the rearview mirror to mainly monitor the A-pillar blind area outside the vehicle. In complex weather such as rain and fog, the camera can capture the real-time situation around through image acquisition technology, assist the driver to observe the blind area road situation, and identify pedestrians and vehicles. When the safety distance is less than the safety distance, it is projected onto the HUD to display and remind the obstacle.

[0100] The vehicle body is equipped with a visibility sensor to timely discover adverse weather conditions, including visibility, temperature, humidity, automatically adjust the brightness of the HUD display, and detect the distance and size of the obstacle. The safety distance of the road, obstacle and other objects can be measured, and the warning is prompted through the car voice broadcast.

[0101] The vehicle body is equipped with a temperature sensor that can sense the temperature change of the surrounding environment, and convert the temperature signal into an electrical signal and send it to the vehicle control system. The installation position is generally the front and side of the vehicle, and the pedestrian outside the vehicle is fed back to the vehicle, thereby triggering the warning system to feedback.

[0102] The vehicle body is equipped with a corner sensor to detect the turning angle, turning direction and turning speed of the steering wheel. The automobile can automatically condition the torque in the three alarm states, thereby actively braking when the automobile loses stability, and changing the motion trajectory to maintain a safe distance from the obstacle, thereby ensuring the safety of driving.

[0103] It should be noted that since there can be multiple blind area obstacles, the embodiments of the present application can also select one from the multiple obstacles in the blind area as the main target, mainly by continuously verifying the safety distance of the obstacle from the vehicle, and after the verification reaches the preset number and the distance verification is passed, it is used as the main target, and the specific process is as shown in Figure 6

[0104] The working process of the blind area obstacle warning system is as shown in Figure 7

[0105] 1. When the safety distance between the vehicle and the blind area obstacle is 50-60 cm, a first level warning is given, triggering the AR information display subsystem, the HUD displays the obstacle image information, and the image display subsystem displays the graphic or text reminder according to the user's definition.

[0106] 2. When the safety distance between the vehicle and the blind area obstacle is 30-50 cm, a second level warning is given, triggering the AR information display subsystem, the image display subsystem, the steering wheel control subsystem, and the sound and light feedback subsystem to remind pedestrians outside the vehicle through the buzzer, and the interior atmosphere lamp performs light beam stroboscopic reminder.

[0107] 3. When the safety distance between the vehicle and the blind area obstacle is less than or equal to 30 cm, a third level warning is given, triggering the AR-HUD information display subsystem, the image display subsystem, and the steering wheel, triggering all subsystems, and the steering torque guidance subsystem adjusts the steering direction and angle to avoid collision according to the collision time and angle.

[0108] The voice feedback unit: collects the user's attention information through the A-pillar camera, and broadcasts the warning according to the safety distance analyzed by the data processor. For example, the driver monitors the blind area obstacle during driving, and the vehicle machine first acquires the user's gaze range and semantic information, and generates a broadcast warning instruction after analysis to remind "please pay attention to the left A-pillar blind area, and a first level warning will be started".

[0109] Two kinds of laser radars, sensors, A-pillar cameras and other hardware are installed for monitoring. When the first level warning condition is met, the AR-HUD will display a reminder on the windshield, and the main information includes vehicle basic state information, auxiliary driving information, navigation prompt information and operation task information, and blind area warning information. The monitoring data cooperate with the voice to remind the driver to make effective feedback in advance, and the specific feedback is executed according to the warning level, as shown in Figure 8

[0110] ​​​When the first level alarm condition is met, the road information is detected by the camera and laser radar, the system realizes the current vehicle blind area range obstacle judgment, the blind area obstacle information is quickly and accurately identified from the current road image, and the self-defined warning information including image and text is projected into the blind area range. However, due to the interference factors such as light intensity, weather change, shadow shielding, surrounding vehicle blocking and the like, the projection information needs to be processed in the original image to make the information display lightweight and clear, so as to achieve the purpose of feature extraction, image visualization and strong recognizability. The specific image preprocessing mode is shown in Figure 9 , which includes camera acquisition of blind area obstacle image, setting of image ROI (Region of interest) region, image graying, filter enhancement, edge detection, which ensures that the information display can better realize the purpose of eliminating interference information in the image and enhancing effective information, thereby improving the efficiency of image processing.

[0111] When the vehicle monitors that there is a pedestrian passing through the blind area range, the driver can define the road projection display pattern, select an expression and / or text for prompting, and make the avoidance more interesting and personalized through the interactive mode, effectively avoiding the contradiction generated during avoidance. The road projection self-defined expression and text are shown in Figure 10 .

[0112] When the second level alarm condition is met, the new steering wheel warning is activated, the left and right sensors and the microcontroller are set on the steering wheel for motion monitoring and heart rate detection. When the speed sensor senses that the vehicle speed exceeds a certain value, the corresponding alarm prompt signal is sent, including the steering wheel vibration warning at different frequencies according to the safety distance, the lamp belt on the steering wheel flashes at the same vibration frequency for warning, and the voice feedback unit cooperates to broadcast the information: "your speed has exceeded the safe speed, please drive slowly".

[0113] When the second level alarm condition is met, the new sound and light warning is activated, and the alarm standard of the warning sound and light alarm includes: 1. visual warning: LED flashing, the flashing frequency must be greater than 2HZ; 2. sound warning: must emit an emergency sound within the warning range, the volume is not less than 90 decibels, and the frequency is greater than 1KHZ. When pedestrians and vehicles are monitored in the blind area range, the in-vehicle and out-vehicle lamp belts flash to remind, the driver can select the reminder sound through the buzzer to transmit, and the pedestrians and vehicles in the blind area are warned.

[0114] When the three-level alarm condition is met, the new active steering torque reminder, the driver's behavior intention and the characteristic recognition are mainly based on the real-time collected driver operation signals, the vehicle driving state data and the driver state obtained through the A-pillar camera to recognize the driver's driving behavior intention and driving characteristics. Through the recognition of the driver's behavior intention and driving characteristics, the electric control system parameters are adjusted, and according to the driver's driving intention, the vehicle is divided into left and right lane changing, lane keeping and emergency obstacle avoidance. The torque calculation mode is:

[0115] τ guide = f (Δθ) ;

[0116] Δθ = θ real - θ target ;

[0117] Wherein, τ guide is the steering torque guidance, Δθ is the difference between the real steering wheel angle θ real and the target steering wheel angle θ target . According to the needs of the vehicle, the steering torque guidance will be divided into three stages with the increase of the steering angle difference: slight guidance stage, linearly increasing torque guidance stage and torque guidance saturation stage.

[0118] Therefore, f (Δθ) can be expressed by the following function:

[0119]

[0120] Wherein, K is the peak value coefficient of the guidance torque, a is the linear difference value curvature of the steering angle, and b is the linear compensation of the steering angle difference.

[0121] For different driving intentions, the size of the steering torque is linearly adjusted. When the driving intention is lane keeping, the torque slightly intervenes to improve the change rate of the torque curve; when the driving intention is left / right lane changing, the corresponding torque is linearly increased until the saturation state; when the driving intention is emergency avoidance, the torque directly reaches the saturation state, so that the vehicle is in a stable state to positively guide the vehicle and change the running track of the vehicle to complete the emergency obstacle avoidance.

[0122] In addition, the HUD information subsystem of the present application also sets up a plurality of modules, as shown in Figure 11 .

[0123] The information display content of vehicle state, environment state, user state and vehicle braking is static state type prompt and feedback about current various situations. Therefore, they can be integrated into the same state change module. The environment warning information and vehicle keeping information involve dynamic scene type prompt and warning around the current situation change, so they can be integrated into the same scene change module. In addition, the complete HUD auxiliary driving system also includes vehicle basic information, navigation prompt information and operation task information, which are all elements of the system. Therefore, the HUD is mainly divided into the following regions:

[0124] 1. State stable display module. Mainly for vehicle basic information, such as vehicle speed, rotating speed, remaining fuel amount and other main driving task information, which is the first level information type in the driving task.

[0125] 2. State change display module. Mainly for vehicle state information, environment state information, user state information and vehicle braking information, respectively containing cruise state, gear state, traffic signal, road speed limit, driving fatigue, vehicle deceleration and other auxiliary driving information content directly related to vehicle driving and control state, which is the second important level in the driving task.

[0126] 3. State change display module. Mainly for vehicle state information, environment state information, user state information and vehicle braking information, respectively containing cruise state, gear state, traffic signal, road speed limit, driving fatigue, vehicle deceleration and other auxiliary driving information content directly related to vehicle driving and control state, which is the second important level in the driving task.

[0127] 4. Scene change display module. Mainly for environment warning information and vehicle keeping information, respectively including vehicle collision, blind area pedestrian, vehicle distance and other auxiliary driving information content directly related to potential danger in vehicle driving process, which is also the second important level in the driving task.

[0128] 5. Space-time change display module. Mainly for navigation prompt information, such as driving direction, turning distance, lane situation, arrival time, road name and other auxiliary driving information changing with space-time position.

[0129] 6. Operation change display module. Mainly for operation task information, such as telephone communication, music playing, temperature regulation and other non-driving task information.

[0130] The blind area HUD information display, mainly in the state display module, can display the recommended driving speed when obstacles are monitored; the operating change display module sets the steering wheel vibration, steering wheel prompt light, buzzer, interior atmosphere light, and exterior light strip display icon, and displays the state according to the alarm level; the scene change display module can display blind area warning prompt information for vehicles and pedestrians, mainly divided into seven kinds of scene changes, including lane position prompt, lane deviation prompt, vehicle distance prompt, overtaking auxiliary prompt, vehicle collision prompt, blind area pedestrian prompt, and surrounding vehicle prompt, which are displayed in green under normal circumstances, displayed in yellow when the secondary alarm condition is met, and displayed in red when the third alarm condition is met. The vehicle speed is color-coded according to the specification, with green indicating safety and feasibility, yellow indicating warning, and red indicating prohibition. The critical relationship between the vehicle speed and the speed limit is indicated by color gradient, such as Figure 12 as shown.

[0131] In summary, the embodiment of the application adopts a multi-modal information interaction system, uses advanced intelligent sensing technology, and can accurately identify various objects and situations in the blind area to provide real-time and reliable information for the driver. According to the safety distance, the system performs hierarchical alarm, integrates visual, sound, vibration, and other sensing methods to provide more comprehensive and diversified blind area information for the driver. The system can flexibly adjust the information transmission method according to the driver's habits, driving scene, and traffic conditions, and through reasonable design of the interaction interface and information feedback mechanism, the driver and the pedestrians and vehicles outside the vehicle can interact with each other. This comprehensive information transmission method can better meet the perception needs of different drivers and improve the effectiveness of information transmission.

[0132] The core is a blind area monitoring and warning display system. The system includes multiple subsystems, among which the vehicle blind area detection subsystem is mainly responsible for monitoring distance, temperature, angle, and visibility data. Once an object is detected in the blind area, the system will activate the AR-HUD information display subsystem to display relevant information on the vehicle's head-up display when the first-level alarm condition is met. The image display subsystem focuses on issuing prompts to pedestrians in the blind area and supports user customization according to individual needs to promote more interaction between vehicles, vehicle owners, and pedestrians. When the second-level alarm condition is met, the steering wheel control subsystem is newly activated to convey information to the driver through sound vibration and image warning; the sound and light feedback subsystem is newly activated to mainly remind the outside to issue a collision warning signal in time. When the third-level alarm condition is met, the driver's intention steering torque guidance subsystem is newly activated to identify the driver's intention, adjust the steering torque to change the driving angle, and effectively avoid obstacles to provide more comprehensive protection for driving safety.

[0133] The embodiment of the application also provides a vehicle including the blind area obstacle warning system as above.

[0134] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program or instructions, and the computer program or instructions are executed by a processor to implement the blind area obstacle early warning method.

[0135] In the description of the specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the specification and the features of the different embodiments or examples without contradiction.

[0136] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "N" is at least two, for example, two, three, etc., unless otherwise explicitly specified.

[0137] Any process or method descriptions in flow charts or described elsewhere herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing the specified logic functions (or steps) and / or can be implemented by hardware or a combination of hardware and software. The preferred embodiments of the present application thus include additional implementation examples that can not be explicitly described in the description of the specification, but are nevertheless included in the scope of the present application.

[0138] It should be understood that parts of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. As in another embodiment, if implemented in hardware, any one or more of the following technologies known in the art can be used: discrete logic circuit with logic gates for implementing logic functions on data signals, application specific integrated circuit with suitable combination logic gates, programmable gate array (PGA), field programmable gate array (FPGA), etc.

[0139] Those skilled in the art can understand that all or part of the steps of the foregoing method embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a computer readable storage medium. When the programs are executed, the steps of the method embodiments or a combination thereof are included.

Claims

1. A blind area obstacle warning method characterized by, The method comprises the following steps: monitoring a blind area image around the vehicle; identifying whether there is an obstacle in the blind area image; if there is the obstacle, warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle; the warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle comprises: determining a warning level based on the distance between the obstacle and the vehicle; warning the driver of the vehicle and / or the obstacle based on the warning level; the determination of the warning level based on the distance between the obstacle and the vehicle comprises: if the distance is greater than a first preset value and less than or equal to a second preset value, the first warning level is determined; if the distance is greater than a third preset value and less than or equal to the first preset value, the second warning level is determined; if the distance is less than or equal to the third preset value, the third warning level is determined, wherein the third preset value is less than the first preset value, and the first preset value is less than the second preset value; the warning mode comprises at least one of displaying an obstacle image, text and / or graphical reminder, voice reminder, sound reminder, light reminder, and steering wheel vibration reminder; the warning the driver of the vehicle and / or the obstacle based on the warning level comprises: if the first warning level is determined, displaying the obstacle image to the driver and performing voice and light reminders, and performing text and / or graphical reminder to the obstacle; if the second warning level or the third warning level is determined, displaying the obstacle image to the driver and performing voice, light and steering wheel vibration reminders, and performing text and / or graphical reminder, sound reminder and light reminder to the obstacle; after the warning the driver of the vehicle and / or the obstacle based on the distance between the obstacle and the vehicle, further comprising: if the third warning level is determined, identifying the driving intention of the driver; adjusting the steering torque of the vehicle based on the driving intention to adjust the motion trajectory of the vehicle to avoid the obstacle; the driving intention comprises: lane keeping, left lane changing, right lane changing, and emergency avoidance; the adjustment of the steering torque of the vehicle based on the driving intention comprises: if the driving intention is the lane keeping, adjusting the change rate of the steering torque; if the driving intention is the left lane changing or the right lane changing, linearly increasing the steering torque to a saturation state; if the driving intention is the emergency avoidance, directly adjusting the steering torque to the saturation state; the calculation formula of the steering torque is: ； wherein is a steering torque guide, is a real steering wheel angle is a difference to a target steering wheel angle , , is a guide torque peak coefficient, is a steering angle linear difference curvature, is a steering angle difference linear compensation.​ 2. A blind zone obstacle warning system, characterized by, The system is used to realize the blind area obstacle warning method of claim 1, and the system comprises: a monitoring subsystem for monitoring a blind area image around the vehicle and obtaining the distance between the obstacle and the vehicle; an information display subsystem for displaying an obstacle image in the blind area image; an image display subsystem for performing text and / or graphical reminder to the obstacle; a steering wheel control subsystem for performing voice, light and steering wheel vibration reminders to the driver of the vehicle; An acousto-optic feedback subsystem for sounding and lighting warning to the obstacle; A steering torque guiding subsystem for identifying the driving intention of the driver and adjusting the steering torque of the host vehicle based on the driving intention; A controller for controlling the monitoring subsystem to monitor, controlling at least one of the information display subsystem, the image display subsystem, the steering wheel control subsystem, the acousto-optic feedback subsystem to give a warning based on the distance, and controlling the steering torque guiding subsystem to adjust the steering torque of the host vehicle.

3. A vehicle characterized by comprising: The blind area obstacle warning system as claimed in claim 2.

4. A computer readable storage medium having stored thereon a computer program or instructions, characterized in that, The computer program or instructions are executed by a processor for implementing the blind area obstacle warning method as claimed in claim 1.

Images