Warning indicator based on driving assistance system

Abstract

The invention discloses a warning indicator based on a driving assistance system, and the warning indicator comprises an environment sensing module which is composed of the following components: a forward millimeter wave radar with the working frequency of 76-77GHz and the horizontal detection angle of + / -45 degrees; the field angle of the wide-angle camera and the optical lens is larger than or equal to 150 degrees, and the image sensor supports 1280 * 720 pixel resolution. The system also comprises a data processing module, and the data processing module is used for carrying out the time synchronization of the detection data of the millimeter wave radar and the image data of the camera, and the synchronization error does not exceed 10ms. Through a cooperative early warning mechanism of millimeter wave radar pre-detection and steering wheel tactile feedback, collision early warning can be triggered in advance when a driver does not adopt brake operation, and compared with a traditional system depending on brake pedal triggering, the response time is greatly shortened; the problem of early warning lag caused by waiting for operation of a driver in the prior art is effectively solved, and the driving safety is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent driving warning, and specifically to a warning device based on a driving assistance system. Background Art

[0002] With the mandatory requirements of the ECE R140 regulation for vehicle blind spot monitoring systems (standard for new EU models starting from 2024), existing warning solutions mainly rely on two types of technical routes:

[0003] Active intervention based on the braking system: such as the pedal travel simulator proposed in the comparative document CN102700462B, which enhances braking perception through hydraulic feedback (the measured braking response time is shortened by 15%)

[0004] Passive warning based on vision: such as the 360° surround view system of Technical Solution A, which uses dynamic auxiliary lines to indicate the distance to obstacles

[0005] Typical application scenarios include:

[0006] The blind spot when a commercial vehicle turns right (EU accident statistics show that it accounts for 32% of fatal accidents of heavy vehicles).

[0007] Low-speed parking conditions (the false alarm rate of traditional ultrasonic radars is as high as 25% when <10 km / h).

[0008] As disclosed in the warning device and warning method in the panoramic assisted driving system with the authorization announcement number CN102700462B, it includes a plurality of image acquisition devices, a video decoder, a data memory, a central processing chip, and an in-vehicle display device; the image acquisition device is connected to the video decoder, and the central processing chip is respectively connected to the video decoder, the data memory, and the in-vehicle display device. For the warning device in the panoramic assisted driving system, wherein, the image acquisition device is a wide-angle camera capable of covering all the field of view ranges around the vehicle; the wide-angle camera is installed on the vehicle. For the warning device in the panoramic assisted driving system, wherein, the in-vehicle display device includes one or a combination of an in-vehicle LCD display screen, an OLED screen, a projection device, an LED screen, and other video input / output devices. After adopting the above technical solution, the beneficial effect of the present utility model is: by collecting the image information around the vehicle through the image acquisition device, forming a top view of the vehicle body, and combining the warning picture data with the top view of the vehicle body to form a picture with warning information, and displaying it on the in-vehicle display device, in this way, the driver can accurately judge a dangerous situation, make a decision in time, and adjust driving.

[0009] Although the above-mentioned patent improves the braking accuracy through hydraulic feedback, it only responds to the driver's braking operation and cannot predict the collision risk when the pedal is not depressed (such as the rear-end collision scenario caused by driver distraction), resulting in a lag in the warning timing (it takes an average of 1.2 seconds from the risk appearance to the pedal action, according to the NHTSA 2023 Driver Response Research Report); it is completely ineffective for non-braking related risks (such as side scratches). Summary of the Invention

[0010] The purpose of the present invention is to provide a warning device based on a driving assistance system to solve the problems raised in the above background technology.

[0011] To achieve the above purpose, the present invention provides the following technical solutions:

[0012] A warning device based on a driving assistance system, characterized in that it includes

[0013] An environmental perception module, which consists of the following components:

[0014] A forward millimeter-wave radar with a working frequency of 76 - 77 GHz and a horizontal detection angle of ±45°;

[0015] A wide-angle camera with an optical lens field of view angle ≥ 150° and an image sensor supporting a resolution of 1280 × 720 pixels;

[0016] It further includes a data processing module, which is used to execute:

[0017] Synchronize the detection data of the millimeter-wave radar and the image data of the camera with a synchronization error not exceeding 10 ms;

[0018] Run an obstacle detection algorithm improved based on YOLOv5 and output obstacle type and position information;

[0019] It further includes a warning output module, which consists of the following components:

[0020] A vibration motor set on the rim of the steering wheel with a vibration frequency range of 50 - 250 Hz;

[0021] A head-up display with an adjustable projection brightness range of 500 - 1500 cd / m 2 .

[0022] In the present invention, the environmental perception module further includes:

[0023] A short-range millimeter-wave radar set on the rear bumper of the vehicle with a detection distance of 0.2 - 8 meters;

[0024] An infrared thermal imager with a response band of 8 - 14 μm for detecting the thermal radiation of organisms.

[0025] In the present invention, when the data processing module performs obstacle tracking:

[0026] The millimeter-wave radar point cloud data is clustered using the DBSCAN algorithm;

[0027] The camera detection targets are tracked using the DeepSORT algorithm;

[0028] The tracking results of the radar and vision are fused through an extended Kalman filter.

[0029] In the present invention, the relationship between the vibration intensity I of the vibration motor and the collision risk value R satisfies: I = 0.2 × R 2 , where R is a dimensionless number from 1 to 5.

[0030] A control method based on the warning device includes the following steps:

[0031] Step S1, obtaining the distance, speed, and type data of obstacles around the vehicle through the environmental perception module;

[0032] Step S2, calculating the time to collision TTC, TTC = obstacle distance / relative speed;

[0033] Step S3, when TTC ≤ 5 seconds, determining the comprehensive risk value K according to the dynamic risk value calculation formula;

[0034] Step S4, performing hierarchical warnings:

[0035] If 3 ≤ K < 5, activate the visual warning component to display a yellow warning icon;

[0036] If K ≥ 5, synchronously trigger the vibration of the tactile warning component and the red flashing of the visual warning component.

[0037] In the present invention, in step S3, the values of the obstacle type coefficients are:

[0038] Pedestrians: 1.0;

[0039] Two-wheeled vehicles: 0.8;

[0040] Motor vehicles: 0.5;

[0041] Static obstacles: 0.3.

[0042] In the present invention, a rear collision warning is also included, and the specific steps are as follows:

[0043] Step S5, when the rear millimeter-wave radar detects an approaching object and the relative speed is greater than 30 km / h:

[0044] Step S6, control the seat belt pretensioner to apply a pulling force of 5 - 8 N;

[0045] Step S7, activate the vibration motor in the seat backrest to vibrate at a frequency of 100 Hz.

[0046] A computer-readable storage medium storing a computer program, where the computer program, when executed by a processor, implements the described control method.

[0047] Compared with the prior art, the beneficial effects of the present invention are:

[0048] 1. Through the collaborative early warning mechanism of pre-detection by millimeter-wave radar and tactile feedback of the steering wheel, the present invention can trigger collision early warning in advance when the driver does not take braking operations, significantly shortening the response time compared with the traditional system that relies on the braking pedal to trigger, effectively solving the problem of early warning lag caused by waiting for the driver's operation in the prior art, and significantly improving driving safety;

[0049] 2. Through the hierarchical perception architecture that integrates radar, vision, and ultrasonic sensors, the present invention realizes high-precision detection of obstacles in complex environments, overcomes the defect that traditional pure vision systems are prone to failure in adverse weather conditions such as backlight, rain, and fog, and at the same time effectively improves the recognition rate of low obstacles, enabling the early warning system to work reliably under various working conditions. Description of the Drawings

[0050] Figure 1 is the system structure block diagram of the present invention;

[0051] Figure 2 is the working flow block diagram of the present invention. Detailed Embodiments

[0052] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0053] Embodiment 1

[0054] Aiming at the problems of high false alarm rate and single warning method in the driving early warning system in complex traffic environments, the present invention realizes accurate identification of collision risks and user-friendly prompts through spatio-temporal fusion of multi-modal sensors and hierarchical tactile-visual linkage warning. Due to the collaborative design of an embedded system and machine learning algorithms, the following technical objectives need to be achieved relying on the hardware architecture:

[0055] Environmental perception delay ≤ 50 ms (meeting the requirements of ISO 26262 ASIL-B level);

[0056] The multi-target tracking accuracy rate ≥ 95% (tested on the CITYSCAPES dataset);

[0057] The warning intensity is adaptively adjusted (dynamically adjusted according to the driver's grip force feedback).

[0058] Based on the above technical requirements, this embodiment adopts the "terminal-edge-cloud" three-level architecture as Figure 1 shown:

[0059] Terminal layer:

[0060] The forward millimeter-wave radar (Continental ARS548) and the wide-angle camera (Sony IMX490) form a perception terminal;

[0061] The steering wheel integrates a 6-zone vibration motor (diameter 20mm, rated voltage 5VDC);

[0062] Edge layer:

[0063] The NVIDIA Xavier NX computing unit runs the improved YOLOv5s model (parameter quantity 7.2M);

[0064] The time synchronization module adopts the IEEE 1588v2 protocol (synchronization accuracy ±1μs);

[0065] Cloud layer:

[0066] Alibaba Cloud ECS stores historical warning data (SQLite database format);

[0067] The risk model online update service (incremental training once a week).

[0068] As Figure 2 shown, the specific process of the multi-level risk warning method executed on this architecture includes:

[0069] Step 1: Obtain the distance, speed, and type data of obstacles around the vehicle through the environmental perception module.

[0070] Specifically, the environmental perception terminal collects data, the millimeter-wave radar outputs point cloud data (10Hz@CAN bus), and the camera outputs RGB images (30fps@GMSL2 interface).

[0071] Step 2: Calculate the time to collision TTC, TTC = obstacle distance / relative speed.

[0072] Specifically, the edge computing unit performs sensor fusion, performs DBSCAN clustering (eps = 0.5) on the radar point cloud, and performs DeepSORT tracking on the image targets.

[0073] The DBSCAN algorithm is as follows:

[0074]

[0075] Among them, N points represents the number of point clouds within the cluster, ∈ represents the neighborhood radius, and v rel represents the relative speed of the target.

[0076] Step 3: When TTC ≤ 5 seconds, determine the comprehensive risk value K according to the dynamic risk value calculation formula.

[0077] Specifically, calculate the dynamic risk value using the following formula:

[0078]

[0079] Among them, D represents the distance between the vehicle and the obstacle, Δ v represents the relative speed, α represents the driver state coefficient, β represents the obstacle type weight, and γ represents the environmental visibility compensation factor.

[0080] α = 0.7 (default), and it drops to 0.5 when the steering wheel grip sensor detects a tense state (grip force > 5N);

[0081] The value of β is as follows:

[0082] Pedestrians: 1.0;

[0083] Two-wheeled vehicles: 0.8;

[0084] Motor vehicles: 0.5;

[0085] Static obstacles: 0.3.

[0086] Step 4: Execute hierarchical warnings:

[0087] If 3 ≤ K < 5, activate the visual warning component to display a yellow warning icon;

[0088] If K ≥ 5, simultaneously trigger the vibration of the tactile warning component and the red flashing of the visual warning component.

[0089] Specifically, trigger hierarchical warnings according to the value of K:

[0090] Level 1 warning (3 ≤ K < 5): AR-HUD displays a yellow exclamation mark (brightness 800 cd / m 2 );

[0091] Level 2 warning (K ≥ 5): Vibration at the 3 / 6 / 9 o'clock positions of the steering wheel (frequency 150 Hz) + red flashing of the HUD (1200 cd / m 2 ).

[0092] Step 5: The cloud records event data, including the timestamp, GPS location, and K-value curve (sampling rate 1Hz), for subsequent model optimization.

[0093] Embodiment 2

[0094] Hypothetical scenario:

[0095] Time: 20:30 at night.

[0096] Weather: Heavy rain (rainfall 3mm / h), with a water film forming on the windshield.

[0097] Road: An urban branch road without streetlights (ambient light ≤ 5 lux).

[0098] Participants:

[0099] This vehicle: Speed 60 km / h, driver holding the steering wheel with both hands (grip force sensor detects 3N). Pedestrian: Wearing dark clothes (reflectivity 10%), suddenly crossing from the right green belt.

[0100] First stage: Risk occurrence (t = 0 - 100 ms)

[0101] Event trigger:

[0102] The pedestrian steps into the lane, forming an oblique distance of 60 meters (angle 30°) with this vehicle.

[0103] Logical necessity: Simulating the common "ghost probe" accident form in the city.

[0104] The sensor input parameters are shown in the following table:

[0105] sensor raw data logical processing millimeter-wave radar Detected a moving target at 60.2 m Calculated the relative speed of 18.06 m / s through the Doppler effect camera Identified "pedestrian" (confidence level 92%) Enabled the rainy-day enhancement mode (ISO21448) steering wheel sensor Continuous grip force of 3 N Triggered the tension status flag bit (α = 0.5)

[0106] Second stage: System decision-making (t = 100 - 300 ms)

[0107] Step A, Risk modeling calculation

[0108] Input parameters:

[0109] Initial distance: 60.2 meters (measured by radar).

[0110] Speed of this vehicle: 60 km / h = 16.67 m / s.

[0111] Relative speed of the pedestrian: 18.06 m / s (measured by millimeter-wave radar Doppler).

[0112] System delay: 150 ms (including sensor transmission + preprocessing).

[0113] Step B, Calculation of time to collision:

[0114]

[0115] When the TTC < 2.5 seconds, the automatic emergency braking (AEB) is triggered.

[0116] 2.5 ≤ TTC < 5.0, a first-level visual warning (current case).

[0117] Step C, dynamic risk grading

[0118] Calculation of the comprehensive risk value:

[0119]

[0120] The weather compensation coefficient γ = 1.2, and the driver state coefficient α = 0.5 (when the steering wheel grip force is 3 N).

[0121] When the K value reaches or exceeds 3.0, the system does not take any active intervention measures.

[0122] When the K value is lower than 1.0, the automatic emergency braking system (AEB) is immediately activated.

[0123] The third stage: human-machine interaction (t = 300 - 500 ms)

[0124] The HUD displays a yellow exclamation mark (the brightness is automatically increased to 600 cd / m 2 )

[0125] Logical basis:

[0126] A heavy rain environment requires a higher brightness (ISO 15008 standard).

[0127] A grip force of 3 N disables vibration (to avoid scaring).

[0128] Driver response: step on the brake pedal at t = 420 ms (deceleration 0.4g).

[0129] It should be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device.

[0130] Although embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A warning device based on a driving assistance system, characterized in that: including an environmental perception module, which consists of the following components: a forward millimeter-wave radar with a working frequency of 76 - 77 GHz and a horizontal detection angle of ±45°; a wide-angle camera with an optical lens field of view angle ≥150° and an image sensor supporting a resolution of 1280×720 pixels; It further includes a data processing module, which is used to execute: synchronize the detection data of the millimeter-wave radar and the image data of the camera, with a synchronization error not exceeding 10 ms; run an obstacle detection algorithm improved based on YOLOv5 and output obstacle type and position information; It further includes a warning output module, which consists of the following components: a vibration motor set on the steering wheel rim with a vibration frequency range of 50 - 250 Hz; Head-up display, the adjustable range of projection brightness is 500 - 1500 cd / m 2 .

2. The warning device based on a driving assistance system according to claim 1, characterized in that: The environmental perception module further includes: a short-range millimeter-wave radar set on the vehicle rear bumper with a detection distance of 0.2 - 8 meters; an infrared thermal imager with a response band of 8 - 14 μm for detecting the thermal radiation of organisms.

3. The warning device based on a driving assistance system according to claim 1, characterized in that: When the data processing module performs obstacle tracking: cluster the millimeter-wave radar point cloud data using the DBSCAN algorithm; track the camera detection targets using the DeepSORT algorithm; fuse the tracking results of the radar and vision through an extended Kalman filter.

4. The warning device based on a driving assistance system according to claim 1, characterized in that: The relationship between the vibration intensity I of the vibration motor and the collision risk value R satisfies: I = 0.2 × R 2 , where R is a dimensionless number from 1 to 5.

5. A control method for a warning device according to any one of claims 1-4, characterized in that, including the following steps: Step S1, obtain the distance, speed, and type data of obstacles around the vehicle through the environmental perception module; Step S2, calculate the time to collision TTC, TTC = obstacle distance / relative speed; Step S3, when TTC ≤ 5 seconds, determine the comprehensive risk value K according to the dynamic risk value calculation formula; Step S4, perform hierarchical warnings: If 3 ≤ K < 5, activate the visual warning component to display a yellow warning icon; If K ≥ 5, synchronously trigger the vibration of the tactile warning component and the red flashing of the visual warning component.

6. The control method according to claim 5, wherein: In the said Step S3, the values of the obstacle type coefficients are: Pedestrians: 1.0; Two-wheeled vehicles: 0.8; Motor vehicles: 0.5; Static obstacles: 0.

3.

7. The control method according to claim 5, wherein: It also includes rear collision warning, and the specific steps are as follows: Step S5, when the rear millimeter-wave radar of the vehicle detects an approaching object and the relative speed is greater than 30 km / h: Step S6, control the seat belt pretensioner to apply a pulling force of 5 - 8 N; Step S7, activate the vibration motor in the seat backrest to vibrate at a frequency of 100 Hz.

8. A computer-readable storage medium storing a computer program, characterized in that: When the said computer program is executed by a processor, it implements the control method described in any one of claims 5 - 7.

Images