Low-speed new energy vehicle driving assistance system based on vision and radar fusion

Abstract

The invention discloses a low-speed new energy vehicle driving assistance system based on vision and radar fusion, and belongs to the technical field of automotive electronics, which can adapt to price sensitive attributes, reduce the popularization threshold, adopt hardware combination of a low-cost monocular camera and an angle radar, and are matched with an integrated ADAS domain controller to replace a plurality of independent ECUs (electronic control units). The method does not need to depend on expensive devices such as a high-end millimeter wave radar and a laser radar, hardware and system cost is greatly reduced while multifunctional assistance is achieved, market positioning of low-speed new energy vehicles is perfectly matched, and through the fusion design of vision and a radar sensor and the combination of a front fusion and rear fusion cooperative verification algorithm, the positioning precision of the low-speed new energy vehicles is greatly improved. Visual data accurately identifies visual information such as lane lines, vehicles and pedestrians, radar data accurately detects parameters such as distance and speed and adapts to severe environments such as rain, fog and dark light, the advantages of the visual data and the radar data complement each other, the functional defects of a single sensor scheme are overcome, and the accuracy and stability of environment perception in different scenes are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of automotive electronics technology, specifically to a low-speed new energy vehicle driving assistance system based on vision and radar fusion. Background Technology

[0002] With the acceleration of urbanization and the deepening of environmental protection concepts, low-speed new energy vehicles (including micro electric vehicles, community commuter vehicles, and logistics delivery vehicles) are being used more and more widely in complex traffic scenarios such as urban roads, residential areas, and industrial parks. The drivers of these vehicles include ordinary people and delivery personnel, not all of whom are professional drivers. Moreover, the driving environment is characterized by dense pedestrian traffic, narrow lanes, and frequent emergencies, making the practicality and adaptability of active safety assistance technologies increasingly urgent.

[0003] However, existing technologies are insufficient to meet these needs and have several pain points: First, traditional ADAS systems rely heavily on high-performance millimeter-wave radar, lidar, or multi-view cameras, resulting in high hardware costs that contradict the price-sensitive nature of low-speed new energy vehicles, hindering widespread adoption. Second, single-sensor solutions have significant drawbacks. Camera-based visual solutions suffer from drastically reduced reliability in adverse environments such as rain, fog, strong light, and low light, and insufficient ranging accuracy. Radar-based solutions cannot identify key visual information such as lane lines and traffic signs, making it difficult to achieve core functions such as lane departure warnings. Third, existing systems have fragmented functions, with multiple auxiliary functions relying on independent control units, leading to complex system architectures, difficulties in data coordination, and further increased costs. Fourth, warning methods are limited, often consisting of simple icons and sound prompts, failing to allow drivers to intuitively perceive the specific location, distance, and movement of threatening targets. This not only reduces driver trust in the system but may also cause confusion due to false alarms, making it difficult to effectively improve driving safety.

[0004] To address this, a low-speed new energy vehicle driving assistance system based on the fusion of vision and radar is proposed. Summary of the Invention

[0005] The purpose of this invention is to provide a low-speed new energy vehicle driving assistance system based on vision and radar fusion to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a low-speed new energy vehicle driving assistance system based on vision and radar fusion, comprising a perception module, a data processing module, a warning execution module, and an environment modeling and display module. The perception module, data processing module, warning execution module, and environment modeling and display module are wirelessly connected. The perception module, through the combination and collaborative work of multiple hardware components, achieves accurate and comprehensive capture of key environmental information around the vehicle, thereby providing a basic data source for subsequent data processing, function implementation, and warning decision-making.

[0007] The data processing module is used for data reception and preprocessing, multi-source data fusion, core calculation and logical judgment, and early warning instruction issuance and data transmission;

[0008] The warning execution module is used to receive the risk assessment results output by the data processing module, and to convey safety risk information to the driver through multi-dimensional warning methods, which is directly related to the driving safety warning effect.

[0009] The environment modeling and display module is used to receive the fused environmental data from the data processing module, render and generate a bird's-eye view or third-person perspective environment model centered on the vehicle in real time, and intuitively display multiple elements such as target objects, lane lines, and warning information on the screen.

[0010] As a further preferred embodiment of this technical solution: the plurality of said hardware includes at least one camera arranged inside the windshield of the vehicle and at least two corner radars arranged on the left and right sides of the rear of the vehicle.

[0011] The camera is used to collect video image information in front of the vehicle and to perform lane line recognition, vehicle recognition, and pedestrian recognition.

[0012] The corner radar is used to detect the distance, speed, and angle information of target objects to the side, rear, and behind the vehicle.

[0013] As a further preferred embodiment of this technical solution: the data processing module adopts an integrated ADAS domain controller, which is connected to the camera and corner radar via CAN bus or high-speed Ethernet to receive and process the raw data from the sensors;

[0014] The data processing module specifically operates as follows:

[0015] A1. Data reception: Information acquisition from multiple sensor sources;

[0016] A2. Data Preprocessing: Time and Spatial Calibration;

[0017] A3. Data Fusion: Front-end and back-end fusion collaborative verification;

[0018] A4. Functional Calculation: Multiple ADAS functions are executed in parallel;

[0019] A5. Command Output: Distribution of warning and display signals.

[0020] As a further preferred embodiment of this technical solution: In A1, the received data includes receiving visual data, radar data, and vehicle status data respectively;

[0021] Among them, visual data is received in real time from the camera via CAN bus or high-speed Ethernet.

[0022] Among them, radar data is received synchronously from the detection data of the radar on the upper corner of the vehicle;

[0023] Among them, vehicle status data is obtained through the CAN bus to acquire the real-time operating status of the vehicle.

[0024] As a further preferred embodiment of this technical solution: In A3, a combination of "pre-fusion + post-fusion" is adopted to improve the accuracy and reliability of environmental perception. Specific operational steps are as follows:

[0025] C1. Pre-fusion basic processing;

[0026] C2, Post-fusion target association;

[0027] C3. Output of fusion results.

[0028] As a further preferred embodiment of this technical solution: In the A4, multiple ADAS functions are executed in parallel, including forward collision warning, lane departure warning, blind spot detection and lane change assist, reversing warning, door opening warning, and environmental modeling and display functions.

[0029] As a further preferred embodiment of this technical solution: the environmental modeling and display module has the following specific operating steps:

[0030] B1. Data Reception and Parsing: Acquiring information about the fusion environment;

[0031] B2. Visualization Transformation: Converting abstract data into graphical elements;

[0032] B3. Multi-terminal display allocation: Output differentiated information according to the scenario;

[0033] B4. Early warning linkage adaptation: Synchronize early warning visual signals;

[0034] B5. Interactive Response: Processes driver operation commands.

[0035] As a further preferred embodiment of this technical solution: In B1, data reception and parsing includes receiving core data and verifying data validity;

[0036] Among them, the core data receiving module receives the "fusion environment model list" generated by the data processing module in real time via CAN bus or LVDS interface.

[0037] Among them, data validity verification checks the completeness and timeliness of the received data, removes data that has expired or has incorrect format, and ensures that the input data can be used for subsequent visualization processing.

[0038] In B2, based on the verified fused data, a graphical algorithm is used to complete the mapping transformation of "data-graphics" and generate a standardized visual element library;

[0039] In B4, risk warning instructions are received from the warning execution module, and the displayed content is dynamically adjusted to achieve "auditory or tactile warning - visual warning" coordination.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] 1. This invention can adapt to price-sensitive attributes, lower the threshold for popularization, and adopt a low-cost hardware combination of monocular camera and corner radar. It is paired with an integrated ADAS domain controller to replace multiple independent ECUs. It does not need to rely on expensive components such as high-end millimeter-wave radar and lidar. While achieving multi-functional assistance, it significantly reduces hardware and system costs, perfectly matching the market positioning of low-speed new energy vehicles and facilitating the popularization of technology.

[0042] 2. This invention can improve perception reliability and cover complex environments. Through the fusion design of visual and radar sensors, combined with the "front fusion + back fusion" collaborative verification algorithm, visual data can accurately identify visual information such as lane lines, vehicles, and pedestrians, while radar data can accurately detect parameters such as distance and speed and is adapted to harsh environments such as rain, fog, and low light. The two complement each other's advantages, solve the functional defects of single sensor solutions, and significantly improve the accuracy and stability of environmental perception in different scenarios.

[0043] 3. This invention simplifies the system architecture and improves operating efficiency. It centrally carries multiple ADAS functions such as forward collision warning, lane departure warning, and blind spot detection with a single ADAS domain controller, realizing hardware resource sharing and deep data collaboration. This avoids the problems of complex architecture and difficult data collaboration caused by multiple control units in traditional systems. While simplifying the structure, it reduces additional costs and improves the overall operating efficiency of the system.

[0044] 4. This invention optimizes the warning experience, accurately responds to low-speed scenarios, and innovatively designs an environmental modeling and display function. It renders abstract, fused data in real time as a bird's-eye view or third-person perspective environmental model centered on the vehicle. Combined with the differentiated dual-screen display of the instrument panel (high-priority information) and the central control screen (panoramic detail information), it allows the driver to intuitively perceive the location, distance, and movement of threatening targets. Simultaneously, it integrates visual, auditory, and tactile multi-dimensional warnings, solving the problems of traditional single-warning methods being unintuitive and having low driver trust. Addressing the unique characteristics of low-speed scenarios, such as dense pedestrian traffic and frequent emergencies, it features specially designed reversing warning and door opening warning functions, accurately responding to high-frequency risks in scenarios such as reversing in parks and roadside parking, effectively improving driving safety and practical experience. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the architecture of a low-speed new energy vehicle driving assistance system based on vision and radar fusion according to the present invention.

[0046] Figure 2 This is a flowchart illustrating the operation of the data processing module in a low-speed new energy vehicle driving assistance system based on vision and radar fusion according to the present invention.

[0047] Figure 3 This is a flowchart illustrating the operation of the environmental modeling and display module in a low-speed new energy vehicle driving assistance system based on vision and radar fusion, according to the present invention. Detailed Implementation

[0048] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0049] Example

[0050] Please see Figures 1-3 This invention provides a technical solution: a low-speed new energy vehicle driving assistance system based on vision and radar fusion, including a perception module, a data processing module, a warning execution module, and an environment modeling and display module. The perception module, data processing module, warning execution module, and environment modeling and display module are wirelessly connected. The perception module, through the combination and collaborative work of multiple hardware components, achieves accurate and comprehensive capture of key environmental information around the vehicle, thereby providing a basic data source for subsequent data processing, function implementation, and warning decision-making.

[0051] The data processing module is used for data reception and preprocessing, multi-source data fusion, core calculation and logical judgment, and early warning command issuance and data transmission.

[0052] The warning execution module is used to receive the risk assessment results output by the data processing module and convey safety risk information to the driver through multi-dimensional warning methods, which is directly related to the driving safety warning effect. The hardware of the warning execution module includes an instrument panel display, a buzzer alarm and a steering wheel vibration motor, which are used to provide the driver with visual, auditory and tactile warning information.

[0053] The environment modeling and display module is used to receive the fused environmental data from the data processing module, render and generate a bird's-eye view or third-person perspective environment model centered on the vehicle in real time, and intuitively display multiple elements such as target objects, lane lines, and warning information on the screen.

[0054] The environmental modeling display module hardware includes a combination instrument display screen and a central multimedia touch screen.

[0055] In this embodiment, specifically: multiple hardware components include at least one camera (monocular camera) arranged inside the windshield of the vehicle and at least two corner radars arranged on the left and right sides of the rear of the vehicle.

[0056] Among them, the camera is used to collect video image information in front of the vehicle to perform lane line recognition, vehicle recognition, and pedestrian recognition;

[0057] Among them, the corner radar is used to detect the distance, speed and angle information of target objects to the side, rear and behind the vehicle.

[0058] In this embodiment, specifically: the data processing module uses an integrated ADAS domain controller, which is connected to the camera and corner radar via CAN bus or high-speed Ethernet to receive and process the raw data from the sensors;

[0059] The data processing module, in particular, operates as follows:

[0060] A1. Data reception: Information acquisition from multiple sensor sources;

[0061] A2. Data Preprocessing: Time and Spatial Calibration;

[0062] A3. Data Fusion: Front-end and back-end fusion collaborative verification;

[0063] A4. Functional Calculation: Multiple ADAS functions are executed in parallel;

[0064] A5. Command Output: Distribution of warning and display signals.

[0065] In this embodiment, specifically: in A1, receiving data includes receiving visual data, radar data, and vehicle status data respectively;

[0066] The visual data is received in real time from the camera via CAN bus or high-speed Ethernet. This includes video image frames of the road ahead, as well as the results of preliminary processing by the camera's built-in ISP (Image Signal Processor) and NPU (Neural Processing Unit), such as lane line pixel coordinates and a list of vehicles / pedestrians ahead (including target type, pixel position, size, etc.).

[0067] Among them, radar data synchronously receives detection data from the vehicle's upper corner radar, including point cloud information of targets to the side and rear (generated by processing through FFT fast Fourier transform and CFAR constant false alarm rate algorithm), as well as key parameters of the targets: distance (relative to the vehicle), speed (relative speed and absolute speed), and azimuth angle (the angular position of the target in the vehicle coordinate system).

[0068] Among them, vehicle status data is obtained through the CAN bus to acquire the real-time operating status of the vehicle, including vehicle speed, gear (such as reverse gear, P gear), turn signal switch status, door handle unlock signal, etc., thereby providing a basis for scene judgment for function triggering.

[0069] In this embodiment, specifically: In A2, time synchronization: the timestamps of the camera, corner radar and vehicle status data are aligned to eliminate the time delay difference in the acquisition and transmission of data from different sensors, ensuring that all data are analyzed based on the same time node, and avoiding target association errors caused by time sequence deviations;

[0070] Coordinate transformation: Unifying multi-source data into the "vehicle coordinate system" (with the vehicle's center of gravity as the origin, forward as the positive X-axis, and right as the positive Y-axis), specifically including:

[0071] Visual data conversion: The target pixel coordinates output by the camera are combined with the camera's intrinsic parameters (focal length, principal point coordinates) and extrinsic parameters (installation position, pitch angle) and converted into world coordinates (meter level) through perspective transformation.

[0072] Radar data conversion: Convert the polar coordinates (range, azimuth) output by the corner radar into rectangular coordinates (X, Y values) to achieve spatial coordinate unification with visual data.

[0073] In this embodiment, specifically: In A3, a combination of "pre-fusion + post-fusion" is used to improve the accuracy and reliability of environmental perception. The specific operational steps are as follows:

[0074] C1. Pre-fusion basic processing: Perform preliminary correlation between the image features of the camera (such as lane line edges and target outlines) and the point cloud features of the radar (such as point cloud clusters of target outlines) to filter out feature data that may belong to the same target and reduce the amount of subsequent calculations.

[0075] C2. Post-fusion target association: Establish "visual-radar" target matching relationships for pre-processed target data;

[0076] C3. Fusion Result Output: Generate a unified "Fusion Environment Model List", which includes the vehicle's status (GPS coordinates, vehicle speed, gear), target list (type, world coordinates, speed of each target), and lane line information (curvature, width, and relative position to the vehicle).

[0077] In C2, establishing a "visual-radar" target matching relationship includes:

[0078] The camera identifies the "car in front" target (including pixel position and size) and matches it with the "point cloud cluster 30 meters ahead with an azimuth angle of 0°" detected by the radar.

[0079] For a successfully matched target, the advantages of both data are combined: visual data is used to confirm the target type (such as "car" or "pedestrian"), and radar data is used to calibrate the target's distance and speed (to compensate for the lack of accuracy in visual ranging).

[0080] For isolated targets that do not match (such as distant traffic signs that are visually recognized but not detected by radar, or small obstacles that are detected by radar but not visually recognized), mark them individually and assess their credibility to avoid missing key risk points.

[0081] In this embodiment, specifically: in A4, multiple ADAS functions are executed in parallel, including Forward Collision Warning (FCW), Lane Departure Warning (LDW), Blind Spot Detection (BSD) and Lane Change Assist (LCA), Rear Cross Traffic Alert (RCTA), Door Opening Warning (DOW), and environmental modeling and display functions.

[0082] In this embodiment, specifically: Forward Collision Warning (FCW): Calculates the relative distance (X coordinate of the target after fusion) and relative speed (difference between the target speed and the vehicle speed) between the vehicle and the target ahead. The collision time is calculated using the formula "TTC = relative distance / relative speed". If TTC < threshold (default 2.7 seconds, calibrable), it is determined to be a collision risk.

[0083] Lane Departure Warning (LDW): Calculates the offset and rate of deviation of the vehicle's center of gravity relative to the center of the lane based on the world coordinates of the lane lines; if no turn signal is detected and the offset is greater than a threshold (such as 1 / 3 of the lane width), it is determined to be an unintentional lane departure.

[0084] Blind Spot Detection (BSD) and Lane Change Assist (LCA): A blind spot area is defined on the side and rear of the vehicle (based on the detection range of the corner radar, approximately a 150° fan-shaped area); if the radar detects a target (with a speed similar to that of the vehicle) entering the blind spot, it is marked as "vehicle in blind spot"; if a turn signal signal on the same side is detected at this time, it is determined to be a lane change risk.

[0085] Reverse Traffic Alert (RCTA): When a reverse gear signal is received, the blind spot monitoring logic is activated, focusing on monitoring moving targets in the lateral direction of the reversing path (such as electric vehicles approaching laterally); if the relative speed of the target is greater than the threshold and the distance is less than 5 meters, it is determined to be a risk of reversing collision.

[0086] Door Opening Warning (DOW): When the vehicle speed is 0 and the gear is in P (or the engine is off), the radar maintains low power consumption operation; if a target (such as a bicycle) is detected to be rapidly approaching from the side and rear, and TTC < threshold (such as 1.5 seconds), and a door handle unlocking signal is received, it is determined to be a door opening risk.

[0087] Environmental modeling and display function: Converts the list of integrated environmental models into graphical data, providing signal support for the subsequent early warning execution module and display module.

[0088] In this embodiment, specifically: In A5, an instruction is output to the warning execution module: based on the risk level of different functions, visual, auditory, and tactile warnings are triggered, specifically including:

[0089] Auditory warning: Sends a signal to the buzzer, such as FCW triggering a "beep beep" sound, and LCA triggering a rapid "beep beep" sound;

[0090] Haptic warning: Sends a signal to the steering wheel vibration motor, such as LDW triggering unilateral vibration (corresponding to deviation from the direction).

[0091] Visual warning: Sends signals to the instrument panel, such as FCW displaying a red vehicle icon, and DOW displaying a red door opening warning icon;

[0092] Output commands to the environmental modeling display module: Distribute graphical data to the instrument display screen and the central multimedia touch screen via CAN bus or LVDS interface.

[0093] Instrument panel display: Outputs key high-priority information, such as lane line graphics, forward target icons and distance bars, and blind spot target icons, ensuring that the driver can obtain core risks simply by looking ahead;

[0094] Central multimedia touchscreen: Outputs complete bird's-eye view data, including 3D vehicle model, surrounding target models (different types of targets use different icons), blind spot color markings (amber), and supports users to zoom and rotate to view the 360° environment using gestures.

[0095] In this embodiment, specifically: the environment modeling and display module, and its specific operating steps are as follows:

[0096] B1. Data Reception and Parsing: Acquiring information about the fusion environment;

[0097] B2. Visualization Transformation: Converting abstract data into graphical elements;

[0098] B3. Multi-terminal display allocation: Output differentiated information according to the scenario;

[0099] B4. Early warning linkage adaptation: Synchronize early warning visual signals;

[0100] B5. Interactive Response: Processes driver operation commands.

[0101] In this embodiment, specifically: in B1, data reception and parsing includes receiving core data and data validity verification;

[0102] Among them, the core data receiving module receives the "fusion environment model list" generated by the data processing module in real time via CAN bus or LVDS interface, including:

[0103] Vehicle status data: GPS coordinates, real-time vehicle speed, current gear (e.g., reverse, P), turn signal switch status;

[0104] Surrounding target data: type of each target (car / pedestrian / electric vehicle), X / Y coordinates in the vehicle coordinate system (world coordinates), relative speed, and credibility score;

[0105] Road environment data: lane curvature, width, offset from the vehicle's center of gravity, and point cloud information of blind spot obstacles detected by radar;

[0106] Among them, data validity verification checks the completeness and timeliness of the received data, and removes data that has timed out (delay > 50ms) or has incorrect format (such as abnormal data with missing target type) to ensure that the input data can be used for subsequent visualization processing; if a certain type of data (such as radar point cloud) is missing, it is marked "information for this dimension needs to be supplemented" to avoid displaying errors;

[0107] In B2, based on the verified fused data, a graphical algorithm is used to complete the "data-graphics" mapping transformation, generating a standardized visual element library, including:

[0108] Target visualization: Assign a unique icon based on the target type (e.g., blue rectangular icon for cars, orange human-shaped icon for pedestrians, and small green icon for electric vehicles), and adjust the icon size according to the principle of "larger for closer targets and smaller for farther targets" (e.g., the icon size of a target 5 meters away from the vehicle is twice that of a target 10 meters away); at the same time, label the key parameters next to the icon (e.g., "distance 3m, speed 2km / h"), and change the font size of the parameters according to the target priority (risk level) (high-risk targets have bolder and larger fonts);

[0109] Road environment visualization: Convert lane line data into white solid / dashed line graphics (curvature dynamically adjusted according to road data, such as lane lines curving synchronously at curves), and mark the offset inside the lane line (such as "left offset 0.3m"); for blind spot obstacles detected by radar, mark them with gray semi-transparent squares (square size matches the estimated actual size of the obstacle), and overlay the text prompt "blind spot obstacle";

[0110] Vehicle Model Construction: Generate a simplified 3D graphic model of the vehicle (with the same proportions as the actual vehicle's exterior outline), and mark the current gear (e.g., a red "R" symbol is displayed at the rear of the vehicle when it is in reverse) and the vehicle speed (the vehicle speed is displayed digitally at the top of the vehicle model) on the model to ensure that the driver can quickly identify the status of his own vehicle.

[0111] In B4, risk warning instructions are received from the warning execution module, and the displayed content is dynamically adjusted to achieve "auditory or tactile warning - visual warning" coordination;

[0112] Among them, the risk area is highlighted: if the DOW warning is triggered (a target is approaching from the side and rear and the door handle is unlocked), the corresponding side and rear area is marked with a red flashing box in the bird's-eye view on the central control screen, and the instrument panel simultaneously displays a simplified "side and rear risk" icon; if the RCTA reversing warning is triggered, the target's movement trajectory is marked with a yellow dynamic arrow in the reversing path direction in the bird's-eye view, intuitively indicating the direction of the collision risk;

[0113] Warning intensity matching: In high-risk scenarios (such as RCTA detecting a vehicle approaching from the left within 5 meters), the flashing frequency of the risk area is increased to 2 times / second, and a semi-transparent warning pop-up window appears on the central control screen (such as "Caution when reversing! Vehicle approaching from the left"); In medium-risk scenarios (such as slight lane departure warning), only the side corresponding to the lane line is highlighted in orange, without additional pop-up windows, to avoid interfering with driving.

[0114] In this embodiment, specifically: in B3, based on the functional positioning of the instrument display screen (driving focus scenario) and the central multimedia touch screen (panoramic observation scenario), different visual content is allocated and displayed.

[0115] The instrument display screen shows the output:

[0116] View selection: The default view is "Simplified Front View", which centers the view on the front of the vehicle model and only displays high-risk targets within a 100-meter range directly in front and a 50-meter range to the side and rear (such as FCW targets with TTC < 2.7 seconds and vehicles entering the BSD blind spot).

[0117] Information simplification: Lane lines retain only the left and right boundary lines, without displaying redundant parameters; risk targets are highlighted with bright colors (FCW targets in red, BSD targets in amber), and are accompanied by distance reduction bars (such as a dynamic progress bar showing "3m→2m" below the FCW target), ensuring that drivers can obtain core risks simply by looking ahead;

[0118] Central multimedia touchscreen display output:

[0119] Panoramic View: Loads a "360° Bird's-eye View", which displays all targets (including distant targets with no risk), complete lane lines (including adjacent lanes), and blind spot obstacles within a 360° radius around the vehicle model.

[0120] Additional details: Directional markers ("front / back / left / right") are added to the edge of the bird's-eye view, and target parameter hovering is supported (clicking the target icon with the mouse or gesture will bring up detailed information such as "pedestrian, distance 4m, stationary"), meeting the needs of drivers to actively observe details.

[0121] In this embodiment, specifically: in B5, the driver's interactive operations (gestures, touch) are received in real time, the display status is adjusted, and the flexibility of use is improved;

[0122] View control: Supports gesture operation on the central control screen, such as two-finger zoom (zoom in / out of the bird's-eye view, minimum zoom to 10 meters, maximum zoom to 200 meters) and single-finger rotation (rotate the bird's-eye view to easily view details in a specific direction); the instrument panel only supports switching between "front view / side and rear view" using the steering wheel buttons to avoid distractions from complex operations;

[0123] Target filtering: Supports "target type filtering" (e.g., clicking "show only vehicles" hides pedestrian and obstacle icons) and "risk level filtering" (e.g., selecting "show only high risk" hides risk-free targets) on the central control screen, helping drivers focus on specific objects of interest; the filtering status is saved in real time and automatically loaded on the next startup;

[0124] Parameter feedback: If the driver adjusts the display parameters (such as the brightness of the instrument panel or the switch of the warning pop-up window on the central control screen), the module will synchronously store the parameter settings to the system and keep them consistent the next time it starts; at the same time, it will provide feedback on the interaction status to the data processing module (such as "the driver has viewed the BSD target") to help optimize the risk judgment logic.

[0125] Working principle: After the vehicle is powered on, all modules of the system automatically start and work together. The complete operation process is as follows:

[0126] Perception Activation and Data Acquisition: The monocular camera in the perception module (located inside the windshield) activates, continuously capturing video images of the road ahead. The built-in ISP and NPU then identify lane lines, vehicles, and pedestrians, outputting pixel coordinates. Simultaneously, the corner radars on both sides of the rear bumper activate, processing echo signals using FFT and CFAR algorithms to detect point cloud information such as distance, speed, and azimuth of targets to the sides and rear in real time. Simultaneously, the system acquires real-time operational data such as vehicle speed, gear position, turn signal status, and door handle unlock signals via the CAN bus.

[0127] Data preprocessing and collaborative fusion: The ADAS domain controller (the core of the data processing module) receives raw data from cameras, corner radars, and vehicle status data via CAN bus or high-speed Ethernet. First, it completes timestamp alignment (eliminating sensor timing deviations) and coordinate transformation (converting image pixel coordinates and radar polar coordinates to the vehicle coordinate system). Then, it adopts a combination of "front fusion + back fusion" to first associate camera image features with radar point cloud features, and then establishes a precise match between visual targets and radar targets. It combines the advantages of both to generate a fusion environment model list that includes the vehicle's status, target list (type, coordinates, speed), and lane line information (curvature, width).

[0128] Parallel computing of multiple ADAS functions: The domain controller executes six core functions in parallel based on the converged environment model:

[0129] Forward Collision Warning (FCW): Calculates the relative distance and speed between the vehicle and the target ahead to determine the Time to Collision (TTC). If the TTC is below the threshold of 2.7 seconds, it is considered a risk.

[0130] Lane Departure Warning (LDW): Analyzes the vehicle's deviation and speed relative to the center of the lane. If the deviation exceeds a threshold when there is no turn signal, it is determined to be an unintentional departure.

[0131] Blind Spot Detection (BSD) and Lane Change Assist (LCA): Define a 150-degree fan-shaped blind spot to the side and rear. If a target enters the blind spot and the turn signal on the same side is activated, it is considered a lane change risk.

[0132] Reverse Traffic Alert (RCTA): Upon receiving a reverse gear signal, the system focuses on monitoring lateral moving targets in the reversing path. If the speed and distance thresholds are met, a collision risk is identified.

[0133] Door Opening Warning (DOW): After the vehicle has come to a complete stop and is in P gear (or the engine is off), the radar operates at low power. When a target on the side or rear approaches rapidly and the TTC exceeds the threshold, and a door handle unlocking signal is detected at the same time, it is determined to be a door opening risk.

[0134] Environment modeling and display: Converting fused data into graphical data to support subsequent display modules.

[0135] Warning command distribution and multi-dimensional alerts: Based on the functional calculation results, the domain controller issues commands to the warning execution module: differentiated auditory warnings are issued through the buzzer (e.g., "beep beep" for FCW, and rapid "beep beep" for LCA), tactile warnings are provided through the steering wheel vibration motor (e.g., vibration on one side corresponding to deviation from the direction for LDW), and visual warning icons are output through the instrument panel display (e.g., red vehicle icon for FCW, red door opening icon for DOW), realizing multi-dimensional coordinated alerts of sight, hearing, and touch.

[0136] Environmental modeling visualization: The environmental modeling display module receives graphical data via CAN bus or LVDS interface to achieve differentiated dual-screen display.

[0137] The instrument cluster display presents high-priority information in simplified graphics, including lane lines, forward targets and distance bars, and blind spot target icons, ensuring that the driver can obtain key risks simply by looking ahead.

[0138] Central multimedia touchscreen: Renders a 360° bird's-eye view or a third-person perspective environment model, uses exclusive icons to distinguish targets such as cars, pedestrians, and two-wheeled vehicles, marks distance values, marks blind spots with colored areas, and overlays warnings such as red flashing boxes on risk targets. It supports gesture zooming and rotation interaction; when a warning is triggered, it automatically switches to the best view and focuses on the risk target, and simultaneously provides voice prompts.

[0139] The entire process forms a closed loop of "perception-fusion-computation-early warning-display", realizing full-scenario safety assistance for low-speed new energy vehicles, while taking into account reliability, cost-effectiveness and user experience.

[0140] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A low-speed new energy vehicle driving assistance system based on vision and radar fusion, comprising a perception module, a data processing module, a warning execution module, and an environment modeling and display module, wherein the perception module, data processing module, warning execution module, and environment modeling and display module are wirelessly connected, characterized in that: The perception module, through the combination and collaborative work of multiple hardware components, achieves accurate and comprehensive capture of key environmental information around the vehicle, thereby providing a basic data source for subsequent data processing, function implementation, and early warning decision-making. The data processing module is used for data reception and preprocessing, multi-source data fusion, core calculation and logical judgment, and early warning instruction issuance and data transmission; The warning execution module is used to receive the risk assessment results output by the data processing module, and to convey safety risk information to the driver through multi-dimensional warning methods, which is directly related to the driving safety warning effect. The environment modeling and display module is used to receive the fused environmental data from the data processing module, render and generate a bird's-eye view or third-person perspective environment model centered on the vehicle in real time, and intuitively display multiple elements such as target objects, lane lines, and warning information on the screen.

2. The low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 1, characterized in that: The hardware includes at least one camera located inside the windshield of the vehicle and at least two corner radars located on the left and right sides of the rear of the vehicle. The camera is used to collect video image information in front of the vehicle and to perform lane line recognition, vehicle recognition, and pedestrian recognition. The corner radar is used to detect the distance, speed, and angle information of target objects to the side, rear, and behind the vehicle.

3. The low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 2, characterized in that: The data processing module uses an integrated ADAS domain controller, which connects to the camera and corner radar via CAN bus or high-speed Ethernet to receive and process the raw data from the sensors. The data processing module specifically operates as follows: A1. Data reception: Information acquisition from multiple sensor sources; A2. Data Preprocessing: Time and Spatial Calibration; A3. Data Fusion: Front-end and back-end fusion collaborative verification; A4. Functional Calculation: Multiple ADAS functions are executed in parallel; A5. Command Output: Distribution of warning and display signals.

4. A low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 3, characterized in that: In A1, the received data includes visual data, radar data, and vehicle status data, respectively. Among them, visual data is received in real time from the camera via CAN bus or high-speed Ethernet. Among them, radar data is received synchronously from the detection data of the radar on the upper corner of the vehicle; Among them, vehicle status data is obtained through the CAN bus to acquire the real-time operating status of the vehicle.

5. A low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 4, characterized in that: In A3, a combination of "pre-fusion + post-fusion" is used to improve the accuracy and reliability of environmental perception. The specific operational steps are as follows: C1. Pre-fusion basic processing; C2, Post-fusion target association; C3. Output of fusion results.

6. A low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 5, characterized in that: In the A4, multiple ADAS functions are executed in parallel, including forward collision warning, lane departure warning, blind spot detection and lane change assist, reversing warning, door opening warning, and environmental modeling and display functions.

7. A low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 6, characterized in that: The specific operating steps of the environment modeling and display module are as follows: B1. Data Reception and Parsing: Acquiring information about the fusion environment; B2. Visualization Transformation: Converting abstract data into graphical elements; B3. Multi-terminal display allocation: Output differentiated information according to the scenario; B4. Early warning linkage adaptation: Synchronize early warning visual signals; B5. Interactive Response: Processes driver operation commands.

8. A low-speed new energy vehicle driving assistance system based on vision and radar fusion according to claim 7, characterized in that: In B1, data reception and parsing includes receiving core data and verifying data validity. Among them, the core data receiving module receives the "fusion environment model list" generated by the data processing module in real time via CAN bus or LVDS interface; Among them, data validity verification checks the completeness and timeliness of the received data, removes data that has expired or has incorrect format, and ensures that the input data can be used for subsequent visualization processing. In B2, based on the verified fused data, a graphical algorithm is used to complete the mapping transformation of "data-graphics" and generate a standardized visual element library; In B4, risk warning instructions are received from the warning execution module, and the displayed content is dynamically adjusted to achieve "auditory or tactile warning - visual warning" coordination.

Images