Electric vehicle anti-collision early warning method and system based on multi-sensor fusion

Abstract

The invention provides an electric vehicle anti-collision early warning method and system based on multi-sensor fusion, and relates to the technical field of electric vehicle collision early warning. According to the invention, the vehicle operation mode is intelligently divided, the multi-sensor fusion strategy is dynamically allocated, and the serial processing architecture is adopted to fuse the data of the laser radar, the millimeter wave radar and the camera in sequence, so that multi-sensor error accumulation and calculation redundancy are effectively reduced, and the obstacle recognition accuracy and real-time performance are improved. The system has the environment self-adaptive capacity, can still stably operate under the severe weather, remarkably reduces the computing power demand and energy consumption, and improves the reliability, economical efficiency and practicability of the auxiliary driving system of the electric vehicle.

Description

Technical Field

[0001] This invention relates to the field of electric vehicle collision warning technology, and more specifically, to an electric vehicle collision avoidance warning method and system based on multi-sensor fusion. Background Technology

[0002] Advanced Driver Assistance Systems (ADAS) refers to a series of technologies that use sensors, cameras, radar, and other devices installed on vehicles to perceive the vehicle's surroundings in real time. These technologies, along with algorithms and control systems, monitor, analyze, and intervene in the vehicle's driving status to assist the driver in completing certain driving tasks, thereby improving driving safety, comfort, and efficiency. ADAS provides assistance to the driver during vehicle operation, making it easier and safer for them to drive on the road. Lane keeping assist systems and automatic parking assist systems are examples of ADAS.

[0003] Multi-sensor fusion is the core of intelligent driving systems. It combines data from different sensors such as cameras, radar, and LiDAR to achieve a more comprehensive and accurate environmental perception than a single sensor. The ideal is to "compensate for each other's weaknesses," but in reality, this complex system also brings many significant drawbacks and challenges. For common obstacle recognition technologies, different sensors have different limitations. For example, millimeter-wave radar has weak capabilities in recognizing static objects; LiDAR has low recognition rates in low-visibility conditions such as heavy rain and dense fog; and camera solutions, which recognize 2D planar images, are prone to misidentification, leading to phenomena like "phantom braking."

[0004] For obstacle recognition in driver assistance systems, sensor data often contains errors. Different algorithms are needed to improve the accuracy of the collected data for different types of sensors. Multi-sensor fusion solutions, which use multiple sensors, can lead to a significant deviation in the final decision-making process due to the superposition of errors, which can easily result in misoperation. In order to improve the accuracy of data acquisition, the computational load of each sensor is also increasing exponentially, and the computing power requirements of driver assistance systems are also gradually increasing.

[0005] Therefore, there is an urgent need to provide a new solution to optimize the collision avoidance warning function of electric vehicles with a multi-sensor fusion architecture, so as to reduce the amount of computation while ensuring accuracy, and make the application of assisted driving to end products more economical. This is a technical problem that needs to be solved. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention is proposed. This invention provides a method and system for collision avoidance warning of electric vehicles based on multi-sensor fusion.

[0007] According to one aspect of the present invention, a collision avoidance warning method for electric vehicles based on multi-sensor fusion is provided, comprising the following specific steps: S1: Use sensors to collect operational and environmental information during vehicle operation; S2: Divided into multiple independent operating modes based on operational and environmental information; S3: Assign different sensor data fusion strategies to each operating mode; collect obstacle information in the vehicle's operating environment based on the preset fusion strategy; S4: Calculate the probability of collision between the vehicle and obstacles in the current operating state and execute different levels of warning strategies.

[0008] Preferably, the vehicle's operating information in step S1 includes the vehicle's direction of movement, speed, gear information, and position information; Environmental information includes road condition information and weather information.

[0009] Preferably, in step S2, the high-speed operating mode, medium-speed operating mode, and low-speed operating mode are divided based on road condition information, vehicle operating direction, gear information, and location information. High-speed operation mode includes driving scenarios without traffic lights on urban expressways or highways, where the vehicle needs to maintain the forward direction and the speed must be greater than or equal to 60km / h. The medium-speed operation mode includes urban driving scenarios with multiple traffic lights. The vehicle needs to maintain a forward direction and a speed greater than or equal to 10 km / h and less than 60 km / h. The low-speed operation mode is a closed road driving scenario, including park roads, parking lots, and indoor garages; the vehicle's direction of travel is forward or backward, and the vehicle's speed needs to be less than 10km / h.

[0010] Preferably, the sensor in step S1 includes: LiDAR: Installed on the roof of the vehicle and facing the direction of travel, it collects data in real time for 3D spatial positioning and obstacle contour detection; Surround view camera: Installed inside the vehicle on the windshield, facing the direction of the vehicle's movement to collect data in real time and identify obstacle categories; Ultrasonic radar: 4-12 units are installed at the front and rear bumpers of the vehicle to detect obstacles at short distances; Millimeter-wave radar: Installed at the front bumper of the vehicle, it collects data in real time facing the direction of the vehicle's movement and tracks obstacles detected by lidar.

[0011] Preferably, for high-speed operation mode, the first fusion strategy is executed; the lidar, surround view camera and millimeter-wave radar are kept on to collect data in real time; the ultrasonic radar is kept off. The first fusion strategy includes: generating a first feature value using point cloud data acquired by LiDAR; The first feature value is used as input to label the data collected by the millimeter-wave radar, and the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, and the third feature value is generated after the camera processes the data. The third feature value is input into the decision layer of the preset early warning model.

[0012] Preferably, for the medium-speed operation mode, the second fusion strategy is executed; when the vehicle speed is greater than or equal to 30km / h, the lidar, surround view camera and millimeter-wave radar are kept on to collect data in real time; the ultrasonic radar is kept off; when the vehicle speed is less than 30km / h, all four sensors are kept on. The second fusion strategy includes: dynamically adjusting the sampling frequency and processing parameters of each sensor based on vehicle speed; and generating the first feature value using point cloud data collected by LiDAR. The first feature value is used as input to label the data collected by the millimeter-wave radar, and the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, and the third feature value is generated after the camera processes the data. The third feature value is input into the decision-making layer of the pre-defined early warning model; When the ultrasonic radar is in operation, it detects obstacles on the side of the vehicle and issues alarms of different frequencies based on the distance between the vehicle and the obstacle.

[0013] Preferably, for low-speed operation mode, the third fusion strategy is executed; all four sensors remain on. The third fusion strategy includes: real-time detection of obstacles around the vehicle by ultrasonic radar, and using the distance between the obstacle and the vehicle as the first feature value; The first feature value is used as input to label the data collected by the lidar and detect low targets in the direction of vehicle movement; the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, identify the type of obstacle, and generate the third feature value after processing by the camera. The third feature value is input into the decision layer of the preset early warning model.

[0014] Preferably, step S3 further includes the following steps: Time synchronization and dynamic position compensation are performed on the four sensors; The obstacle information collected by the lidar is classified, and static obstacles and dynamic obstacles are marked. Visibility data from meteorological information is collected. The confidence level of the feature values ​​obtained after processing the data collected by the sensor is evaluated. For feature values ​​with low confidence, their weight in the fusion step is reduced until the feature fusion step is exited.

[0015] Preferably, in step S4, the third feature value output in step S3 is input into the early warning model to predict the vehicle's trajectory over a certain period of time in the future and to determine whether the trajectory intersects with the obstacle. The probability of collision is calculated based on the type of obstacle and the intersection with the obstacle; the warning intensity is adjusted according to the probability of collision.

[0016] A second aspect of the present invention provides an electric vehicle collision avoidance warning system based on multi-sensor fusion, which uses the above-described method to detect obstacles in a moving vehicle, including: The data acquisition unit includes perception sensors and environmental information sensors installed on the vehicle. The perception sensors include lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras, which are used to collect vehicle operation information. It also includes a data acquisition module that communicates with the vehicle's ECU to collect the vehicle's real-time speed and location information. The environmental information sensors include a communication unit that communicates with maps and weather service websites to obtain road condition information and weather information of the vehicle's location. Distributed computing unit: Integrated into the sensing sensors, it extracts features from the raw data collected by lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras to generate feature values; The computational processing unit is used to calculate and process the obtained feature values ​​to determine the probability of a collision between the vehicle and an obstacle. Decision output unit: Based on the collision probability output by the calculation and processing unit, adjust the warning intensity according to the predetermined model and output the warning information.

[0017] Compared with existing technologies, the electric vehicle collision avoidance warning method and system based on multi-sensor fusion provided by this invention intelligently divides the vehicle into three operating modes—high-speed, medium-speed, and low-speed—based on vehicle speed, road conditions, and environmental information, and dynamically allocates different multi-sensor fusion strategies for each mode, realizing on-demand scheduling and efficient utilization of sensor resources. Employing a serial data processing architecture, it extracts and fuses features from LiDAR, millimeter-wave radar, and surround-view cameras, effectively reducing error accumulation and computational redundancy in traditional parallel fusion, significantly improving the accuracy and real-time performance of obstacle recognition and tracking. It avoids the consumption of useless sensor operation, reducing computing power and energy consumption while ensuring comprehensive perception, solving the problems of high computational load and high cost in traditional multi-sensor fusion, and making it more economically applicable to terminal products. Simultaneously, the system possesses sensor fault self-diagnosis and dynamic weight adjustment capabilities, maintaining stable operation under adverse weather conditions. This method significantly reduces system computing power requirements and energy consumption while ensuring high-precision collision avoidance warning, improving the economy, reliability, and practicality of electric vehicle assisted driving systems. Finally, through trajectory prediction and collision probability-based graded warning, it accurately matches warning measures, effectively improving the safety and reliability of electric vehicle collision avoidance warning. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart of a collision avoidance warning method for electric vehicles based on multi-sensor fusion according to an embodiment of the present invention.

[0019] Figure 2 This is a block diagram of an electric vehicle collision avoidance warning system based on multi-sensor fusion according to an embodiment of the present invention. Detailed Implementation

[0020] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein.

[0021] As mentioned in the background section, the high computational load and computing power requirements of multi-sensor fusion architecture-based collision avoidance warning functions for electric vehicles present a pressing technical challenge: how to reduce the computational load while ensuring accuracy, so that assisted driving can be applied more economically to end products.

[0022] Example 1: Figure 1 This is a flowchart of a multi-sensor fusion-based collision avoidance warning method for electric vehicles according to an embodiment of the present invention. Figure 1 As shown, a collision avoidance warning method for electric vehicles based on multi-sensor fusion includes the following specific steps: S1: Use sensors to collect operational and environmental information during vehicle operation; The vehicle's operational information in step S1 includes its direction of movement, speed, gear position, and location; the environmental information includes road conditions and weather information. The sensors used in step S1 include: LiDAR: Installed on the roof and facing the direction of vehicle travel to collect data in real time for 3D spatial positioning and obstacle contour detection; Surround view camera: Installed inside the vehicle's windshield and facing the direction of vehicle travel to collect data in real time for obstacle category identification; Ultrasonic radar: 4-12 units are installed at the front and rear bumpers of the vehicle for short-range obstacle detection; Millimeter wave radar: Installed at the front bumper of the vehicle and facing the direction of vehicle travel to collect data in real time for target tracking of obstacles detected by LiDAR.

[0023] S2: Based on operational and environmental information, the system is divided into multiple independent operating modes. Specifically, in step S2, based on road condition information, vehicle direction of travel, gear information, and location information, the system is divided into high-speed, medium-speed, and low-speed operating modes. Among these, the high-speed operating mode includes traffic light-free driving scenarios on urban expressways or highways, where the vehicle's speed must maintain the forward direction and be greater than or equal to 60 km / h. In the specific process of dividing the operating modes, it is first necessary to determine the vehicle's operating status and speed, and then combine the vehicle's location information and road condition information for identification. Specifically, the vehicle must be in drive; the vehicle speed must be greater than or equal to 60 km / h; the vehicle's location information must be confirmed by combining BeiDou positioning and other information to determine whether the road it is on is an urban expressway or highway; if all of the above constraints are met, it can be considered to be in high-speed operation mode.

[0024] The medium-speed operation mode includes urban driving scenarios with multiple traffic lights. The vehicle's operating speed needs to maintain a forward direction and be greater than or equal to 10 km / h and less than 60 km / h. In the specific operation mode classification process, it is first necessary to determine the vehicle's operating status and speed, and then combine the vehicle's location information and road condition information for identification. Specifically, the vehicle must be in drive; the vehicle speed must be greater than or equal to 10 km / h and less than 60 km / h; the vehicle's location must be on a public road, excluding internal roads, closed roads, or parking lots; if all of the above constraints are met, the vehicle can be considered to be in medium-speed operation mode.

[0025] Low-speed operation mode refers to closed-road driving scenarios, including park roads, parking lots, and indoor garages; the vehicle's direction of travel is forward or reverse, and the vehicle's speed must be less than 10 km / h. In the specific process of classifying operation modes, it is first necessary to determine the vehicle's operating status and speed, and then combine this with the vehicle's location information and road condition information for identification. Specifically, the vehicle must be in forward or reverse gear; the vehicle speed must be less than 10 km / h; and the vehicle's location must be on public roads, internal roads of the park, closed roads, parking lots, etc. If all of the above constraints are met, it can be considered to be in low-speed operation mode.

[0026] To facilitate understanding of this solution, this embodiment only uses a simple speed model for division; the solution in this embodiment can be applied to other special situations such as narrow urban roads and congested road conditions; it is only necessary to set specific constraints, such as fine division based on the number of obstacles in medium-speed and low-speed operation modes; the specific division steps and constraints are not described in detail here.

[0027] S3: Assign different sensor data fusion strategies to each operating mode; collect obstacle information in the vehicle's operating environment based on the preset fusion strategy; High-speed operation mode: For high-speed operation mode, the first fusion strategy is executed; the lidar, surround view camera, and millimeter-wave radar remain on to collect data in real time; the ultrasonic radar remains off. The first fusion strategy includes: using point cloud data collected by lidar to generate a first feature value; using the first feature value as input to label the data collected by millimeter-wave radar, and then fusing them to generate a second feature value; using the second feature value as input to label the data collected by surround-view cameras, and then processing the data by the cameras to generate a third feature value; and inputting the third feature value into the decision layer of a preset early warning model.

[0028] For high-speed operation, lidar, surround-view cameras, and millimeter-wave radar are activated simultaneously and collect data in real time; given the characteristics of high-speed operation, the vehicle travels at a relatively high speed; it is necessary to identify obstacles at a distance and allow for braking distance and safety distance. Therefore, the LiDAR needs to collect point cloud data within the design range at the highest sampling frequency. For example, if a certain brand of LiDAR has a sampling frequency of 20Hz and a detection range of 200 meters, then in this embodiment, the LiDAR used will detect all obstacles within a 200-meter range in the direction of vehicle travel at a sampling frequency of 20Hz, obtaining 200,000 raw data points per frame, including outliers and ground points. 3D point cloud data is generated by dimensionality reduction of the raw data using statistical filtering and ground separation algorithms. Then, filtering and clustering algorithms are used to detect obstacles in the direction of travel, obtaining the 3D spatial coordinates, contour dimensions, and relative distance information of the obstacles. A feature vector is generated for each detected obstacle, and the resulting feature vector is used as the first feature value input to the millimeter-wave radar. In this embodiment, the first obstacle closest to the vehicle is used as an example, and its first feature value is: ; where d 激光 This is the set of relative distances to obstacles detected by the lidar; Q is the three-dimensional coordinates of the obstacle in the vehicle coordinate system; P is the size of the obstacle; v0 is the relative velocity of the obstacle; each obstacle generates an independent feature vector, which is then input into the millimeter-wave radar as an independent first feature value. Millimeter-wave radar has poor ability to identify static obstacles. Therefore, after scanning obstacles in the forward direction with lidar, algorithms can be used to mark them. Millimeter-wave radar can detect static obstacles during scanning. In addition, lidar has a long detection range. Taking this embodiment as an example, assuming the maximum detection range is 200 meters, considering the vehicle speed and the lidar's processing time, the detection range of the millimeter-wave radar can be adjusted to be shorter. Taking a vehicle traveling at 120 km / h as an example, if the lidar detects an obstacle at 200 meters, the lidar's calculation and processing time is 15 ms. After obtaining the calculation result, the vehicle has already traveled 0.5 meters. Therefore, the detection range of the millimeter-wave radar can be adjusted within a certain range according to the actual speed of the vehicle to allow sufficient time for lidar calculation and processing. Millimeter-wave radar detects obstacles only those captured by lidar, tracks targets, and calculates the distance between the moving vehicle and obstacles in real time. The feature vector generated by the millimeter-wave radar is fused with the first feature value of the lidar to form a second feature value, which is then input to the surround-view camera. The second feature value is... d is the relative distance between the obstacle target detected by the millimeter-wave radar and the lidar, after fusion; v r Let a be the relative velocity. r For relative acceleration, Here, 'id' represents the standard deviation of speed measurement, and 'id' is a unique identifier for obstacle tracking. The fusion of LiDAR and millimeter-wave radar can compensate for the poor static obstacle recognition capabilities of millimeter-wave radar. Utilizing computation time differences and serial processing can also eliminate decision-making errors caused by anomalies in the data from the two radars. For example, in a parallel processing scheme, if the LiDAR identifies a nearby obstacle while the millimeter-wave radar fails to detect it, the usual practice is to dynamically adjust the weights of the two sensors at the decision-making end based on confidence levels. However, as the vehicle travels at high speeds, if the problem of undetected obstacles persists, the system's computational load will increase dramatically, significantly reducing decision accuracy. This can lead to erroneous phantom braking or traffic accidents caused by insufficient safe distance due to excessively long decision times, impacting the user experience of the assisted driving system.

[0029] After the second feature value is input into the surround-view camera, the surround-view camera classifies the obstacles marked by the second feature value, selects them as regions of interest (ROIs), classifies the obstacle types, and generates a third feature value after processing by the computing unit of the surround-view camera; the third feature value is... ;c represents the obstacle category, including small cars, large vehicles, pedestrians, and non-motorized vehicles;flag represents the risk label, with 0 for low-risk targets and 1 for high-risk targets; for the third feature containing high-risk targets, computational resources are prioritized for decision output. The third feature value serves as the input to the model's decision layer, utilizing a pre-trained early warning model to make decisions and output different early warning strategies. By using the recognition data from the surround-view camera, the type of obstacle can be identified more accurately, and different early warning strategies can be implemented based on the risk level of the obstacle.

[0030] Medium speed operation mode: For medium-speed operation mode, a second fusion strategy is implemented. When the vehicle speed is greater than or equal to 30 km / h, the LiDAR, surround-view camera, and millimeter-wave radar remain on to collect data in real time; the ultrasonic radar remains off. When the vehicle speed is less than 30 km / h, all four sensors remain on. The second fusion strategy includes: dynamically adjusting the sampling frequency and processing parameters of each sensor based on the vehicle speed; generating a first feature value using the point cloud data collected by the LiDAR; using the first feature value as input to mark the data collected by the millimeter-wave radar, and generating a second feature value after fusion; using the second feature value as input to mark the data collected by the surround-view camera, and generating a third feature value after camera processing; inputting the third feature value into the decision layer of the preset warning model; when the ultrasonic radar remains on, it detects obstacles on the side of the vehicle and executes alarm sounds of different frequencies according to the distance between the vehicle and the obstacle. In medium-speed operation mode, the vehicle speed is lower than in high-speed mode, and the road conditions are relatively complex, such as urban roads with many traffic lights or rural roads. The vehicle speed is typically between 20-60 km / h. At this speed, the relative motion between static or dynamic obstacles and the vehicle is less than in high-speed mode. Therefore, the sampling rate of each sensor can be reduced by a certain order of magnitude. For example, in high-speed mode, the detection range of the lidar is 200 meters and the sampling frequency is 20Hz. In medium-speed mode, it can detect targets within a 100-meter range, and the corresponding sampling frequency can be reduced to 10-12Hz to reduce the power consumption of each sensor and the computational load. Simultaneously, the ultrasonic radar, which is activated, monitors other vehicles nearby during the vehicle's operation. An alarm is triggered when the distance is too close. In medium-speed operation mode, the first, second, and third characteristic values ​​are similar to those in high-speed operation mode and will not be elaborated further here.

[0031] For low-speed operation mode, the third fusion strategy is executed; all four sensors remain on. The third fusion strategy includes: ultrasonic radar detecting obstacles around the vehicle in real time and using the distance between the obstacle and the vehicle as the first feature value; using the first feature value as input to label the data collected by lidar and detect low-lying targets in the vehicle's direction of travel; generating a second feature value after fusion; using the second feature value as input to label the data collected by the surround-view camera, identifying the type of obstacle, and generating a third feature value after camera processing; and inputting the third feature value into the decision layer of a preset warning model. In low-speed operation mode, the typical scenario is internal roads, and because internal roads lack high-precision map support, the types of obstacles are diverse; and vehicles generally travel at low speeds on internal roads; therefore, ultrasonic radar can be used as the main sensor to detect obstacles within a 10-meter radius around the vehicle and generate the first feature value; at this time, the first feature value is... In the formula, d 超声The relative distance to the obstacle; The azimuth angle of the obstacle; zone is the area division, corresponding to the installation location of the ultrasonic radar, such as directly in front, directly behind, left front, right front, left rear, or right rear.

[0032] The first feature value is input into the LiDAR and a second feature value is generated when the vehicle is moving forward. If the vehicle is reversing, the second feature value step is skipped, and the third feature value is generated directly after processing by the distributed computing unit of the surround-view camera. The second feature value is... ;d represents the distance after the obstacles are merged. is the azimuth angle of the obstacle, P is the size data of the obstacle, and flag is the label of the obstacle. Obstacles with a height less than 0.5m are labeled 1, and those with a height greater than 0m are labeled 0. The third feature value is... Where park represents available parking spaces identified by surround-view cameras; available parking spaces are marked as 1, and unavailable parking spaces are marked as 0.

[0033] As an optional solution, step S3 also includes the following steps: Time synchronization and dynamic position compensation are performed for the four sensors; other time synchronization and dynamic compensation schemes besides those in this embodiment can also be used; this embodiment is only introduced as an exemplary scheme, as detailed below: First, it is necessary to obtain the spatial transformation relationship from the radar coordinate system to the camera coordinate system; this step can eliminate the difference in the spatial position of the sensor caused by differences in installation location; then, a rigid body transformation matrix is ​​established. This includes a rotation matrix R and a translation vector t; By jointly calibrating the target and simultaneously capturing it with both a camera and radar, the intrinsic parameter matrix K and distortion coefficient dist of the camera are obtained. The matrix is ​​then solved based on an optimization algorithm to eliminate spatial errors. For different sampling rates of different sensors, interpolation can be used to handle the issue, and a conventional time synchronization algorithm is used for time calibration.

[0034] For point cloud data acquired by radar, the raw point cloud is sparse and disordered, requiring processing for feature extraction. Common methods include using RANSAC or plane fitting algorithms to remove ground point clouds, retaining only points that might be obstacles. The remaining point cloud is then clustered, grouping points belonging to the same object. A 3D bounding box is generated. For each cluster, its minimum and maximum boundaries across the entire point set are found, forming a 3D cube. This box can be represented by its center point, size, and orientation angle θ.

[0035] Since radar collects point cloud data while cameras collect image data, a 3D-to-2D projection is required before fusion. First, the eight vertices of the 3D cube are transformed from the radar coordinate system to the camera coordinate system. Then, the 3D points in the camera coordinate system are projected onto the camera's 2D imaging plane to obtain normalized camera coordinates. The camera's intrinsic parameter matrix and distortion coefficients are used to convert the normalized coordinates into pixel coordinates. At this point, the pixel coordinates of the eight vertices can be labeled, defining the Region of Interest (ROI). The camera only processes data within the ROI, significantly reducing the computational load.

[0036] In this embodiment, the obstacle information collected by the lidar can also be classified, and static obstacles and dynamic obstacles can be marked; this facilitates subsequent identification by millimeter-wave radar or cameras.

[0037] In this embodiment, visibility data from meteorological information is collected. The confidence level of the feature values ​​obtained after processing the data collected by the sensors is evaluated. For feature values ​​with low confidence, their weight in the fusion step is reduced until the feature fusion step is exited. Specifically, the data continuity of each sensor can be monitored in real time. For a sensor with no data for three consecutive frames, a fault is indicated; and the sensor is immediately put into sleep mode after a fault. After a fault, the fusion weights of other sensors are automatically adjusted. For example, in clear weather, the weight of the lidar in high-speed mode is 0.4, the weight of the millimeter-wave radar is 0.4, and the weight of the camera is 0.2; while in low visibility weather, the weights of the lidar and camera are reduced, and the weight of the millimeter-wave radar is increased accordingly to ensure that the fusion accuracy does not decrease significantly.

[0038] S4: Calculate the probability of collision between the vehicle and obstacles under the current operating state and execute different levels of warning strategies. In step S4, the third feature value output from step S3 is input into the warning model; the vehicle's trajectory is predicted within a certain time period in the future; it is determined whether the trajectory intersects with the obstacle; the collision probability is calculated based on the type of obstacle and the intersection with the obstacle; and the warning intensity is adjusted according to the collision probability.

[0039] In this embodiment, trajectory prediction is performed using an Extended Kalman Filter (EKF) to predict the trajectories of the vehicle and obstacles within the next 3 seconds, with a prediction step of 0.1 seconds. The focus is on determining whether the trajectories intersect. The safety distance is dynamically adjusted based on the obstacle type. For example, the safety distance for pedestrians = reaction distance + braking distance + 1.5m margin; the safety distance for heavy trucks = reaction distance + braking distance + 3m margin. The calculation formula is as follows: , D1 is the reaction distance; V1 is the vehicle's current speed; T1 is the driver's reaction time; the default is 1.2s, which can be adjusted according to driving habits. , D2 is the braking distance; V1 is the current vehicle speed; a1 is the vehicle braking acceleration; Collision probability calculation: If the predicted trajectories intersect and the actual distance is less than the safe distance, the collision probability is calculated based on the ratio of "actual distance / safe distance"; if the ratio is less than 0.3, the collision probability is greater than 80%, and if the ratio is between 0.3 and 0.5, the collision probability is between 50% and 80%.

[0040] The warning levels are divided into four levels based on the probability of collision, and the warning intensity is adjusted according to the type of obstacle, as shown in Table 1 below: Table 1 Collision Warning Classification Table Collision probability range Warning Level Early warning measures Applicable Scenarios ＜20% Level 1 warning The instrument panel displays a green warning: "Caution: Target Ahead." Obstacles in the distance, no immediate risk 20%-50% Level II Warning Audible and visual warning + flashing yellow indicator light The obstacle is approaching; the driver needs to be cautious. 50%-80% Level III Warning Active speed reduction + strong audio and visual warning The risk of collision is high and the driver may not react in time. ＞80% Level IV Warning Emergency braking, maximum deceleration braking + hazard lights activated Collisions are unavoidable; proactive intervention is necessary. This embodiment pre-divides multiple modes based on common scenarios in actual vehicle operation, sets different strategies in each mode, and makes full use of the advantages of each sensor; it changes the problem of large errors caused by the traditional parallel processing fusion scheme; it innovatively adopts a serial processing method, adjusts the detection range of subsequent sensors based on the sensor reaction time, ensures the temporal and spatial alignment of obstacles in the serial processing mode, and helps to improve the accuracy of the final decision-making end; it sets different warning modes according to the probability of collision between obstacles and vehicles; and realizes the obstacle collision avoidance warning function.

[0041] Example 2: like Figure 2 As shown, Figure 2 This is a block diagram of an electric vehicle collision avoidance warning system based on multi-sensor fusion according to an embodiment of the present invention; this embodiment provides an electric vehicle collision avoidance warning system based on multi-sensor fusion, which uses the method in Embodiment 1 to detect obstacles in a moving vehicle, including: The data acquisition unit includes perception sensors and environmental information sensors installed on the vehicle. The perception sensors include lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras, which are used to collect vehicle operation information. It also includes a data acquisition module that communicates with the vehicle's ECU to collect the vehicle's real-time speed and location information. The environmental information sensors include a communication unit that communicates with maps and weather service websites to obtain road condition information and weather information of the vehicle's location. Distributed computing unit: Integrated into the sensing sensors, it extracts features from the raw data collected by lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras to generate feature values; The computational processing unit is used to calculate and process the obtained feature values ​​to determine the probability of a collision between the vehicle and an obstacle. Decision output unit: Based on the collision probability output by the calculation and processing unit, adjust the warning intensity according to the predetermined model and output the warning information.

[0042] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative implementations described above.

[0043] In another aspect, the present invention also provides a computer-readable medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to implement the multi-sensor fusion-based electric vehicle collision avoidance warning method described in the above embodiments.

[0044] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0045] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, portable hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of the present invention.

[0046] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein.

[0047] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A collision avoidance warning method for electric vehicles based on multi-sensor fusion, characterized in that, The specific steps include the following: S1: Use sensors to collect operational and environmental information during vehicle operation; S2: Divided into multiple independent operating modes based on operational and environmental information; S3: Assign different sensor data fusion strategies to each operating mode; Obstacle information in the vehicle's operating environment is collected based on a preset fusion strategy; S4: Calculate the probability of collision between the vehicle and obstacles in the current operating state and execute different levels of warning strategies.

2. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 1, characterized in that, The vehicle's operating information in step S1 includes the vehicle's direction of movement, speed, gear information, and position information; Environmental information includes road condition information and weather information.

3. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 2, characterized in that, In step S2, based on road condition information, vehicle direction of travel, gear information, and location information, high-speed operating mode, medium-speed operating mode, and low-speed operating mode are distinguished. High-speed operation mode includes driving scenarios without traffic lights on urban expressways or highways, where the vehicle needs to maintain the forward direction and the speed must be greater than or equal to 60km / h. The medium-speed operation mode includes urban driving scenarios with multiple traffic lights. The vehicle needs to maintain a forward direction and a speed greater than or equal to 10 km / h and less than 60 km / h. The low-speed operation mode is a closed road driving scenario, including park roads, parking lots, and indoor garages; the vehicle's direction of travel is forward or backward, and the vehicle's speed needs to be less than 10km / h.

4. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 2 or 3, characterized in that, The sensors in step S1 include: LiDAR: Installed on the roof of the vehicle and facing the direction of travel, it collects data in real time for 3D spatial positioning and obstacle contour detection; Surround view camera: Installed inside the vehicle on the windshield, facing the direction of the vehicle's movement to collect data in real time and identify obstacle categories; Ultrasonic radar: 4-12 units are installed at the front and rear bumpers of the vehicle to detect obstacles at short distances; Millimeter-wave radar: Installed at the front bumper of the vehicle, it collects data in real time facing the direction of the vehicle's movement and tracks obstacles detected by lidar.

5. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 4, characterized in that, For high-speed operation mode, the first fusion strategy is executed; the lidar, surround view camera and millimeter-wave radar remain on to collect data in real time; The ultrasonic radar remains off. The first fusion strategy includes: generating a first feature value using point cloud data acquired by LiDAR; The first feature value is used as input to label the data collected by the millimeter-wave radar, and the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, and the third feature value is generated after the camera processes the data. The third feature value is input into the decision layer of the preset early warning model.

6. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 4, characterized in that, For medium-speed operation mode, the second fusion strategy is implemented; when the vehicle speed is greater than or equal to 30km / h, the lidar, surround view camera and millimeter-wave radar remain on to collect data in real time. The ultrasonic radar remains off; all four sensors remain on when the vehicle speed is less than 30 km / h. The second fusion strategy includes: dynamically adjusting the sampling frequency and processing parameters of each sensor based on vehicle speed; and generating the first feature value using point cloud data collected by LiDAR. The first feature value is used as input to label the data collected by the millimeter-wave radar, and the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, and the third feature value is generated after the camera processes the data. The third feature value is input into the decision-making layer of the pre-defined early warning model; When the ultrasonic radar is in operation, it detects obstacles on the side of the vehicle and issues alarms of different frequencies based on the distance between the vehicle and the obstacle.

7. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 4, characterized in that, For low-speed operation mode, the third fusion strategy is executed; all four sensors remain on. The third fusion strategy includes: real-time detection of obstacles around the vehicle by ultrasonic radar, and using the distance between the obstacle and the vehicle as the first feature value; The first feature value is used as input to label the data collected by the lidar and detect low targets in the direction of vehicle movement; the data is then fused to generate the second feature value. The second feature value is used as input to label the data collected by the surround-view camera, identify the type of obstacle, and generate the third feature value after processing by the camera. The third feature value is input into the decision layer of the preset early warning model.

8. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to any one of claims 5-7, characterized in that, Step S3 also includes the following steps: Time synchronization and dynamic position compensation are performed on the four sensors; The obstacle information collected by the lidar is classified, and static obstacles and dynamic obstacles are marked. Visibility data from meteorological information is collected. The confidence level of the feature values ​​obtained after processing the data collected by the sensor is evaluated. For feature values ​​with low confidence, their weight in the fusion step is reduced until the feature fusion step is exited.

9. The electric vehicle collision avoidance warning method based on multi-sensor fusion according to claim 1, characterized in that, In step S4, the third feature value output from step S3 is input into the early warning model to predict the vehicle's trajectory over a certain period of time in the future and to determine whether the trajectory intersects with the obstacle. The probability of collision is calculated based on the type of obstacle and the intersection with the obstacle; the warning intensity is adjusted according to the probability of collision.

10. A multi-sensor fusion-based collision avoidance warning system for electric vehicles, comprising using the method described in any one of claims 1-8 to detect obstacles in a moving vehicle, characterized in that, include: The data acquisition unit includes perception sensors and environmental information sensors installed on the vehicle. The perception sensors include lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras, which are used to collect vehicle operation information. It also includes a data acquisition module that communicates with the vehicle's ECU to collect the vehicle's real-time speed and location information. The environmental information sensors include a communication unit that communicates with maps and weather service websites to obtain road condition information and weather information of the vehicle's location. Distributed computing unit: Integrated into the sensing sensors, it extracts features from the raw data collected by lidar, millimeter-wave radar, ultrasonic radar, and surround-view cameras to generate feature values; The computational processing unit is used to calculate and process the obtained feature values ​​to determine the probability of a collision between the vehicle and an obstacle. Decision output unit: Based on the collision probability output by the calculation and processing unit, adjust the warning intensity according to the predetermined model and output the warning information.

Images