Anti-collision driving assistance method based on multi-sensor fusion, electronic equipment and readable storage medium

Abstract

The invention provides an anti-collision driving assistance method based on multi-sensor fusion, electronic equipment and a readable storage medium, and relates to the technical field of automatic driving, and the method comprises the steps: obtaining point cloud data obtained by detecting surrounding obstacles through a laser radar of a current vehicle, and three-dimensional image data obtained by detecting the obstacles through a vehicle-mounted camera, the sound source data is acquired by the microphone array; determining image positioning information of an obstacle based on the three-dimensional image data; determining sound positioning information of the dangerous sound source based on the sound source data; performing data fusion on the point cloud data, the image positioning information and the sound positioning information to obtain environment sensing data of the obstacle; and finally, for any obstacle, according to the environment sensing data of the obstacle and the driving data of the current vehicle, judging whether a collision risk exists between the current vehicle and the obstacle or not. And an anti-collision auxiliary driving strategy is generated when the collision risk exists. According to the method, accurate monitoring of obstacles is realized through multi-sensor fusion, so that the accuracy of an anti-collision driving technology is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to an anti-collision driving assistance method based on multi-sensor fusion, an electronic device and a readable storage medium. Background Art

[0002] With the acceleration of urbanization and the surge in car ownership, the economic losses caused by traffic accidents worldwide have exceeded one trillion US dollars each year. In complex traffic scenarios, drivers need to deal with 16 types of dynamic risk factors such as lane deviation, pedestrian crossing, emergency braking, etc. Based on the average reaction time of human drivers, at normal vehicle speeds, this means there is a large braking blind spot.

[0003] At present, vehicles are usually equipped with laser radars to realize autonomous obstacle avoidance functions, but traditional ranging technology has problems such as insufficient accuracy and poor reliability. It cannot effectively warn of collision risks and has difficulty coping with complex traffic environments. Summary of the invention

[0004] The embodiments of the present invention provide a collision avoidance driving assistance method, an electronic device and a readable storage medium based on multi-sensor fusion to solve the problem of poor accuracy of existing vehicle obstacle avoidance methods.

[0005] In a first aspect, an embodiment of the present invention provides an anti-collision driving assistance method based on multi-sensor fusion, comprising: Obtain the point cloud data obtained by the laser radar of the current vehicle detecting surrounding obstacles; Acquire three-dimensional image data obtained by the on-board camera of the current vehicle from obstacle detection, and determine image positioning information of at least one obstacle based on the three-dimensional image data; Acquiring sound source data collected by a microphone array of the current vehicle, and determining sound positioning information of a dangerous sound source based on the sound source data; Performing data fusion on the point cloud data, the image positioning information and the sound positioning information to obtain fused environmental perception data of the obstacle; the environmental perception data includes the position and speed of the obstacle; For any obstacle, determine whether there is a risk of collision between the current vehicle and the obstacle based on the environmental perception data of the obstacle and the driving data of the current vehicle; and generate an anti-collision assisted driving strategy when there is a risk of collision between the current vehicle and the obstacle.

[0006] In a possible implementation, the environmental perception data includes longitudinal distance; the driving data includes speed and acceleration; For any obstacle, judging whether there is a collision risk between the current vehicle and the obstacle according to the environment perception data of the obstacle and the driving data of the current vehicle includes: The longitudinal distance from the first obstacle to the current vehicle is divided by the first relative speed to obtain the collision time; the first obstacle is any obstacle around the current vehicle, and the first relative speed is the relative speed between the current vehicle and the first obstacle; Determining a minimum braking distance according to the current vehicle speed and the maximum deceleration; If the collision time is less than a first preset emergency braking threshold, or the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, or the first relative speed is greater than a preset relative speed threshold, or the acceleration of the current vehicle is greater than a preset acceleration threshold, it is determined that there is a collision risk between the current vehicle and the first obstacle.

[0007] In a possible implementation, when there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the collision time is not greater than a first preset emergency braking threshold and greater than a second preset emergency braking threshold, a first collision warning signal is generated; and the second preset emergency braking threshold is less than the first preset emergency braking threshold; If the collision time is not greater than the second preset emergency braking threshold and greater than a third preset emergency braking threshold, an obstacle avoidance driving route of the current vehicle is generated according to the driving data of the current vehicle and the environmental perception data of the first obstacle; and the third preset emergency braking threshold is less than the second preset emergency braking threshold; If the collision time is not greater than the third preset emergency braking threshold, an emergency braking signal is generated.

[0008] In a possible implementation, when there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, a second collision warning signal is generated, and the current vehicle is controlled by the adaptive cruise control system to adjust the longitudinal distance between the current vehicle and the first obstacle; If the longitudinal distance between the current vehicle and the first obstacle is less than a preset distance lower limit, and no braking signal and no turning signal are detected, the current vehicle is controlled to decelerate.

[0009] In a possible implementation, when there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the first relative speed is greater than a preset relative speed threshold, and the collision time is less than a fourth preset emergency braking threshold, controlling the current vehicle to steer to avoid the collision through an electric power steering system; The fourth preset emergency braking threshold is smaller than the first preset emergency braking threshold and larger than the second preset emergency braking threshold.

[0010] In a possible implementation, when there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the acceleration of the current vehicle is greater than a preset acceleration threshold and a brake system fault code signal is detected, the current vehicle is braked by the electronic parking brake system.

[0011] In a possible implementation, generating the obstacle avoidance driving route of the current vehicle according to the driving data of the current vehicle and the environmental perception data of the first obstacle includes: Determining driving data of adjacent vehicles of the current vehicle based on the vehicle group network; An obstacle avoidance driving route of the current vehicle is generated based on the driving data of the current vehicle, the driving data of the adjacent vehicles, and the environmental perception data of obstacles around the current vehicle.

[0012] In a possible implementation, fusing the point cloud data, the image positioning information, and the sound positioning information to obtain fused environment perception data of the obstacle includes: The point cloud data, the image positioning information and the sound positioning information are input into a deep learning model, and the environmental perception data of the fused obstacle is output.

[0013] In a second aspect, an embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method in any possible implementation manner of the first aspect is implemented as above.

[0014] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, implements the method in any possible implementation manner of the first aspect above.

[0015] The embodiment of the present invention provides a collision avoidance driving assistance method and device based on multi-sensor fusion, which successively obtains the point cloud data obtained by the laser radar of the current vehicle to detect surrounding obstacles, the three-dimensional image data obtained by the vehicle-mounted camera to detect obstacles, and the sound source data collected by the microphone array of the current vehicle; and determines the image positioning information of at least one obstacle based on the three-dimensional image data; determines the sound positioning information of the dangerous sound source based on the sound source data; then fuses the point cloud data, the image positioning information and the sound positioning information to obtain the environmental perception data of the fused obstacle; finally, for any obstacle, according to the environmental perception data of the obstacle and the driving data of the current vehicle, it is judged whether there is a collision risk between the current vehicle and the obstacle; and when there is a collision risk between the current vehicle and the obstacle, an collision avoidance assistance driving strategy is generated. The method realizes accurate monitoring of obstacles through multi-sensor fusion, thereby improving the accuracy of collision avoidance driving technology and ensuring driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of a collision avoidance driving assistance method based on multi-sensor fusion provided by an embodiment of the present invention; Figure 2 is a schematic diagram of an obstacle positioning method using a binocular camera provided in an embodiment of the present invention; Figure 3 is a schematic structural diagram of a collision avoidance driving assistance device based on multi-sensor fusion provided by an embodiment of the present invention; Figure 4 is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0018] See also Figure 1 , which shows a flowchart of an implementation of a collision avoidance driving assistance method based on multi-sensor fusion provided by an embodiment of the present invention, and is described in detail as follows: S101: Obtain point cloud data obtained by the laser radar of the current vehicle detecting surrounding obstacles.

[0019] The execution subject of this embodiment is a controller, which can be an on-board controller or an off-board controller, or a combination of an on-board controller and an off-board controller, which can be determined according to the specific situation. Here, the execution subject is an on-board controller, such as a vehicle controller unit (VCU), that is, a powertrain controller, as an example for explanation.

[0020] Specifically, the vehicle is equipped with a laser radar, a microphone array and an on-board camera. Among them, the laser radar is installed at the center of the top of the vehicle to ensure that it has a wide field of view, while avoiding being blocked by the body of the vehicle, and is convenient for daily maintenance and calibration. The laser radar is used to detect obstacles around the vehicle; the microphone array can also be set on the front side of the vehicle to detect the sound source around the vehicle. The on-board camera is installed in the front of the vehicle in a stable and fixed manner. The field of view is not blocked by the vehicle, and the camera lens should point to the front of the vehicle. Specifically, the on-board camera can be a binocular camera, a depth camera, or a combination of a binocular camera and a depth camera. The following takes a binocular camera as an example to explain the method provided in this embodiment.

[0021] Specifically, the optical axes of the two lenses of the binocular camera should be kept parallel. The binocular camera obtains the three-dimensional image around the vehicle and locates the three-dimensional position information of the obstacle. The microphone array is installed at a specific position on the roof, which can achieve 360-degree sound source collection without blind spots. The microphone array can include multiple groups of microphone arrays working together to achieve accurate judgment of the direction of the sound source.

[0022] The vehicle controller is respectively connected to the laser radar, microphone array and binocular camera to obtain data collected by the three sensors.

[0023] Specifically, the laser radar uses a triangulation ranging method, and the specific method is described as follows: (1) The laser radar emits a beam of laser, which shines onto the object being measured.

[0024] (2) The laser is reflected from the surface of the object and the reflected light is received by the lidar detector.

[0025] (3) The imaging position on the detector records the spot position of the reflected light, which together with the distance from the center of the lens to the detector forms a similar triangle.

[0026] (4) By measuring the distance between the imaging position and the center point of the detector and combining it with the known distance from the center of the lens to the detector, the distance, size, attributes and angle between the object being measured and the lidar can be calculated using the properties of similar triangles.

[0027] S102: Acquire three-dimensional image data obtained by the onboard camera of the current vehicle from detecting obstacles, and determine image positioning information of at least one obstacle based on the three-dimensional image data.

[0028] Specifically, Figure 2 As shown in the figure, the obstacle positioning method of the binocular camera is described in detail as follows: A fixed-baseline binocular vision camera is used to obtain high-definition images around the current vehicle, imitating the perception process of the human eye and human vision, observing the same scene from two viewpoints to obtain images from different perspectives, and then using the stereo imaging geometry method and triangle similarity to calculate the spatial three-dimensional coordinates of any point of the target obstacle of interest, thereby obtaining the three-dimensional features of the target obstacle. The data in the Z-axis direction is the relative distance information of the target, and the discrete point cloud formed by all three-dimensional coordinates is the relative three-dimensional image of the target.

[0029] The similarity of triangles can be used to solve the spatial three-dimensional coordinate information of any point of the target obstacle, and the mathematical relationship between depth information and image parallax can be obtained by formula derivation. Figure 2 As shown in the figure, O1 and O2 are the positions of the two cameras, and their placement angles are known. P1 and P2 are the projection points of object P on the camera imaging plane, that is, the points where P is photographed in the photo. According to the placement of the two cameras and the positions of points P1 and P2, the three-dimensional position of point P in space is calculated. The three-dimensional coordinate information of point P can be obtained based on the geometric similarity of triangles.

[0030] S103: Acquire sound source data collected by the microphone array of the current vehicle, and determine sound positioning information of the dangerous sound source based on the sound source data.

[0031] Specifically, the microphone array can determine the location of the sound source based on the strength and direction of the sound signal. The specific process is as follows: The microphone array includes two microphone arrays, which use an algorithm based on TDOA (Time Difference of Arrival) to measure the arrival angle of the sound source to array 1 (BS1) and array 2 (BS2), combined with the known straight-line distance between the two arrays, and use trigonometric geometry to accurately locate the location information of the sound source. The acquisition of the sound source signal relies on the microphone array, and the relative direction angle of the sound source is determined based on vector synthesis technology by analyzing the difference in the intensity of multi-channel acoustic signals received by different microphones.

[0032] In terms of data optimization, after collecting the sound signal, it is also necessary to perform mean filtering, Kalman filtering and low-pass filtering, which are respectively used to smooth fluctuations, real-time noise compensation in dynamic environments, and high-frequency interference suppression, thereby improving data reliability and continuity, ensuring high precision and high stability of positioning results, and laying a solid foundation for the practical application of the system.

[0033] Specifically, the sound source localization process of the microphone array is described as follows: Assume that the measurement angles of the two microphones are ; The distance between microphones is d, and the coordinates of microphones A and B are .

[0034] In the triangle formed by A, B and the target point C, the angles α and β are known (corresponding to angles CAB and CBA respectively), so the third angle can be calculated:

[0035] Using the sine theorem, we can get:

[0036] because , so we have:

[0037] Similarly: .

[0038] Thus, the formula for calculating the coordinates of the target point C with microphone A as reference is as follows:

[0039] The formula for calculating the coordinates of target point C using microphone B as a reference is as follows: .

[0040] This embodiment can also calculate the distance from the target point C to the corresponding camera through the coordinates of the target point C. In order to improve accuracy, this embodiment calculates the average value of the coordinates, angles, and distances of the same target point in multiple microphone arrays as the final sound positioning information (coordinates, angles, and distances) of the target point.

[0041] This embodiment adopts a dual-microphone array layout and combines the AOA (Angle of Arrival) algorithm to achieve high-precision sound source positioning. However, the angle measurement is interfered by the high-frequency noise of the environment, resulting in large data fluctuations. In response to this, a low-pass filter is used to suppress high-frequency noise and smooth the angle data; a mean filter is introduced to perform time-series averaging of the angle values ​​to reduce random fluctuations; and the Kalman filter is further used to dynamically optimize the angle estimation, comprehensively handle noise and uncertainty, and significantly improve the accuracy and stability of sound source positioning.

[0042] S104: fusing the point cloud data, the image positioning information, and the sound positioning information to obtain fused environment perception data of the obstacle; the environment perception data includes the position and speed of the obstacle.

[0043] In this embodiment, in order to perform multi-source data fusion, this embodiment needs to perform coordinate conversion. First, the sound localization information collected by the microphone array and the image localization information collected by the binocular camera are converted to the radar coordinate system. Then, the sound source localization coordinates and image localization coordinates converted to the radar coordinate system are synchronized by time stamps to obtain time-synchronized point cloud data, sound source localization information, and image localization information.

[0044] The specific process of converting image positioning information into the radar coordinate system includes: obtaining high-resolution RGB images and depth images through the depth camera, capturing high-precision point cloud data of obstacles through the lidar, and establishing the basic data source for multimodal perception. In the data fusion stage, the ORB (Oriented FAST and Rotated BRIEF) algorithm is used to extract key feature points from the RGB image, and these feature points are mapped to the three-dimensional space through the depth image. At the same time, the ICP (Iterative Closest Point) algorithm is used to complete the precise alignment of the lidar point cloud and depth data.

[0045] Specifically, the sound source positioning information includes the sound source intensity in addition to the polar coordinates of the sound source, and the image positioning information includes the three-dimensional coordinates of the obstacle, the obstacle type, and the distance between the obstacle and the vehicle.

[0046] After completing time synchronization and coordinate system one, the Kalman filter algorithm is used to fuse the point cloud data, sound source positioning information and image positioning information to obtain the environmental perception data of each obstacle detected after fusion. The environmental perception data includes coordinates, distance, size and type, etc. The environmental perception data of continuous frames are processed and analyzed to obtain the speed and acceleration of the obstacle.

[0047] In a possible implementation manner, another specific implementation process of S104 includes: The point cloud data, the image positioning information and the sound positioning information are input into a deep learning model, and the environmental perception data of the fused obstacle is output.

[0048] S105: For any obstacle, determine whether there is a risk of collision between the current vehicle and the obstacle based on the environmental perception data of the obstacle and the driving data of the current vehicle; and generate an anti-collision assisted driving strategy when there is a risk of collision between the current vehicle and the obstacle.

[0049] In a possible implementation, the environmental perception data includes longitudinal distance; the driving data includes speed and acceleration; the specific implementation process of S105 includes: S201: Divide the longitudinal distance from the first obstacle to the current vehicle by the first relative speed to obtain a collision time; the first obstacle is any obstacle around the current vehicle, and the first relative speed is the relative speed between the current vehicle and the first obstacle; S202: determining a minimum braking distance according to the current vehicle speed and the maximum deceleration; S203: If the collision time is less than a first preset emergency braking threshold, or the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, or the first relative speed is greater than a preset relative speed threshold, or the acceleration of the current vehicle is greater than a preset acceleration threshold, it is determined that there is a collision risk between the current vehicle and the first obstacle.

[0050] In a possible implementation manner, the specific implementation process of S105 further includes: If the collision time is not greater than a first preset emergency braking threshold and greater than a second preset emergency braking threshold, a first collision warning signal is generated; and the second preset emergency braking threshold is less than the first preset emergency braking threshold; If the collision time is not greater than the second preset emergency braking threshold and greater than a third preset emergency braking threshold, an obstacle avoidance driving route of the current vehicle is generated according to the driving data of the current vehicle and the environmental perception data of the first obstacle; and the third preset emergency braking threshold is less than the second preset emergency braking threshold; If the collision time is not greater than the third preset emergency braking threshold, an emergency braking signal is generated.

[0051] In this embodiment, when the current vehicle and the obstacle are traveling towards each other, the relative speed is the sum of the speed of the current vehicle and the speed of the obstacle. When the current vehicle and the obstacle are traveling in the same direction, the relative speed is the difference between the speed of the current vehicle and the speed of the obstacle.

[0052] In a possible implementation, the further process of S105 includes: If the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, a second collision warning signal is generated, and an adaptive cruise control (ACC) system is used to control the current vehicle to adjust the longitudinal distance between the current vehicle and the first obstacle; If the longitudinal distance between the current vehicle and the first obstacle is less than a preset distance lower limit, and no braking signal and no turning signal are detected, the current vehicle is controlled to decelerate.

[0053] In this embodiment, if the longitudinal distance between the current vehicle and the first obstacle is less than the preset distance lower limit, and no braking signal or turning signal is detected, it means that the driver has not made any response based on the risk of collision. At this time, the on-board controller can automatically perform a deceleration operation, and by decelerating, the distance between the current vehicle and the first obstacle is increased to above the safe distance.

[0054] In a possible implementation, the further process of S105 further includes: If the first relative speed is greater than a preset relative speed threshold, and the collision time is less than a fourth preset emergency braking threshold, controlling the current vehicle to steer to avoid the collision through an Electric Power Steering system (EPS); The fourth preset emergency braking threshold is smaller than the first preset emergency braking threshold and larger than the second preset emergency braking threshold.

[0055] In this embodiment, after receiving the first collision warning signal and the second collision warning signal, the vehicle controller displays the first collision warning signal and the second collision warning signal on the central control screen to remind the user to avoid obstacles. After obtaining the emergency braking signal, the vehicle controller controls the vehicle to perform emergency braking.

[0056] In a possible implementation, the further process of S105 further includes: If the acceleration of the current vehicle is greater than a preset acceleration threshold and a brake system fault code signal is detected, the current vehicle is braked by an electronic parking brake (EPB) system.

[0057] In a possible implementation manner, the specific implementation process of generating the obstacle avoidance driving route of the current vehicle according to the driving data of the current vehicle and the environmental perception data of the first obstacle includes: Determining driving data of adjacent vehicles of the current vehicle based on the vehicle group network; An obstacle avoidance driving route of the current vehicle is generated based on the driving data of the current vehicle, the driving data of the adjacent vehicles, and the environmental perception data of obstacles around the current vehicle.

[0058] In this embodiment, the driving data of the adjacent vehicles of the current vehicle is obtained through the vehicle network. At the same time, the environmental perception data of the surrounding obstacles calculated by the adjacent vehicles can also be obtained, and then the environmental perception data of the surrounding obstacles of the adjacent vehicles are converted to the radar coordinate system of the current vehicle, and the environmental perception data of the same obstacle are merged. Finally, based on the driving data of the current vehicle, the driving data of the adjacent vehicles, and the environmental perception data of the surrounding obstacles, the obstacle avoidance driving route of the current vehicle is generated. The vehicle controller automatically controls the steering, braking, driving, etc. of the vehicle according to the obstacle avoidance driving route.

[0059] Specifically, this embodiment generates an obstacle avoidance driving route for the current vehicle based on an artificial potential field method, a Dijkstra algorithm or a rapidly-exploring random tree (RRT) algorithm, as well as driving data of the current vehicle, driving data of adjacent vehicles, and environmental perception data of surrounding obstacles.

[0060] In this embodiment, after determining in step S105 that there is a risk of collision between the current vehicle and the obstacle, the controller can send the collision risk to adjacent vehicles in the Internet of Vehicles to remind the adjacent vehicles to avoid it. At the same time, the vehicle with the risk of collision can be marked in the surrounding obstacle map of the adjacent vehicles, thereby facilitating the user to monitor the approximate location of the collision risk and take preventive actions in advance.

[0061] The vehicle's surrounding obstacle map is a map drawn by the controller based on environmental perception data of obstacles around the vehicle obtained through monitoring.

[0062] Furthermore, this embodiment can send the obstacle avoidance route of the current vehicle to adjacent vehicles, so that the adjacent vehicles can take corresponding obstacle avoidance measures based on the obstacle avoidance route of the current vehicle. For example, when the obstacle avoidance path of the current vehicle is to turn left, the obstacle avoidance path of the vehicle on its left can be to decelerate. This avoids emergencies caused by the current vehicle suddenly changing its driving path and affecting other vehicles, thereby improving road safety.

[0063] In this embodiment, in order to avoid confusion in the execution of the controller when multiple signals appear at the same time, this embodiment sets priorities for various anti-collision driving signals, and the priorities are from high to low: emergency braking signal>electronic parking signal>signal for generating obstacle avoidance driving route>deceleration signal>active steering avoidance signal>collision warning signal (first collision warning signal and second collision warning signal).

[0064] It can be seen from the above embodiments that this embodiment can provide more comprehensive environmental perception capabilities and improve ranging accuracy and reliability through the multi-sensor fusion of laser radar, vehicle-mounted camera and microphone array; among them, the use of deep learning algorithms to fit multi-sensor data can improve data processing efficiency and real-time performance, ensuring that the system responds quickly to changes in complex traffic environments. In addition, this embodiment innovatively introduces sound source classification and positioning technology, which can identify potential dangerous sound sources in complex environments and accurately locate their positions. At the same time, through vehicle networking technology, real-time information exchange and collaborative obstacle avoidance between vehicles can be realized, which can improve the intelligence level of the transportation system.

[0065] It can be seen from the above embodiments that the embodiments of the present invention can be widely used in the fields of intelligent transportation systems, self-driving cars, advanced driver assistance systems (ADAS), etc. Through multi-sensor fusion and vehicle networking technology, it can significantly improve vehicle driving safety, reduce the occurrence of traffic accidents, and enhance the intelligence level of the transportation system. With the continuous development of autonomous driving technology, the application prospects of the present invention are broad and it is expected to become one of the core technologies of future intelligent transportation systems.

[0066] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0067] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.

[0068] Figure 3 The structure diagram of the anti-collision driving assistance device based on multi-sensor fusion provided by an embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows: like Figure 3 As shown, the anti-collision driving assistance device 100 based on multi-sensor fusion includes: The point cloud data acquisition module 110 is used to acquire point cloud data obtained by the laser radar of the current vehicle detecting surrounding obstacles; The image positioning information acquisition module 120 is used to acquire three-dimensional image data obtained by the vehicle-mounted camera of the current vehicle from detecting obstacles, and determine the image positioning information of at least one obstacle based on the three-dimensional image data; The sound positioning information acquisition module 130 is used to acquire the sound source data collected by the microphone array of the current vehicle, and determine the sound positioning information of the dangerous sound source based on the sound source data; A data fusion module 140 is used to fuse the point cloud data, the image positioning information and the sound positioning information to obtain fused environment perception data of the obstacle; the environment perception data includes the position and speed of the obstacle; The anti-collision driving strategy acquisition module 150 is used to determine whether there is a collision risk between the current vehicle and any obstacle based on the environmental perception data of the obstacle and the driving data of the current vehicle; and generate an anti-collision auxiliary driving strategy when there is a collision risk between the current vehicle and the obstacle.

[0069] In a possible implementation, the environmental perception data includes longitudinal distance; the driving data includes speed and acceleration; the anti-collision driving strategy acquisition module 150 includes: The longitudinal distance from the first obstacle to the current vehicle is divided by the first relative speed to obtain the collision time; the first obstacle is any obstacle around the current vehicle, and the first relative speed is the relative speed between the current vehicle and the first obstacle; Determining a minimum braking distance according to the current vehicle speed and the maximum deceleration; If the collision time is less than a first preset emergency braking threshold, or the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, or the first relative speed is greater than a preset relative speed threshold, or the acceleration of the current vehicle is greater than a preset acceleration threshold, it is determined that there is a collision risk between the current vehicle and the first obstacle.

[0070] In a possible implementation, the anti-collision driving strategy acquisition module 150 includes: an emergency braking unit, configured to generate a first collision warning signal if the collision time is not greater than a first preset emergency braking threshold and greater than a second preset emergency braking threshold; the second preset emergency braking threshold is less than the first preset emergency braking threshold; an obstacle avoidance unit, configured to generate an obstacle avoidance driving route for the current vehicle according to the driving data of the current vehicle and the environmental perception data of the first obstacle if the collision time is not greater than the second preset emergency braking threshold and greater than a third preset emergency braking threshold; the third preset emergency braking threshold is less than the second preset emergency braking threshold; If the collision time is not greater than the third preset emergency braking threshold, an emergency braking signal is generated.

[0071] In a possible implementation, the anti-collision driving strategy acquisition module 150 includes: If the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, a second collision warning signal is generated, and the current vehicle is controlled by the adaptive cruise control system to adjust the longitudinal distance between the current vehicle and the first obstacle; If the longitudinal distance between the current vehicle and the first obstacle is less than a preset distance lower limit, and no braking signal and no turning signal are detected, the current vehicle is controlled to decelerate.

[0072] In a possible implementation, the anti-collision driving strategy acquisition module 150 includes: If the first relative speed is greater than a preset relative speed threshold, and the collision time is less than a fourth preset emergency braking threshold, controlling the current vehicle to steer to avoid the collision through an electric power steering system; The fourth preset emergency braking threshold is smaller than the first preset emergency braking threshold and larger than the second preset emergency braking threshold.

[0073] In a possible implementation, the anti-collision driving strategy acquisition module 150 includes: If the acceleration of the current vehicle is greater than a preset acceleration threshold and a brake system fault code signal is detected, the current vehicle is braked by the electronic parking brake system.

[0074] In a possible implementation, the obstacle avoidance unit includes: Determining driving data of adjacent vehicles of the current vehicle based on the vehicle group network; An obstacle avoidance driving route of the current vehicle is generated based on the driving data of the current vehicle, the driving data of the adjacent vehicles, and the environmental perception data of obstacles around the current vehicle.

[0075] In a possible implementation, the data fusion module 140 includes: The point cloud data, the image positioning information and the sound positioning information are input into a deep learning model, and the environmental perception data of the fused obstacle is output.

[0076] Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, the electronic device 4 of this embodiment includes: a processor 40 and a memory 41. The memory 41 stores a computer program 42. When the processor 40 executes the computer program 42, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 40 executes the computer program 42, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0077] Exemplarily, the computer program 42 may be divided into one or more modules / units, which are stored in the memory 41 and executed by the processor 40 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 42 in the electronic device 4.

[0078] The electronic device 4 may include, but is not limited to, a processor 40 and a memory 41. Those skilled in the art will appreciate that Figure 4It is only an example of the electronic device 4 and does not constitute a limitation of the electronic device 4. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 4 may also include input and output devices, network access devices, buses, etc.

[0079] The processor 40 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0080] The memory 41 may be an internal storage unit of the electronic device 4, such as a hard disk or memory of the electronic device 4. The memory 41 may also be an external storage device of the electronic device 4, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 4. Further, the memory 41 may also include both an internal storage unit and an external storage device of the electronic device 4. The memory 41 is used to store the computer program 42 and other programs and data required by the electronic device 4. The memory 41 may also be used to temporarily store data that has been output or is to be output.

[0081] For the convenience and simplicity of description, only the division of the above functional modules / units is used as an example for illustration. In actual applications, the above functions can be assigned to different functional modules / units as needed. The above modules / units can be implemented in the form of hardware, software, or a combination of hardware and software.

[0082] The embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.

[0083] The embodiment of the present invention further provides a computer program product, including a computer program. When the computer program is executed by a processor, the methods in the above method embodiments are implemented.

[0084] The computer program includes computer program code, which may be in source code form, object code form, executable file or some intermediate form, etc. Computer readable media may include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0085] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. If there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other. The technical features in different embodiments can be combined to form a new embodiment according to their internal logical relationship.

[0086] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A collision avoidance driving assistance method based on multi-sensor fusion, characterized in that: include: Obtain the point cloud data obtained by the laser radar of the current vehicle detecting surrounding obstacles; Acquire three-dimensional image data obtained by the on-board camera of the current vehicle from obstacle detection, and determine image positioning information of at least one obstacle based on the three-dimensional image data; Acquiring sound source data collected by a microphone array of the current vehicle, and determining sound positioning information of a dangerous sound source based on the sound source data; fusing the point cloud data, the image positioning information and the sound positioning information to obtain fused environmental perception data of the obstacle; The environmental perception data includes the position and speed of obstacles; For any obstacle, determine whether there is a risk of collision between the current vehicle and the obstacle based on the environmental perception data of the obstacle and the driving data of the current vehicle; and generate an anti-collision assisted driving strategy when there is a risk of collision between the current vehicle and the obstacle.

2. The anti-collision driving assistance method based on multi-sensor fusion according to claim 1, characterized in that: The environmental perception data includes longitudinal distance; the driving data includes speed and acceleration; For any obstacle, judging whether there is a collision risk between the current vehicle and the obstacle according to the environmental perception data of the obstacle and the driving data of the current vehicle includes: The longitudinal distance from the first obstacle to the current vehicle is divided by the first relative speed to obtain the collision time; the first obstacle is any obstacle around the current vehicle, and the first relative speed is the relative speed between the current vehicle and the first obstacle; Determining a minimum braking distance according to the current vehicle speed and the maximum deceleration; If the collision time is less than a first preset emergency braking threshold, or the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, or the first relative speed is greater than a preset relative speed threshold, or the acceleration of the current vehicle is greater than a preset acceleration threshold, it is determined that there is a collision risk between the current vehicle and the first obstacle.

3. The anti-collision driving assistance method based on multi-sensor fusion according to claim 2 is characterized in that: When there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the collision time is not greater than a first preset emergency braking threshold and greater than a second preset emergency braking threshold, a first collision warning signal is generated; and the second preset emergency braking threshold is less than the first preset emergency braking threshold; If the collision time is not greater than the second preset emergency braking threshold and greater than a third preset emergency braking threshold, an obstacle avoidance driving route of the current vehicle is generated according to the driving data of the current vehicle and the environmental perception data of the first obstacle; and the third preset emergency braking threshold is less than the second preset emergency braking threshold; If the collision time is not greater than the third preset emergency braking threshold, an emergency braking signal is generated.

4. The anti-collision driving assistance method based on multi-sensor fusion according to claim 2, characterized in that: When there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the longitudinal distance between the current vehicle and the first obstacle is less than the minimum braking distance, a second collision warning signal is generated, and the current vehicle is controlled by the adaptive cruise control system to adjust the longitudinal distance between the current vehicle and the first obstacle; If the longitudinal distance between the current vehicle and the first obstacle is less than a preset distance lower limit, and no braking signal and no turning signal are detected, the current vehicle is controlled to decelerate.

5. The anti-collision driving assistance method based on multi-sensor fusion according to claim 2, characterized in that: When there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the first relative speed is greater than a preset relative speed threshold, and the collision time is less than a fourth preset emergency braking threshold, controlling the current vehicle to steer to avoid the collision through an electric power steering system; The fourth preset emergency braking threshold is smaller than the first preset emergency braking threshold and larger than the second preset emergency braking threshold.

6. The anti-collision driving assistance method based on multi-sensor fusion according to claim 2, characterized in that: When there is a risk of collision between the current vehicle and the obstacle, generating an anti-collision auxiliary driving strategy includes: If the acceleration of the current vehicle is greater than a preset acceleration threshold and a brake system fault code signal is detected, the current vehicle is braked by the electronic parking brake system.

7. The anti-collision driving assistance method based on multi-sensor fusion according to claim 3 is characterized in that: The step of generating an obstacle avoidance driving route of the current vehicle according to the driving data of the current vehicle and the environmental perception data of the first obstacle comprises: Determining driving data of adjacent vehicles of the current vehicle based on the vehicle group network; An obstacle avoidance driving route of the current vehicle is generated based on the driving data of the current vehicle, the driving data of the adjacent vehicles, and the environmental perception data of obstacles around the current vehicle.

8. The anti-collision driving assistance method based on multi-sensor fusion according to claim 1, characterized in that: The step of fusing the point cloud data, the image positioning information and the sound positioning information to obtain the fused environment perception data of the obstacle includes: The point cloud data, the image positioning information and the sound positioning information are input into a deep learning model, and the environmental perception data of the fused obstacle is output.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images