Vehicle distance calculation method and system based on multi-sensor fusion

Abstract

The present invention provides a multi-sensor fusion vehicle distance calculation method and system, the method comprising: executing a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle; executing a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle; executing a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle. The system executes the method. The present invention improves the accuracy of the obtained safe vehicle distance between the following vehicle and the preceding vehicle by fusing the vehicle distance between the following vehicle (i.e., the first vehicle) and the preceding vehicle (i.e., the second vehicle) measured by multiple sensors such as a depth camera and the vehicle distance between the following vehicle and the preceding vehicle measured by a laser radar, thereby improving the road traffic efficiency and overall traffic safety.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and in particular to a vehicle distance calculation method and system using multi-sensor fusion. Background Art

[0002] Autonomous driving is a technology that uses sensors, computers, control systems and other technologies to achieve autonomous perception, decision-making and control to complete traffic driving tasks. In the process of autonomous driving, it is very important to determine the distance from the vehicle in front. A safe distance can avoid collision accidents and maintain smooth traffic, thereby improving road efficiency and overall traffic safety.

[0003] Existing methods for measuring distance between vehicles in autonomous driving include: 1. Determine the congestion of the road section through navigation information to determine the working parameters of the imaging device; then calculate the predicted distance between vehicles through the imaging device and the vehicle body parameters; finally, filter the collected data of the radar device based on the predicted distance to determine the final distance between vehicles. 2. Collect driving data sets through cameras, and combine edge devices and sensor devices to predict driving distance. 3. Calculate the distance between vehicles by performing contour extraction in an image sequence with equal time intervals.

[0004] The existing autonomous driving distance measurement method mainly uses a multi-sensor (lidar, camera, etc.) series discrimination method. The current method can achieve good results in cloudy and overcast weather. However, the results obtained in this way are not accurate, and the accuracy depends on the last sensor in series. In addition, in some undesirable environments, such as sunny days with strong sunlight, rainy days, etc., the measurement of a sensor in the system may have a large error, and the entire system is prone to failure, which greatly reduces the reliability of the results and reduces safety. Summary of the invention

[0005] The multi-sensor fusion vehicle distance calculation method and system provided by the present invention are used to solve the problem of low road traffic safety caused by low accuracy of vehicle distance measured by existing automatic driving vehicle distance measurement methods.

[0006] The present invention provides a vehicle distance calculation method using multi-sensor fusion, comprising:

[0007] Execute a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0008] Execute a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval;

[0009] Execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0010] According to a multi-sensor fusion vehicle distance calculation method provided by the present invention, obtaining the vehicle distance between the first vehicle and the second vehicle according to the third vehicle distance and the fourth vehicle distance includes:

[0011] The vehicle distance between the first vehicle and the vehicle is obtained according to the third vehicle distance, the fourth vehicle distance, the first variance and the second variance, the first variance being the variance of the data distribution measured by the depth camera, and the second variance being the variance of the data distribution measured by the lidar.

[0012] According to a multi-sensor fusion vehicle distance calculation method provided by the present invention, determining the safe vehicle distance between the first vehicle and the second vehicle according to the fifth vehicle distance and the sixth vehicle distance includes:

[0013] The smaller value between the fifth vehicle distance and the sixth vehicle distance is used as the safe vehicle distance.

[0014] According to a multi-sensor fusion vehicle distance calculation method provided by the present invention, a method for obtaining the speed of the second vehicle at the second moment includes:

[0015] The speed of the second vehicle at the second moment is acquired according to the speed of the first vehicle at the second moment, the first vehicle distance, the second vehicle distance and the target time interval.

[0016] According to a multi-sensor fusion vehicle distance calculation method provided by the present invention, a method for obtaining the speed of the first vehicle at the second moment includes:

[0017] The speed of the first vehicle at the second moment is determined according to the distance traveled by the first vehicle from the first moment to the second moment and the target time interval measured by the odometer.

[0018] According to a multi-sensor fusion vehicle distance calculation method provided by the present invention, the first vehicle distance between the first vehicle and the second vehicle is obtained according to a depth image and an RGB image of a second vehicle acquired at a first moment by a depth camera deployed on the first vehicle, comprising:

[0019] Performing target detection on the RGB image to obtain the second vehicle and the license plate position of the second vehicle;

[0020] Obtaining an average vehicle distance between the license plate positions of the first vehicle and the second vehicle according to the depth image and the license plate position of the second vehicle;

[0021] The first vehicle distance is determined according to the average vehicle distance.

[0022] The present invention also provides a multi-sensor fusion vehicle distance calculation system, comprising:

[0023] A first acquisition module is used to perform a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle obtained at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0024] a second acquisition module, configured to execute a vehicle distance measurement process at a second moment to acquire a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: acquiring the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, wherein the third vehicle distance is the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, and the fourth vehicle distance is the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment is separated from the first moment by a target time interval;

[0025] A third acquisition module is used to execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to a speed of the second vehicle at the second moment, a speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0026] The present invention also provides an electronic device, comprising a processor and a memory storing a computer program, wherein when the processor executes the program, the vehicle distance calculation method using multi-sensor fusion as described above is implemented.

[0027] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the vehicle distance calculation method using multi-sensor fusion as described above is implemented.

[0028] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the vehicle distance calculation method using multi-sensor fusion as described above is implemented.

[0029] The multi-sensor fusion vehicle distance calculation method and system provided by the present invention improves the accuracy of the safe vehicle distance between the following vehicle and the leading vehicle by fusing the vehicle distance between the following vehicle (i.e., the first vehicle) and the leading vehicle (i.e., the second vehicle) measured by multiple sensors such as a depth camera and the vehicle distance between the following vehicle and the leading vehicle measured by a laser radar, thereby improving the road traffic efficiency and overall traffic safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 This is one of the flow charts of the vehicle distance calculation method using multi-sensor fusion provided by the present invention;

[0032] Figure 2 This is the second flow chart of the vehicle distance calculation method of multi-sensor fusion provided by the present invention;

[0033] Figure 3It is a structural schematic diagram of the vehicle distance calculation system of multi-sensor fusion provided by the present invention;

[0034] Figure 4 It is a schematic diagram of the physical structure of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] Figure 1 This is one of the flow charts of the vehicle distance calculation method of multi-sensor fusion provided by the present invention. Figure 1 The vehicle distance calculation method based on multi-sensor fusion provided by the present invention may include:

[0037] Step 110, executing a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle obtained at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0038] Step 120, executing a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval;

[0039] Step 130, executing a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment and the second moment are separated by the target time interval.

[0040] It should be noted that the execution subject of the above method may be a computer device.

[0041] Optionally, a vehicle distance measurement process is performed at a first moment to calculate the vehicle distance between a first vehicle and a second vehicle, i.e., a first vehicle distance, wherein the first vehicle is a following vehicle, the second vehicle is a front vehicle, and the first vehicle follows the second vehicle, and the following vehicle and the front vehicle can be autonomous driving vehicles or non-autonomous driving vehicles.

[0042] The vehicle distance measurement process at the first moment may specifically include obtaining the vehicle distance between the following vehicle and the vehicle in front at the first moment by taking a depth image and an RGB image of the vehicle in front at the first moment through a depth camera deployed on the following vehicle, that is, the first vehicle distance, wherein the first moment may be any moment.

[0043] The vehicle distance measurement process at the second moment is executed to calculate the vehicle distance between the first vehicle and the second vehicle, that is, the second vehicle distance. The vehicle distance measurement process at the second moment may specifically include: calculating the second vehicle distance between the first vehicle and the second vehicle according to the third vehicle distance and the fourth vehicle distance.

[0044] The third vehicle distance may be specifically the vehicle distance between the following vehicle and the vehicle in front at the second moment obtained by performing the vehicle distance measurement process of the first moment at the second moment. The fourth vehicle distance is the vehicle distance between the following vehicle and the vehicle in front at the second moment obtained by using a laser radar ranging method at the second moment. Specifically, the data measured by the laser radar deployed on the following vehicle is filtered after dynamic compensation to obtain the vehicle distance between the following vehicle and the vehicle in front at the second moment. The second moment is a moment separated from the first moment by a target time interval, and the target time interval can be set according to the refresh rate of each sensor (for example, including a depth camera, a laser radar, and an odometer) used in the present invention.

[0045] By fusing the obtained third vehicle distance and the fourth vehicle distance, the vehicle distance between the following vehicle and the front vehicle at the second moment, ie, the second vehicle distance, is obtained.

[0046] The vehicle distance measurement process at the third moment is executed to calculate the safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment may specifically include: determining the safe vehicle distance between the first vehicle and the second vehicle according to the fifth vehicle distance and the sixth vehicle distance.

[0047] Among them, the fifth vehicle distance is the vehicle distance between the front vehicle and the front vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment. The sixth vehicle distance can be specifically determined based on the speed of the front vehicle at the second moment, the speed of the following vehicle at the third moment and the second vehicle distance. The third moment is the moment separated from the second moment by a target time interval.

[0048] The safe vehicle distance between the following vehicle and the vehicle in front is calculated based on the calculated fifth vehicle distance and sixth vehicle distance.

[0049] The multi-sensor fusion vehicle distance calculation method provided by the present invention improves the accuracy of the safe vehicle distance between the following vehicle and the leading vehicle by fusing the vehicle distance between the following vehicle (i.e., the first vehicle) and the leading vehicle (i.e., the second vehicle) measured by multiple sensors such as a depth camera and the vehicle distance between the following vehicle and the leading vehicle measured by a laser radar, thereby improving the road traffic efficiency and overall traffic safety.

[0050] Further, in one embodiment, determining the safe vehicle distance between the first vehicle and the second vehicle according to the fifth vehicle distance and the sixth vehicle distance may include:

[0051] The smaller value between the fifth vehicle distance and the sixth vehicle distance is used as the safe vehicle distance.

[0052] Optionally, the fifth vehicle distance and the sixth vehicle distance obtained are compared, and the smaller value of the two is used as the safe vehicle distance between the following vehicle and the vehicle in front. If the difference between the two vehicle distances is too large, it is considered that some sensors have a problem in ranging, and an alarm is issued.

[0053] The multi-sensor fusion vehicle distance calculation method provided by the present invention calculates the safe vehicle distance between the following vehicle and the vehicle in front by fusing the vehicle distance between the following vehicle and the vehicle in front measured by multiple sensors, so as to avoid rear-end collision accidents and ensure road traffic safety.

[0054] Further, in one embodiment, acquiring the vehicle distance between the first vehicle and the second vehicle according to the third vehicle distance and the fourth vehicle distance may include:

[0055] The vehicle distance between the first vehicle and the vehicle is obtained according to the third vehicle distance, the fourth vehicle distance, the first variance and the second variance, the first variance being the variance of the data distribution measured by the depth camera, and the second variance being the variance of the data distribution measured by the lidar.

[0056] Optionally, the variance of the data distribution measured by the depth camera is calculated (i.e., the first variance, assumed to be ), calculate the variance of the data distribution measured by the lidar (i.e., the second variance, assumed to be ).

[0057] According to the third vehicle distance d c2 and the fourth vehicle distance d l2 The vehicle distance d2 (i.e., the second vehicle distance) between the first vehicle and the second vehicle is calculated by combining the following formula:

[0058]

[0059] The multi-sensor fusion vehicle distance calculation method provided by the present invention fuses the vehicle distances between the following vehicle and the leading vehicle measured by multiple sensors in the form of variance (the error of the sensor itself is roughly determined by its calibrated variance), which can improve the accuracy of the overall measured vehicle distance between the following vehicle and the leading vehicle.

[0060] Furthermore, in one embodiment, the method for obtaining the speed of the second vehicle at the second moment may include:

[0061] The speed of the second vehicle at the second moment is acquired according to the speed of the first vehicle at the second moment, the first vehicle distance, the second vehicle distance and the target time interval.

[0062] Optionally, according to the speed v of the following vehicle at the second moment a2 、First vehicle distance d t1 、Second vehicle distance d t2 And the target time interval t, calculate the speed v of the vehicle ahead at the second moment b2 :

[0063]

[0064] By adopting the method of vehicle speed estimation, a relatively ideal result can be obtained even in some undesirable environments.

[0065] Furthermore, in one embodiment, the method for obtaining the speed of the first vehicle at the second moment may include:

[0066] The speed of the first vehicle at the second moment is determined according to the distance traveled by the first vehicle from the first moment to the second moment and the target time interval measured by the odometer.

[0067] Optionally, the speed of the following vehicle at the second moment can be calculated based on the travel distance of the following vehicle from the first moment to the second moment measured by the odometer and the target time interval.

[0068] Further, in one embodiment, acquiring a first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at a first moment by a depth camera deployed on the first vehicle may include:

[0069] Performing target detection on the RGB image to obtain the second vehicle and the license plate position of the second vehicle;

[0070] Obtaining an average vehicle distance between the license plate positions of the first vehicle and the second vehicle according to the depth image and the license plate position of the second vehicle;

[0071] The first vehicle distance is determined according to the average vehicle distance.

[0072] Optionally, target recognition / detection is performed on the acquired RGB image of the front vehicle to determine the location of the front vehicle and the license plate of the front vehicle. A trained yolov5 model can be used to perform target recognition / detection on the acquired RGB image of the front vehicle, wherein the trained yolov5 model can be obtained by collecting vehicle and license plate data sets to train the yolov5 model.

[0073] According to the license plate position of the front vehicle and the depth map of the front vehicle measured by the depth camera, an average vehicle distance between the following vehicle and the license plate position of the front vehicle is calculated, and the average vehicle distance is used as the first vehicle distance.

[0074] In practical applications, the vehicle distances measured by multiple sensors are fused through the following steps, specifically:

[0075] 1. Determine the processing time interval t, i.e., the target time interval, by the refresh rates of different sensors.

[0076] 2. The first moment of vehicle distance measurement process: The vehicle in front is identified by performing target recognition on the RGB image of the vehicle in front obtained by the depth camera, and the license plate position of the vehicle in front is identified. The depth map of the vehicle in front obtained by the depth camera is used to obtain the average vehicle distance d between the following vehicle and the license plate position of the vehicle in front. c1 The distance between the vehicles is the distance d from the vehicle in front at this moment. t1 , that is, the first vehicle distance.

[0077] 3. The second time interval measurement process: Repeat the first time interval measurement process to obtain the distance d c2 , which is the third vehicle distance. The data measured by the laser radar is filtered after dynamic compensation to obtain the distance d between the following vehicle and the vehicle in front. l2 , which is the fourth vehicle distance. c2 and d l2Fusion is performed to obtain the second distance between the following vehicle and the vehicle in front at this moment The second vehicle distance is taken as the vehicle distance d between the following vehicle and the vehicle in front at this moment. t2 . Combined with the speed v of the following vehicle obtained by the odometer a2 and the distance d calculated at the previous moment t1 , solve for the speed of the vehicle ahead at the second moment The vehicle speed v b2 It can not only be used for subsequent predictions, but also for controlling the vehicle at a safe distance.

[0078] 4. The third time interval measurement process: Repeat the second time interval measurement process to obtain the distance d3 between the following vehicle and the vehicle in front, i.e., the fifth distance. Assuming that the front vehicle is moving at a constant speed in a short period of time, the speed v of the vehicle in front at the previous moment is used. b2 Combined with the speed v of the following vehicle at the third moment a3 , predict the sixth distance d between the following vehicle and the vehicle in front p3 is d p3 =(v b2 -v a3 )t+d t2 The smaller value of the fifth and sixth vehicle distances is taken as the safe distance d between the following vehicle and the vehicle in front at the current moment. t3 To avoid rear-end collisions, d t3 =min{d3,d p3 If the difference between the two vehicle distances is too large, it is considered that some sensors have distance measurement problems and an alarm is issued.

[0079] 5. Repeat the vehicle distance measurement process of the third moment at subsequent moments.

[0080] Figure 2 This is a second flow chart of the vehicle distance calculation method of multi-sensor fusion provided by the present invention, referring to Figure 2 ,include:

[0081] 1.1, using vehicle detection and license plate detection models such as the yolov5 model, collect vehicle and license plate data sets for training. Calibrate the data of the depth camera and lidar to obtain the variance of their respective data distributions. In general, the radar refresh rate < camera refresh rate < odometer refresh rate, and the appropriate time interval is determined by the refresh rate.

[0082] 1.2, the RGB image of the front vehicle measured by the depth camera is used to detect the vehicle and the license plate using yolov5. On the depth map of the depth camera, the average distance between the following vehicle and the license plate of the front vehicle is calculated as the distance at the current moment.

[0083] 1.3, repeat step 1.2. The data obtained after dynamic compensation of the data measured by the laser radar is screened to obtain the distance between the following vehicle and the vehicle in front, and the distance is combined with the distance obtained in step 1.2 at the current moment to obtain the optimized distance, that is, the second distance, which is used as the distance at the current moment, and the speed of the vehicle in front at the second moment is calculated.

[0084] 1.4, repeat step 1.3. Calculate the estimated value of the distance between the following vehicle and the vehicle in front at the current third moment by using the speed of the vehicle in front at the second moment, i.e., the sixth distance, and take the smaller value of the distance calculated by step 1.3 at the current moment (i.e., the fifth distance) and the estimated value as the safe distance at the current moment.

[0085] 1.5, repeat step 1.4.

[0086] The multi-sensor fusion vehicle distance calculation method provided by the present invention improves the existing vehicle distance calculation method, adopts the vehicle speed estimation vehicle distance and multi-sensor fusion method, which can improve the calculation accuracy of vehicle distance and adapt to more environments.

[0087] The vehicle distance calculation system of multi-sensor fusion provided by the present invention is described below. The vehicle distance calculation system of multi-sensor fusion described below and the vehicle distance calculation method of multi-sensor fusion described above can be referred to each other.

[0088] Figure 3 Schematic diagram of the structure of the multi-sensor fusion vehicle distance calculation system provided by the present invention. Figure 3 The vehicle distance calculation system of multi-sensor fusion provided by the present invention may include:

[0089] The first acquisition module 310 is used to perform a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle obtained at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0090] A second acquisition module 311 is used to perform a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, wherein the third vehicle distance is the vehicle distance between the first vehicle and the second vehicle obtained by performing the vehicle distance measurement process at the first moment at the second moment, and the fourth vehicle distance is the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment is separated from the first moment by a target time interval;

[0091] The third acquisition module 312 is used to perform a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by performing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0092] The multi-sensor fusion vehicle distance calculation system provided by the present invention improves the accuracy of the safe vehicle distance between the following vehicle and the leading vehicle by fusing the vehicle distance between the following vehicle (i.e., the first vehicle) and the leading vehicle (i.e., the second vehicle) measured by multiple sensors such as a depth camera and the vehicle distance between the following vehicle and the leading vehicle measured by a laser radar, thereby improving the road traffic efficiency and overall traffic safety.

[0093] Figure 4 is a schematic diagram of the physical structure of an electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 411, a memory 412 and a bus 413, wherein the processor 410, the communication interface 411 and the memory 412 communicate with each other through the bus 413. The processor 410 may call the logic instructions in the memory 412 to execute the following method:

[0094] Execute a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0095] Execute a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval;

[0096] Execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0097] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer power screen (which can be a personal computer, a server, or a network power screen, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0098] Furthermore, the present invention discloses a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the vehicle distance calculation method of multi-sensor fusion provided by the above-mentioned method embodiments, for example, including:

[0099] Execute a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0100] Execute a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval;

[0101] Execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0102] On the other hand, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to execute the vehicle distance calculation method of multi-sensor fusion provided in the above embodiments, for example, including:

[0103] Execute a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle;

[0104] Execute a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval;

[0105] Execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

[0106] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0107] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer power screen (which can be a personal computer, a server, or a network power screen, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0108] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle distance calculation method based on multi-sensor fusion, characterized in that: include: Execute a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle; Execute a vehicle distance measurement process at a second moment to obtain a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: obtaining the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, the third vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, the fourth vehicle distance being the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment being separated from the first moment by a target time interval; Execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to the speed of the second vehicle at the second moment, the speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

2. The vehicle distance calculation method based on multi-sensor fusion according to claim 1, characterized in that: The acquiring a second vehicle distance between the first vehicle and the second vehicle according to the third vehicle distance and the fourth vehicle distance includes: The second vehicle distance between the first vehicle and the second vehicle is obtained according to the third vehicle distance, the fourth vehicle distance, the first variance and the second variance, the first variance being the variance of the data distribution measured by the depth camera, and the second variance being the variance of the data distribution measured by the lidar.

3. The vehicle distance calculation method based on multi-sensor fusion according to claim 1, characterized in that: The determining the safe vehicle distance between the first vehicle and the second vehicle according to the fifth vehicle distance and the sixth vehicle distance includes: The smaller value between the fifth vehicle distance and the sixth vehicle distance is used as the safe vehicle distance.

4. The vehicle distance calculation method based on multi-sensor fusion according to claim 1, characterized in that: The method for obtaining the speed of the second vehicle at the second moment includes: The speed of the second vehicle at the second moment is acquired according to the speed of the first vehicle at the second moment, the first vehicle distance, the second vehicle distance and the target time interval.

5. The vehicle distance calculation method based on multi-sensor fusion according to claim 4 is characterized in that: The method for obtaining the speed of the first vehicle at the second moment includes: The speed of the first vehicle at the second moment is determined according to the distance traveled by the first vehicle from the first moment to the second moment and the target time interval measured by the odometer.

6. The vehicle distance calculation method based on multi-sensor fusion according to any one of claims 1 to 5, characterized in that: The step of acquiring a first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle acquired at the first moment by a depth camera deployed on the first vehicle includes: Performing target detection on the RGB image to obtain the second vehicle and the license plate position of the second vehicle; Obtaining an average vehicle distance between the license plate positions of the first vehicle and the second vehicle according to the depth image and the license plate position of the second vehicle; The first vehicle distance is determined according to the average vehicle distance.

7. A multi-sensor fusion vehicle distance calculation system, characterized in that: include: A first acquisition module is used to perform a vehicle distance measurement process at a first moment to obtain a first vehicle distance between a first vehicle and a second vehicle, wherein the vehicle distance measurement process at the first moment includes: obtaining the first vehicle distance between the first vehicle and the second vehicle according to a depth image and an RGB image of the second vehicle obtained at the first moment by a depth camera deployed on the first vehicle, wherein the first vehicle follows the second vehicle; a second acquisition module, configured to execute a vehicle distance measurement process at a second moment to acquire a second vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the second moment includes: acquiring the second vehicle distance between the first vehicle and the second vehicle according to a third vehicle distance and a fourth vehicle distance, wherein the third vehicle distance is the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the first moment at the second moment, and the fourth vehicle distance is the vehicle distance between the first vehicle and the second vehicle measured at the second moment according to a laser radar deployed on the first vehicle, and the second moment is separated from the first moment by a target time interval; A third acquisition module is used to execute a vehicle distance measurement process at a third moment to obtain a safe vehicle distance between the first vehicle and the second vehicle, wherein the vehicle distance measurement process at the third moment includes: determining a safe vehicle distance between the first vehicle and the second vehicle according to a fifth vehicle distance and a sixth vehicle distance, the fifth vehicle distance being the vehicle distance between the first vehicle and the second vehicle obtained by executing the vehicle distance measurement process at the second moment at the third moment, the sixth vehicle distance being determined according to a speed of the second vehicle at the second moment, a speed of the first vehicle at the third moment, and the second vehicle distance, and the third moment is separated from the second moment by the target time interval.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the vehicle distance calculation method based on multi-sensor fusion according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the vehicle distance calculation method based on multi-sensor fusion as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the vehicle distance calculation method based on multi-sensor fusion as claimed in any one of claims 1 to 6 is implemented.

Images