Fusion target processing method, device, equipment and storage medium

Abstract

The embodiments of the present application provide a fusion target processing method, device, equipment and storage medium. The method includes: determining the coordinate error of the fusion target based on the set true value data, and then calculating the expanded size of the fusion target based on the size and coordinate error of the fusion target; determining the final target from the fusion target based on the intersection and union ratio between at least two fusion targets under the expanded size. The embodiments of the present application solve the problem that target splitting is prone to occur in the target detection and fusion process in the prior art, resulting in insufficient vehicle safety, avoids the recognition error of the size and position of the fusion target due to sensor measurement error, increases the size of the fusion target that can be used to calculate the intersection and union ratio, and improves the reliability of the calculation result, thereby greatly reducing the problem of target splitting caused by sensor error and improving the safety of vehicle driving.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of data processing technology, and in particular to a fusion target processing method, device, equipment and storage medium. Background Art

[0002] With the emergence of diversified transportation needs, autonomous driving technology has gradually been more widely used. Autonomous driving vehicles are usually equipped with a variety of different sensors (such as cameras, lidar, millimeter wave radar, etc.) to detect objects around the vehicle in real time during driving, and realize vehicle positioning, position control and other operations through target detection algorithms.

[0003] However, different sensors have differences in the position and accuracy of target detection around the vehicle, which leads to target splitting (that is, two or more targets are detected simultaneously for the same object) when the target detection results based on different sensors are fused. As a result, the vehicle cannot effectively achieve its own positioning and position control, and may also cause the vehicle to brake, affecting the vehicle safety. Summary of the invention

[0004] The embodiments of the present application provide a fusion target processing method, device, equipment and storage medium to solve the problem that target splitting is prone to occur during target detection and fusion in the prior art, resulting in insufficient vehicle safety.

[0005] In a first aspect, an embodiment of the present application provides a fusion target processing method, and the fusion target processing method includes:

[0006] Based on the set true value data, determine the coordinate error of the fusion target, the coordinate error includes the lateral error and the longitudinal error, and there are at least two fusion targets;

[0007] Based on the size and coordinate error of the fused object, the expanded size of the fused object is calculated;

[0008] A final object is determined from the fused objects based on an intersection-over-union ratio between at least two fused objects at the expanded size.

[0009] It can be seen that by obtaining the true value data and comparing it with the fusion target obtained by sensor fusion, the coordinate error of the fusion target is obtained, and the fusion target is expanded, and then the final target is determined by the intersection and union ratio between the fusion targets. Therefore, the error of the true value evaluation and the speed difference between the fusion targets can be combined to filter out the fusion targets that are split, reduce the target splitting problem after the target fusion processing, increase the size of the fusion target in the calculation through expansion processing, avoid the recognition error of the fusion target size and position due to sensor measurement error, increase the size of the fusion target that can be used to calculate the intersection and union ratio, and improve the reliability of the calculation results, thereby greatly reducing the target splitting problem caused by sensor error and improving the safety of vehicle driving.

[0010] Optionally, based on the set true value data, the coordinate error of the fused target is determined, including: obtaining the true value data corresponding to the fused target, the true value data including the real coordinates and real size of the actual target corresponding to the fused target; based on the relative distance from the fused target to the vehicle and the true value data, determining the lateral error and longitudinal error of the fused target at the relative distance.

[0011] It can be seen that the lateral error and longitudinal error of the fused target can be determined by comparing the true value data with the sensor data; by combining the relative distance between the fused target and the vehicle, different lateral errors and longitudinal errors can be used for calculation under different distance conditions to ensure the real-time and accuracy of the lateral error and longitudinal error selection, thereby ensuring the accuracy and reliability of subsequent calculation results.

[0012] Optionally, the size and coordinate error of the fused target are used to calculate the expanded size of the fused target, including: determining the longitudinal expansion size of the fused target based on the speed difference between at least two fused targets; determining the lateral expansion size of the fused target based on the coordinate error; determining the expanded size of the fused target based on the fused target size, the longitudinal expansion size and the lateral expansion size.

[0013] It can be seen that by combining the speed difference between the fusion targets to determine the longitudinal expansion size, the influence of the vehicle speed on the sensor error is fully considered, and the accuracy of the final result is further guaranteed; by adding the longitudinal expansion size and the lateral expansion size on the basis of the fusion target size, the fusion target size used to calculate the intersection-over-merge ratio is made larger, and the influence of the sensor error is fully considered, and the probability of calculating the intersection-over-merge ratio becomes larger, thereby increasing the probability of removing the split fusion target, and further ensuring the safety of vehicle driving.

[0014] Optionally, based on the speed difference between at least two fusion targets, the longitudinal expansion size of the fusion target is determined, including: determining a reference value of the longitudinal expansion size of the fusion target based on the speed difference and a preset influencing factor; and determining the maximum value between the reference value and the longitudinal error as the longitudinal expansion size of the fusion target.

[0015] It can be seen that by selecting a maximum value from the reference value and the longitudinal error as the longitudinal expansion size, the problem of the sensor error not being fully reflected when the vehicle speed is low and the monitored speed difference is small, which in turn affects the calculation of the intersection-and-combination ratio, is avoided. This effectively ensures that the sensor error factor is not omitted, thereby ensuring the reliability of the calculation results and the safety of vehicle driving.

[0016] Optionally, determining the lateral expansion size of the fusion target based on the coordinate error includes: determining the lateral error as the lateral expansion size of the fusion target.

[0017] It can be seen that by determining the lateral error as the lateral expansion size of the fusion target, the error caused by the sensor is combined to ensure the reliability of the calculation result.

[0018] Optionally, based on the fusion target size, the longitudinal expansion size and the lateral expansion size, the expanded size of the fusion target is determined, including: taking the sum of the longitudinal size and the longitudinal expansion size of the fusion target as the longitudinal size of the fusion target after expansion; taking the sum of the lateral size and the lateral expansion size of the fusion target as the lateral size of the fusion target after expansion.

[0019] It can be seen that by adding the longitudinal expansion size and the lateral expansion size to the original size of the fusion target, the size of the fusion target used for calculation is increased, so that when calculating the intersection-over-union ratio, the fusion targets that originally did not have an intersection may have an intersection due to the increase in size, or the intersection-over-union ratio corresponding to the intersection is increased, thereby increasing the probability of finding a split target that can be excluded from multiple fusion targets and improving the safety of vehicle driving.

[0020] Optionally, based on the intersection-and-union ratio between at least two fusion targets at the expanded size, the final target is determined from the fusion targets, including: determining its confidence based on the sensor corresponding to the fusion target; if the intersection-and-union ratio of at least two fusion targets at the expanded size is greater than a set value, determining the fusion target with higher confidence as the final target; if the intersection-and-union ratio of at least two fusion targets at the expanded size is less than a set value, determining that at least two fusion targets are both the final targets.

[0021] It can be seen that by comparing the intersection-and-union ratio with the set value, when the intersection-and-union ratio is large, it means that the positions of the fused targets are close. Due to the existence of sensor errors, if the positions of different fused targets are close or mostly overlapped, then these fused targets are very likely to be split targets corresponding to the same actual target. Therefore, some fused targets can be directly excluded according to the confidence of the fused target, thereby achieving the effect of reducing target splitting, thereby reducing erroneous braking during vehicle driving and improving vehicle driving safety.

[0022] In a second aspect, an embodiment of the present application provides a fusion target processing device, the fusion target processing device comprising:

[0023] An analysis module is used to determine the coordinate error of the fusion target based on the set true value data. The coordinate error includes a lateral error and a longitudinal error. There are at least two fusion targets.

[0024] A calculation module, used for calculating the expanded size of the fused object based on the size and coordinate error of the fused object;

[0025] The determination module is used to determine the final target from the fused targets based on the intersection-over-union ratio between at least two fused targets at the expanded size.

[0026] Optionally, the analysis module is specifically used to obtain true value data corresponding to the fusion target, the true value data including the real coordinates and real size of the actual target corresponding to the fusion target; based on the relative distance from the fusion target to the vehicle and the true value data, determine the lateral error and longitudinal error of the fusion target at the relative distance.

[0027] Optionally, the calculation module is specifically used to determine the longitudinal expansion size of the fused target based on the speed difference between at least two fused targets; determine the lateral expansion size of the fused target based on the coordinate error; and determine the expanded size of the fused target based on the fused target size, the longitudinal expansion size and the lateral expansion size.

[0028] Optionally, the calculation module is specifically used to determine a reference value of the longitudinal expansion size of the fusion target based on the speed difference and a preset influencing factor; and determine the maximum value between the reference value and the longitudinal error as the longitudinal expansion size of the fusion target.

[0029] Optionally, the calculation module is specifically configured to determine a maximum value of the lateral errors as a lateral expansion size of the fusion target.

[0030] Optionally, the calculation module is specifically used to use the sum of the longitudinal size and the longitudinal expansion size of the fusion target as the longitudinal size of the fusion target after expansion; and use the sum of the transverse size and the transverse expansion size of the fusion target as the transverse size of the fusion target after expansion.

[0031] Optionally, the determination module is specifically used to determine the confidence of the fusion target based on the sensor corresponding to the fusion target; if the intersection and union ratio of at least two fusion targets at the expanded size is greater than a set value, the fusion target with higher confidence is determined to be the final target; if the intersection and union ratio of at least two fusion targets at the expanded size is less than a set value, it is determined that at least two fusion targets are both the final targets.

[0032] In a third aspect, an embodiment of the present application further provides a control device, the control device comprising:

[0033] at least one processor;

[0034] and a memory communicatively coupled to the at least one processor;

[0035] Among them, the memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable the control device to execute a fusion target processing method corresponding to any embodiment in the first aspect of the embodiments of the present application.

[0036] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by a processor, they are used to implement any fusion target processing method as described in the first aspect of the embodiment of the present application.

[0037] In a fifth aspect, an embodiment of the present application further provides a computer program product, which includes computer execution instructions. When the computer execution instructions are executed by a processor, they are used to implement a fusion target processing method of any embodiment corresponding to the first aspect of the embodiment of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 An application scenario diagram of the fusion target processing method provided in an embodiment of the present application;

[0039] Figure 2 A flowchart of a fusion target processing method provided for one embodiment of the present application;

[0040] Figure 3a A flowchart of a fusion target processing method provided in yet another embodiment of the present application;

[0041] Figure 3b for Figure 3a A flowchart of calculating the longitudinal expansion size of a fusion target provided in the illustrated embodiment;

[0042] Figure 4 A schematic diagram of the structure of a fusion target processing device provided in yet another embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a control device provided in yet another embodiment of the present application. DETAILED DESCRIPTION

[0044] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the embodiments of the present application. Instead, they are only examples of devices and methods consistent with some aspects of the embodiments of the present application as detailed in the attached claims.

[0045] The following specific embodiments are used to describe in detail the technical solutions of the embodiments of the present application and how the technical solutions of the embodiments of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0046] With the emergence of diversified transportation needs, autonomous driving technology has gradually been more widely used. Autonomous driving vehicles are usually equipped with a variety of different sensors (such as cameras, lidar, millimeter wave radar, etc.) to detect objects around the vehicle in real time during driving, and realize vehicle positioning, position control and other operations through target detection algorithms.

[0047] However, different sensors have different positions and accuracies in detecting targets around the vehicle. For example, the longitudinal position deviation of the camera is large, and the lateral deviation of the millimeter-wave radar is large. As a result, when the results of target detection based on different sensors are fused, target splitting is likely to occur (that is, two or more targets are detected simultaneously for the same object), resulting in the vehicle being unable to effectively achieve its own positioning and position control. If the speed difference between the split fusion targets is large, it may also cause the vehicle to brake (because the position and speed of the vehicle cannot be accurately controlled at this time, so braking is required to ensure driving safety). This type of abnormal braking will seriously affect the safety of the vehicle and reduce the riding experience of the autonomous vehicle.

[0048] In order to solve the above problems, an embodiment of the present application provides a fusion target processing method, which analyzes the positions between fusion targets and filters the split fusion targets according to whether there is an intersection between the fusion targets and the size of the intersection-union ratio, so as to ensure the reliability of the fusion targets and further ensure the reliability of vehicle driving.

[0049] Figure 1 This is an application scenario diagram of the fusion target processing method provided in the embodiment of the present application. Figure 1As shown, in the fusion target processing flow, the vehicle control unit 100 obtains the measurement data transmitted by multiple sensors 110, obtains the coordinates and size information of the fusion target through fusion processing, and then obtains the true value data of the actual target object corresponding to the fusion target transmitted by the server 120, and processes the fusion target to reduce target splitting and improve vehicle driving safety.

[0050] It should be noted that Figure 1 In the scenario shown, only one vehicle control unit, sensor, and server are used as an example for illustration, but the embodiments of the present application are not limited to this, that is, the number of vehicle control units, sensors, and servers can be arbitrary.

[0051] The fusion target processing method provided by the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0052] Figure 2 This is a flow chart of a fusion target processing method provided in one embodiment of the present application. Figure 2 As shown, including but not limited to the following steps:

[0053] Step S201: Determine the coordinate error of the fusion target based on the set true value data.

[0054] The coordinate error includes lateral error and longitudinal error, and there are at least two fusion targets.

[0055] Specifically, the fusion target is the target obtained by the vehicle's processor based on multiple sensors detecting objects (i.e., targets) around the vehicle's driving environment, and then fusing the target coordinates and sizes detected by multiple sensors through a target fusion algorithm. The fusion target includes the coordinates and sizes of the fused target (the size here refers to the length and width of the rectangular box / border that can contain the target object, rather than the actual shape and size of the irregular target object).

[0056] By fusing the target, the vehicle can easily locate its position and relative speed during driving and adjust its driving state. For example, if the distance between the fusion target and the vehicle is too close, the vehicle needs to change its driving state (such as acceleration, steering, braking, etc.) to deviate from the too close fusion target.

[0057] The fusion processing method may select any existing fusion processing algorithm, such as a clustering algorithm, a weighted average algorithm, and the like.

[0058] When a vehicle has three or more sensors, the fusion process may fuse the targets detected by any two or three sensors. If the detection results (target coordinates and sizes) of different sensors are the same, the fused targets obtained by the fusion process will overlap, that is, it can be considered that only one fused target is actually obtained in the end; but if the detection results of different sensors are different, the fused targets obtained by the fusion process will not overlap, but there may be multiple targets with different positions and sizes, that is, target splitting occurs.

[0059] In order to solve the problem of target splitting, it is necessary to process the fusion target obtained by fusion processing during the vehicle testing phase, and determine the sensor combination used to generate the fusion target when applying it, so as to avoid the problem of target splitting to the greatest extent possible, and ensure that there will be no problem of vehicle incorrect braking and driving safety affected due to target splitting during the application phase.

[0060] At this time, the vehicle can be driven in the test environment, and the fused target detected by the vehicle can be compared with the actual coordinates and size (i.e., true value data) of the target corresponding to the fused target that is predetermined in the test environment, and the fused target can be further processed accordingly to solve the target splitting problem.

[0061] In actual testing, the vehicle needs to be driven and tested in different test environments multiple times to ensure the safety and reliability of further processing results. However, for the convenience of description in this solution, only the processing process in one test is described.

[0062] By comparing (subtracting) the coordinates and size of the fused target with the true value data, the coordinate error of the fused target can be obtained.

[0063] Step S202: Calculate the expanded size of the fused object based on the size and coordinate error of the fused object.

[0064] Specifically, in order to eliminate the influence of the measurement error of the sensor, the corresponding value (or the absolute value of the value) of the coordinate error may be added to the size of the fusion target.

[0065] Since this process will increase the size of the fusion target, it is called dilation processing of the fusion target. The size of the fusion target after the dilation process is the dilated size of the fusion target.

[0066] Step S203: determining a final target from the fused targets based on an intersection-over-union ratio between at least two fused targets at the expanded size.

[0067] Specifically, since the emergence of multiple fusion targets is caused by the detection error of the sensor, if the multiple fusion targets after the expansion processing can have more overlaps (that is, the intersection and union ratio is greater than the set value), it can be considered that these fusion targets correspond to the same actual target. Therefore, according to the confidence of these targets, a target with the highest confidence can be selected as the final target.

[0068] If these fusion targets do not overlap even after expansion, then these fusion targets are more likely to correspond to different actual targets rather than the same actual target. Therefore, different final targets can be determined for these fusion targets respectively.

[0069] After the final target is determined, the fusion target corresponding to the final target can be used as the fusion target selected as a reference during actual driving of the vehicle. When the vehicle needs to generate a fusion target through sensors during actual driving, it is no longer necessary to arbitrarily select a combination of different sensors, but directly select a combination of sensors corresponding to the final target.

[0070] The fusion target processing method provided in the embodiment of the present application obtains the true value data and compares it with the fusion target obtained by sensor fusion to obtain the coordinate error of the fusion target, and performs expansion processing on the fusion target, and then determines the final target through the intersection and union ratio between the fusion targets. Therefore, the error of the true value evaluation and the speed difference between the fusion targets can be combined to filter out the fusion targets that are split, reduce the target splitting problem that occurs after the target fusion processing, increase the size of the fusion target in the calculation through expansion processing, avoid the recognition error of the fusion target size and position due to the sensor measurement error, increase the size of the fusion target that can be used to calculate the intersection and union ratio, and improve the reliability of the calculation result, thereby greatly reducing the target splitting problem caused by sensor error and improving the safety of vehicle driving.

[0071] Figure 3a The following is a flow chart of a fusion target processing method provided in an embodiment of the present application. Figure 3a As shown, the fusion target processing method provided in this embodiment includes the following steps:

[0072] Step S301: Obtain the true value data corresponding to the fusion target.

[0073] The true value data includes the real coordinates and real size of the fused target corresponding to the actual target.

[0074] Specifically, the fusion target is the data measured by the vehicle based on the sensors, which is fused to obtain the size and coordinates of the target object. On this basis, if multiple sensors obtain multiple fusion targets of the same object (or overlapping or similar positions but not actually the same object) at the same time, the fusion target needs to be further processed in order to effectively guide the vehicle to control its own position and driving status.

[0075] The true value data is a predetermined actual value. When the vehicle's sensor obtains the sensor data corresponding to the target object and obtains the corresponding size and coordinates of the target object through fusion processing, the predetermined true value data can be queried based on the coordinates obtained by the fusion processing to determine the real coordinates and real data of the actual target corresponding to the target object.

[0076] In some embodiments, the true value data of predetermined (multiple) target objects around the vehicle can also be acquired in real time based on the real-time position of the vehicle, and the true value data of the corresponding target objects can be called for comparison based on the sensor data (coordinates).

[0077] The true value data is usually stored in the server, and the vehicle's control system (or vehicle control unit) can download it from the server in real time as needed; it can also be pre-downloaded to the vehicle control unit for a certain test environment, and the corresponding true value data can be called according to the real-time position of the vehicle for comparison and evaluation.

[0078] Step S302: Based on the relative distance from the fusion target to the vehicle and the true value data, determine the lateral error and longitudinal error of the fusion target at the relative distance.

[0079] Specifically, when the relative distance between the fusion target and the vehicle is different, the error generated by the sensor is different. Therefore, on the one hand, it is necessary to record the corresponding relative distance when calculating the lateral error and longitudinal error of the fusion target; on the other hand, the vehicle needs to be tested multiple times, and the test in this embodiment is repeated when the relative distance relative to the same actual target (generated fusion target) is different to determine how to select the fusion target as the final target at different relative distances, so as to maximize the guarantee that when the vehicle is at different relative distances from the fusion target, the final target (coordinates and size) obtained can be as close to the actual target as possible, thereby ensuring the safety of vehicle driving.

[0080] Step S303: Determine the longitudinal expansion size of the fused object based on the speed difference between at least two fused objects.

[0081] Specifically, due to the difference in coordinates between the fusion targets, their relative directions with the vehicle are different, and their absolute values ​​(or speed values) and directions of relative speeds relative to the vehicle are usually different. Therefore, there will be a speed difference between different fusion targets, that is, there may be two fusion targets with the same absolute value of their relative speeds relative to the vehicle, but different directions, such as moving away from the vehicle in two different directions respectively. In this case, there will be a relative speed between the two fusion targets, which is the speed difference. It is also possible that there are differences in both relative speeds and directions between different fusion targets, in which case there will obviously be a speed difference.

[0082] Since the speed difference can reflect both the moving speed of the vehicle relative to the fusion target and the error of the sensor, the speed difference is combined to determine the expansion size of the fusion target relative to the vehicle direction (i.e., longitudinal direction), so that the calculated expansion size can better reflect factors such as the relative speed of the sensor, vehicle and fusion target, thereby improving the reliability of the calculation results.

[0083] Furthermore, if Figure 3b As shown, it is a flow chart for calculating the longitudinal expansion size of the fusion target, which includes the following steps:

[0084] Step S3031: Determine a reference value of the longitudinal expansion size of the fusion target based on the speed difference and a preset influencing factor.

[0085] Specifically, the influence factor is a correction coefficient for the speed difference. By multiplying the influence factor by the speed difference, a reference value of the longitudinal expansion size of the fusion target is obtained.

[0086] This reference value is used to compare with the longitudinal error so that when the longitudinal error is not obvious (for example, the fused target is close to the vehicle, resulting in a small error, but the actual speed of the vehicle is very fast), it can fully reflect the possible deviation between the fused target and the actual target. Because at this time, the deviation is more likely to be caused by the relative speed between the vehicle and the fused target, rather than the sensor deviation. If only the longitudinal error is considered, the relative speed between the vehicle and the fused target will be ignored, resulting in a large difference between the result and the actual situation.

[0087] Step S3032: determine the maximum value between the reference value and the longitudinal error as the longitudinal expansion size of the fusion target.

[0088] Specifically, if the reference value is greater than the longitudinal error, the reference value should be determined as the longitudinal expansion size. The reason is as described in the analysis of the previous step. If the longitudinal error is greater than the reference value, the longitudinal error should be determined as the longitudinal expansion size, because at this time the vehicle is usually far away from the target object corresponding to the fusion target, and the measurement error of the sensor is the main factor. In order to reduce the impact of this error, it is necessary to perform longitudinal expansion processing on the fusion target size based on the longitudinal error.

[0089] Therefore, by comparing the reference value with the longitudinal error, factors such as the relative position and relative speed (the speed difference caused by the above two factors) and sensor error between the fusion target and the vehicle can be fully considered to ensure the reliability of the calculation results.

[0090] Step S304: Determine the lateral expansion size of the fused object based on the coordinate error.

[0091] Specifically, since the lateral direction of the fused target is perpendicular to the driving direction of the vehicle and is not affected by the relative speed between the vehicle and the fused target (or the impact is minimal), the lateral error can be directly determined as the lateral expansion size of the fused target.

[0092] Step S305: Determine the expanded size of the fused target based on the fused target size, the longitudinal expansion size, and the lateral expansion size.

[0093] Specifically, after the longitudinal expansion size and the lateral expansion size are determined, the fusion target can be expanded based on the longitudinal expansion size and the lateral expansion size.

[0094] The specific expansion processing method may be to use the sum of the longitudinal size of the fused target and the longitudinal expansion size as the longitudinal size of the fused target after expansion; and then use the sum of the transverse size of the fused target and the transverse expansion size as the transverse size of the fused target after expansion.

[0095] The longitudinal size and lateral size of the fused target, i.e., the longitudinal size and lateral size of the target object measured by the sensor obtained by the vehicle through fusion processing in step S301. In actual calculation, based on the coordinates of the fused target as the center, half of the longitudinal expansion size can be added to both ends of the longitudinal size of the fused target, and half of the lateral expansion size can be added to both ends of the lateral size of the fused target, as the longitudinal size and lateral size of the fused target after expansion.

[0096] By using the fused target after expansion processing for analysis, the influence of coordinate and size deviations caused by sensor measurement errors can be reduced, and the robustness and reliability of the calculation results can be improved.

[0097] Step S306: Determine the confidence level of the fusion target based on the sensor corresponding to the fusion target.

[0098] Specifically, since the measurement results of different sensors have different confidence levels, such as the lateral and longitudinal deviations detected by the lidar sensor are relatively small, the confidence level is higher, while the confidence levels of the camera and millimeter-wave radar are relatively low. Therefore, the corresponding confidence levels of the fusion targets will vary depending on the sensors they are fused with.

[0099] In some embodiments, the confidence of the fusion target can be obtained by multiplying the confidence of the fusion sensor on which it is based (for example, if the confidences of the two sensors corresponding to the fusion target are T1 and T2, respectively, then the confidence of the fusion target is T(1X2)=T2).

[0100] In some embodiments, the confidence of the fusion target may also be calculated using other existing confidence calculation methods for fusion targets without affecting subsequent calculations.

[0101] Step S307: If the intersection-over-union ratio of at least two fusion targets at the expanded size is greater than a set value, the fusion target with a higher confidence level is determined as the final target.

[0102] Specifically, the intersection ratio refers to the ratio of the area of ​​intersection (this area can be easily calculated when the coordinates and sizes of the center points of the two fused targets are determined) in the area enclosed by the bounding boxes of the two fused targets (see the description in step S201) to the area of ​​union. The higher the ratio, the higher the overlap of the two fused targets, and the more likely it is that the two fused targets correspond to the same actual target.

[0103] The setting value can be 0. At this time, when the two fusion targets overlap, it means that there is an overlapping part between the two fusion targets. Therefore, the split targets can be minimized to ensure the reliability of the results. The setting value can also be a value greater than 0 and less than 1. At this time, the two fusion targets need to have a large overlap to be considered to correspond to the same target object, so as to retain the sensor data and improve the accuracy of the fusion target when the overall confidence of the sensor is high.

[0104] For overlapping targets, they can be directly considered to correspond to the same target object. At this time, the fusion target with the highest confidence can be retained as the final target. This can quickly filter out the split targets, improve the robustness of the target fusion results, and ensure the stability of vehicle driving.

[0105] Step S308: If the intersection-over-union ratio of at least two fused targets at the expanded size is less than a set value, it is determined that the at least two fused targets are both final targets.

[0106] Specifically, since the fusion target may actually be different targets or multiple targets with similar positions (one sensor measures a single parked bicycle, and the other sensor measures another bicycle with the same color and adjacent to the bicycle), the fusion process may result in two fusion targets with low overlap or no overlap, that is, a fusion target with an intersection-over-union ratio less than a set value. In this case, both fusion targets should be directly used as the final target instead of being directly fused to ensure the accuracy of the result and avoid erroneous filtering that affects the vehicle's own control and driving safety.

[0107] The fusion target processing method provided in the embodiment of the present application obtains the true value data corresponding to the fusion target, calculates the coordinate error at different relative distances from the vehicle, then expands the fusion target based on the coordinate error, and determines the final target based on the intersection-and-union ratio and confidence between the fusion targets after the expansion process. Thus, the relative speed between the vehicle and the fusion target, the relative position between the fusion target and the vehicle, the error of the sensor and other factors can be combined to jointly analyze whether the fusion target is a split target corresponding to the same target object, and split target filtering can be implemented, thereby achieving effective processing of the fusion target, reducing vehicle erroneous braking, and improving vehicle driving safety.

[0108] Figure 4 This is a schematic diagram of the structure of a fusion target processing device provided in an embodiment of the present application. Figure 4 As shown, the fusion target processing device 400 includes: an analysis module 410, a calculation module 420 and a determination module 430. Among them:

[0109] An analysis module 410 is used to determine the coordinate error of the fusion target based on the set true value data, the coordinate error includes a lateral error and a longitudinal error, and there are at least two fusion targets;

[0110] A calculation module 420, configured to calculate the expanded size of the fused object based on the size and coordinate error of the fused object;

[0111] The determination module 430 is configured to determine a final target from the fused targets based on an intersection-over-union ratio between at least two fused targets at the expanded size.

[0112] Optionally, the analysis module 410 is specifically used to obtain true value data corresponding to the fusion target, the true value data including the real coordinates and real size of the actual target corresponding to the fusion target; based on the relative distance from the fusion target to the vehicle and the true value data, determine the lateral error and longitudinal error of the fusion target at the relative distance.

[0113] Optionally, the calculation module 420 is specifically used to determine the longitudinal expansion size of the fused target based on the speed difference between at least two fused targets; determine the lateral expansion size of the fused target based on the coordinate error; and determine the expanded size of the fused target based on the fused target size, the longitudinal expansion size and the lateral expansion size.

[0114] Optionally, the calculation module 420 is specifically configured to determine a reference value of the longitudinal expansion size of the fusion target based on the speed difference and a preset influencing factor; and determine a maximum value between the reference value and the longitudinal error as the longitudinal expansion size of the fusion target.

[0115] Optionally, the calculation module 420 is specifically configured to determine a maximum value of the lateral errors as a lateral expansion size of the fusion target.

[0116] Optionally, the calculation module 420 is specifically configured to use the sum of the longitudinal size and the longitudinal expansion size of the fused target as the longitudinal size of the fused target after expansion; and use the sum of the transverse size and the transverse expansion size of the fused target as the transverse size of the fused target after expansion.

[0117] Optionally, the determination module 430 is specifically used to determine the confidence of the fusion target based on the sensor corresponding to the fusion target; if the intersection and union ratio of at least two fusion targets at the expanded size is greater than a set value, the fusion target with higher confidence is determined to be the final target; if the intersection and union ratio of at least two fusion targets at the expanded size is less than a set value, it is determined that at least two fusion targets are both the final targets.

[0118] In this embodiment, the fusion target processing device can solve the problem of target splitting that is prone to occur in the target detection fusion process in the prior art, resulting in insufficient vehicle safety, through the combination of various modules. It can greatly reduce the target splitting problem caused by sensor errors and improve the safety of vehicle driving.

[0119] Figure 5 A schematic diagram of a control device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the control device 500 includes: a memory 510 and a processor 520 .

[0120] The memory 510 stores a computer program that can be executed by at least one processor 520. The computer program is executed by at least one processor 520 to enable the control device to implement the fusion target processing method provided in any of the above embodiments.

[0121] The memory 510 and the processor 520 may be connected via a bus 530 .

[0122] The relevant instructions can be understood by referring to the relevant descriptions and effects corresponding to the method embodiments, which will not be repeated here.

[0123] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the following Figures 2 to 3b The corresponding fusion target processing method of any embodiment.

[0124] Among them, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0125] An embodiment of the present application provides a computer program product, which includes computer-executable instructions, which are used to implement the following when the computer-executable instructions are executed by a processor: Figures 2 to 3bThe corresponding fusion target processing method of any embodiment.

[0126] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or modules, which can be electrical, mechanical or other forms.

[0127] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the disclosure disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope of the present application is indicated by the claims.

[0128] It should be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for processing fused targets, characterized in that, the method for processing fused targets includes: Based on the set ground truth data, determining the coordinate error of the fused target, where the coordinate error includes a lateral error and a longitudinal error, and there are at least two fused targets; Based on the size of the fused target and the coordinate error, calculating the size of the fused target after inflation; Based on the intersection over union (IoU) between at least two fused targets under the inflated size, determining the final target from the fused targets; The determining the coordinate error of the fused target based on the set ground truth data includes: Obtaining the ground truth data corresponding to the fused target, where the ground truth data includes the true coordinates and true size of the actual target corresponding to the fused target; Based on the relative distance from the fused target to the host vehicle and the ground truth data, determining the lateral error and longitudinal error of the fused target at the relative distance; The calculating the size of the fused target after inflation based on the size of the fused target and the coordinate error includes: Based on the speed difference between at least two fused targets, determining the longitudinal inflation size of the fused target; Based on the coordinate error, determining the lateral inflation size of the fused target; Based on the size of the fused target, the longitudinal inflation size, and the lateral inflation size, determining the size of the fused target after inflation; The determining the longitudinal inflation size of the fused target based on the speed difference between at least two fused targets includes: Based on the speed difference and a preset influence factor, determining a reference value for the longitudinal inflation size of the fused target; Determining the maximum value of the reference value and the longitudinal error as the longitudinal inflation size of the fused target; The determining the lateral inflation size of the fused target based on the coordinate error includes: Determining the lateral error as the lateral inflation size of the fused target.

2. The method for processing fused targets according to claim 1, characterized in that, the determining the size of the fused target after inflation based on the size of the fused target, the longitudinal inflation size, and the lateral inflation size includes: Taking the sum of the longitudinal size of the fused target and the longitudinal inflation size as the longitudinal size of the fused target after inflation; Taking the sum of the lateral size of the fused target and the lateral inflation size as the lateral size of the fused target after inflation.

3. The method for processing fused targets according to claim 1 or 2, characterized in that, the determining the final target from the fused targets based on the intersection over union (IoU) between at least two fused targets under the inflated size includes: Based on the sensor corresponding to the fused target, determining its confidence level; If the intersection over union (IoU) of at least two fused targets under the inflated size is greater than the set value, determining the fused target with a higher confidence level as the final target; If the intersection over union (IoU) of at least two fused targets under the inflated size is less than the set value, determining that at least two fused targets are both final targets.

4. A device for processing fused targets, characterized in that, it includes: An analysis module, configured to determine the coordinate error of the fused target based on the set ground truth data, where the coordinate error includes a lateral error and a longitudinal error, and there are at least two fused targets; A calculation module, configured to calculate the expanded size of the fused object based on the size of the fused object and the coordinate error; a determination module, configured to determine a final target from the fused targets based on an intersection-over-union ratio between at least two fused targets at the expanded size; The analysis module is specifically used for: Acquire true value data corresponding to the fused target, wherein the true value data includes the real coordinates and real size of the actual target corresponding to the fused target; Based on the relative distance from the fusion target to the vehicle and the true value data, determining the lateral error and the longitudinal error of the fusion target at the relative distance; The computing module is specifically used for: determining a longitudinal expansion size of the fused object based on a velocity difference between at least two fused objects; Based on the coordinate error, determining a lateral expansion size of the fused object; Determining the expanded size of the fused object based on the fused object size, the longitudinal expansion size, and the lateral expansion size; The computing module is further specifically used for: Determining a reference value of the longitudinal expansion size of the fusion target based on the speed difference and a preset influencing factor; Determine the maximum value between the reference value and the longitudinal error as the longitudinal expansion size of the fusion object; The computing module is further specifically used for: The lateral error is determined as a lateral expansion size of the fused object.

5. A control device, It is characterized in that include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the control device to perform the fusion target processing method as described in any one of claims 1 to 3.

6. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the fusion target processing method according to any one of claims 1 to 3 when executed by a processor.

Images