Target association method and device, and nonvolatile storage medium and vehicle
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA FAW CO LTD
- Filing Date
- 2023-01-05
- Publication Date
- 2026-06-19
Smart Images
Figure CN115824247B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of autonomous driving, and more specifically, to a target association method, apparatus, non-volatile storage medium, and vehicle. Background Technology
[0002] In the field of automotive ADAS (Advanced Driving Assistance Systems), cameras and millimeter-wave radar are the most common and important sensors. Cameras have advantages in target recognition and lane detection, but are easily affected by lighting and weather conditions; while millimeter-wave radar has advantages in ranging and speed measurement, and is also cheaper. To complement the advantages of both, more and more ADAS suppliers are inclined to a fusion solution, namely a 1V1R (one camera and one millimeter-wave radar) target-level fusion solution. In the target-level fusion solution, an important issue is how to determine whether multiple tracks from different sensors belong to the same track, and then perform information fusion on the same track.
[0003] However, traditional target association methods cannot solve the problem of misassociation caused by different association orders, which leads to instability of the fused targets and seriously affects the stability and security of the backend control algorithm.
[0004] There is currently no effective solution to the above problems. Summary of the Invention
[0005] This invention provides a target association method, apparatus, non-volatile storage medium, and vehicle to at least solve the technical problem of target misassociation in related technologies.
[0006] According to one aspect of the present invention, a target association method is provided, comprising: during vehicle operation, acquiring first sensor data of a first target using a first sensor installed on the vehicle, and acquiring second sensor data of a second target using a second sensor installed on the vehicle; determining historical association information of the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period; in response to the historical association information indicating that the first target and the second target were successfully associated within the historical time period, determining current association information of the first target and the second target based on the first sensor data and the second sensor data, wherein the current association information is used to characterize whether the first target and the second target were successfully associated at the current moment.
[0007] Further, based on the first sensor data and the second sensor data, the current association information of the first target and the second target is determined, including: based on the first sensor data, determining a threshold corresponding to the first target, wherein the threshold is used to characterize the position association threshold and velocity association threshold of the associated targets associated with the first target; in response to the position information in the second sensor data being within the position association threshold and the velocity information in the second sensor data being within the velocity association threshold, determining a first association degree of the first target and the second target based on the distance information in the first sensor data; in response to the first association degree exceeding a first preset threshold, determining the current association information as the first target and the second target being successfully associated at the current moment; in response to the position information in the second sensor data not being within the position association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold, determining the current association information as the first target and the second target being unassociated at the current moment.
[0008] Furthermore, in response to the position information in the second sensor data not being within the position association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold, the method further includes: determining whether there is a third target that meets preset conditions within the threshold corresponding to the first target, wherein the third target is used to characterize the target sensed by the second sensor; in response to the existence of a third target within the threshold corresponding to the first target, determining that the first target and the third target are successfully associated; in response to the absence of a third target within the threshold corresponding to the first target, determining that there is no associated target related to the first target at the current moment.
[0009] Furthermore, in response to the existence of a third target within a threshold corresponding to the first target, the method includes: determining a second correlation degree between the first target and the third target based on distance information in the first sensor data; determining whether there is a historical target identical to the third target in the target correlation information table, wherein the target correlation information table is used to store the correlation degree between the first target and the associated targets within a historical time period, and the associated targets are used to characterize targets sensed by the second sensor and associated with the first target within the historical time period; and updating the target correlation information table based on the third target and the second correlation degree in response to the absence of a historical target in the target correlation information table.
[0010] Furthermore, the method includes: in response to the existence of historical targets in the target association information table, updating the historical association degree in the target association information table based on the second association degree, wherein the historical association degree is used to characterize the association degree between the first target stored in the target association information table and the historical targets.
[0011] Further, determining the second correlation degree between the first target and the third target includes: determining the distance information between the first target and the vehicle based on the distance information in the first sensor data; determining the target distance range to which the distance information belongs from multiple preset distance ranges; and determining the second correlation degree based on the target cumulative value corresponding to the target distance range, wherein the cumulative value corresponding to different preset distance ranges is different.
[0012] Furthermore, the method also includes: in response to the historical association information indicating that the association between the first target and the second target failed within a historical time period, determining whether there is a fourth target that meets preset conditions within a threshold corresponding to the first target, wherein the fourth target is used to characterize the target sensed by the second sensor; in response to the existence of a fourth target that meets preset conditions within the threshold corresponding to the first target, determining that the association between the first target and the fourth target is successful; in response to the absence of a fourth target that meets preset conditions within the threshold corresponding to the first target, determining that there is no fourth target associated with the first target at the current moment.
[0013] According to another aspect of the present invention, a target association device is also provided, comprising: a data acquisition module, configured to acquire first sensor data of a first target using a first sensor installed on the vehicle, and acquire second sensor data of a second target using a second sensor installed on the vehicle during vehicle operation; a historical association information determination module, configured to determine historical association information of the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period; and a current association information determination module, configured to determine current association information of the first target and the second target based on the first sensor data and the second sensor data in response to the historical association information indicating that the first target and the second target were successfully associated within a historical time period, wherein the current association information is used to characterize whether the first target and the second target were successfully associated at the current moment.
[0014] According to another aspect of the present invention, a vehicle is also provided, including one or more processors and a storage device, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to perform the above-described target association method.
[0015] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described target association method.
[0016] According to another aspect of the present invention, a processor is also provided, which is used to run a program, wherein the program executes the target association method described above during runtime.
[0017] In this embodiment of the invention, during vehicle operation, a first sensor data of a first target is collected using a first sensor installed on the vehicle, and a second sensor data of a second target is collected using a second sensor installed on the vehicle. Historical association information of the first and second targets is determined, whereby the historical association information characterizes whether the first and second targets were successfully associated within a historical time period. In response to the historical association information indicating successful association between the first and second targets within the historical time period, current association information of the first and second targets is determined based on the first and second sensor data, whereby the current association information characterizes whether the first and second targets are successfully associated at the current moment. It is readily apparent that before determining the current association information between the visual target and the radar target at the current moment, it is first determined whether the visual target and the radar target have been continuously associated within a historical time period. If they have been continuously associated within a historical time period, then the current association information between the visual target and the radar target at the current moment is re-evaluated. By determining the historical association information of the visual target and the radar target, the target association judgment at the current moment can be made more accurate, achieving the technical effect of improving the association accuracy between the visual target and the radar target, thereby improving the stability of the fused target of the visual target and the radar target, and thus solving the technical problem of target misassociation in related technologies. Attached Figure Description
[0018] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:
[0019] Figure 1 This is a flowchart of a target association method according to an embodiment of the present invention;
[0020] Figure 2 This is a bird's-eye view schematic diagram of a target association method according to an embodiment of the present invention;
[0021] Figure 3 This is an algorithm flowchart of a target association method according to an embodiment of the present invention;
[0022] Figure 4 This is a schematic diagram of a target association device according to an embodiment of the present invention. Detailed Implementation
[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0025] Traditional target association methods, such as the Nearest Neighbors (NN) algorithm, center on the track of a certain type of sensor, set an association threshold around it, and consider the nearest other sensor track falling within the association threshold as the same target track, while the others are considered clutter interference. The advantages of the NN algorithm are low complexity, ease of implementation, and suitability for sparse and stable sensor targets. However, the choice of threshold shape and size has a great impact on the association effect, and it cannot solve the problem of misassociation caused by different association orders.
[0026] To address the aforementioned problems, this invention provides a multi-sensor target association method based on local nearest neighbors. This method improves upon the traditional Neural Network (NN) algorithm and can effectively solve the problem of misassociation caused by different association orders in the NN algorithm.
[0027] Example 1
[0028] According to an embodiment of the present invention, an embodiment of a target association method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.
[0029] Figure 1 This is a flowchart of a target association method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:
[0030] Step S102: During the vehicle's operation, the first sensor data of the first target is collected using the first sensor installed on the vehicle, and the second sensor data of the second target is collected using the second sensor installed on the vehicle.
[0031] Specifically, the aforementioned first sensor can be understood as a vision sensor. The aforementioned first target can be understood as the target sensed by the vision sensor, i.e., a visual target, which may include, but is not limited to, vehicles, obstacles, etc. The aforementioned first sensor data can be understood as data such as the lateral distance and lateral speed of the visual target relative to the vehicle, collected by the vision sensor on the vehicle, but it is not limited to these, and the aforementioned vehicle is assumed to be the vehicle itself.
[0032] The second sensor mentioned above can be understood as a radar sensor. The second target mentioned above can be understood as the target sensed by the radar sensor, i.e., the radar target, which may include, but is not limited to, vehicles, obstacles, etc. The second sensor data mentioned above can be understood as data such as the longitudinal distance and longitudinal speed of the radar target relative to the vehicle collected by the radar sensor on the vehicle, but it is not limited to these.
[0033] Generally, the aforementioned visual and radar targets need to be limited to the effective range of the vehicle. Otherwise, if they are outside the effective range of the vehicle, the error in acquiring visual and radar targets will be large.
[0034] In one optional embodiment, during vehicle operation, visual sensors on the vehicle can collect data on the lateral distance and lateral speed of vehicles ahead relative to the vehicle itself, used to determine visual data during the vehicle's movement. Simultaneously, radar sensors on the vehicle can collect data on the longitudinal distance and longitudinal speed of vehicles ahead relative to the vehicle itself, used to determine radar data during the vehicle's movement. After obtaining the visual and radar data, subsequent visual-radar target association determination can be performed based on this data.
[0035] Step S104: Determine the historical association information of the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period;
[0036] Specifically, the aforementioned historical correlation information can be characterized as whether visual targets and radar targets were successfully correlated within a historical time period.
[0037] It is important to note that when determining the historical association information between visual targets and radar targets within a given time period, a target association information table can be used for assistance. Specifically, using the first sensor data, the ID of a specific visual target among multiple collected visual targets can be determined. By referring to the target association information table, it can be determined whether there are any associated radar targets under that ID. If so, the radar target IDs associated with that visual target within the historical time period and their corresponding association degrees can be determined by referring to the target association information table. Conversely, if the radar target ID corresponding to that visual target ID is empty, it means that there are no radar targets associated with that visual target within the historical time period.
[0038] Generally, visual sensors are more accurate in detecting the lateral distance and lateral velocity of targets, while radar sensors are more accurate in detecting the longitudinal distance and longitudinal velocity. When fusing information about the same associated target, the longitudinal distance and velocity of the fused target are trusted by the radar sensor, while the lateral distance and lateral velocity are trusted by the visual sensor. If misassociation occurs, the fused target information will have a large error compared to the actual information, significantly impacting the backend control algorithm. Through actual testing, the two sensors exhibit the following patterns in target detection: When the target vehicle in front is close to the vehicle, both the radar and visual sensors are accurate in detecting the lateral and longitudinal distances of the target. As the target moves further away from the vehicle, the longitudinal distance error detected by the visual sensor increases, and the lateral error detected by the radar sensor increases. Furthermore, distant, parallel targets are more prone to misassociation.
[0039] Figure 2 This is a bird's-eye view diagram of a sensor target association method according to an embodiment of the present invention. Figure 2 As shown, the left side represents the position of three historical driving positions of the same vehicle ahead relative to the vehicle itself, and the right side is a bird's-eye view of the sensors. In this view, triangles represent targets detected by the visual sensors, asterisks represent targets detected by the radar sensors, and squares represent the association thresholds established with the visual targets as the center. The shape of the thresholds is generally rectangular. Figure 2In this context, (V1, R1), (V2, R2), and (V3, R3) represent the same target. However, when using the Neural Network (NN) algorithm for association matching, all radar and visual targets need to be matched one by one. If the matching order is V1, V2, V3, then the matching result for the current frame is: V1 matches R3, V2 matches R2, and V3 does not match, which is inconsistent with the actual result and has a significant impact on subsequent information fusion. If the matching order is V2, V3, V1, then the NN algorithm can correctly associate them. However, the association order is generally uncertain. Therefore, before determining whether the visual target and the radar target are associated at the current moment, the association between the visual sensor and the radar sensor in the historical time period can be considered to avoid the erroneous association problem caused by the uncertain association order in the NN algorithm.
[0040] Step S106: In response to the historical association information indicating that the first target and the second target were successfully associated within a historical time period, the current association information of the first target and the second target is determined based on the first sensor data and the second sensor data. The current association information is used to characterize whether the first target and the second target are successfully associated at the current moment.
[0041] Specifically, the aforementioned current association information is used to characterize whether the visual target and the radar target have been successfully associated at the current moment.
[0042] In one optional embodiment, if a visual target and a radar target have been successfully associated for multiple consecutive frames within a historical time period, it indicates that the visual target and radar target are likely to be the same vehicle target after fusion. Therefore, based on the determination that the historical association information shows that the visual target and radar target have been successfully associated, the association information of the visual target and radar target at the current moment can be determined.
[0043] In another optional embodiment, the association information between the visual target and the radar target at the current moment can be determined by a variety of conditions. Specifically, the various conditions may include, but are not limited to, whether the radar target is within the position association threshold and velocity association threshold of the visual target at the current moment, and whether the association degree between the visual target and the radar target at the current moment is greater than a preset threshold.
[0044] Generally, position and velocity correlation thresholds, established centered on the visual target, are used to determine whether a radar target exists within these thresholds. The presence of a radar target within these thresholds can be determined using data from the second sensor, such as longitudinal distance and longitudinal relative velocity. Specifically, if the longitudinal distance of the radar target is within the position correlation threshold and its longitudinal relative velocity is within the velocity correlation threshold, then the radar target exists within these two thresholds.
[0045] Other than that Figure 2As shown, assuming V1 and R1 have been associated multiple times in the past (V2 and R2 can be considered as the historical frame positions of V1 and R1), and the correlation between the visual target and the radar target is higher than a set threshold, when associating in the current frame, it is first determined that the visual target V1 was associated with the radar target R1 in the previous frame, and the radar target R1 in the current frame is within the threshold range of the visual target V1. In this case, R1 and V1 are directly matched successfully. Conversely, if the traditional NN algorithm is followed, the current frame V1 will be matched with R3 because R3 is closest to V1. This can lead to misassociation problems due to the uncertain association order.
[0046] In summary, before determining the current association information between visual and radar targets, it is necessary to first determine whether the visual and radar targets have been continuously associated within a historical time period. If they have been continuously associated within a historical time period, the current association information between the visual and radar targets at the current moment should be reassessed. By determining the historical association information between visual and radar targets, the determination of target association at the current moment becomes more accurate, avoiding erroneous associations due to different association sequences, which could lead to instability in the fused targets.
[0047] Optionally, determining the current association information of the first target and the second target based on the first sensor data and the second sensor data includes: determining a threshold corresponding to the first target based on the first sensor data, wherein the threshold is used to characterize the position association threshold and velocity association threshold of the associated targets associated with the first target; in response to the position information in the second sensor data being within the position association threshold and the velocity information in the second sensor data being within the velocity association threshold, determining a first association degree of the first target and the second target based on the distance information in the first sensor data; in response to the first association degree exceeding a first preset threshold, determining the current association information as the first target and the second target being successfully associated at the current moment; in response to the position information in the second sensor data not being within the position association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold, determining the current association information as the first target and the second target being unassociated at the current moment.
[0048] Specifically, the threshold corresponding to the first target mentioned above can be characterized as a threshold established with the visual target as the center and used to determine the associated targets of the visual target. This includes position association thresholds and velocity association thresholds, but is not limited to these.
[0049] The speed-related threshold mentioned above is related to the longitudinal distance between the visual target and the vehicle; the greater the distance, the larger the threshold. It can generally be pre-calibrated based on the sensor's characteristics. Specifically, a range of distances corresponds to a threshold, and an array is constructed based on the pre-calibrated numbers (i.e., the corresponding thresholds). In practical applications, the array can be matched one-to-one with the current longitudinal distance between the visual target and the vehicle.
[0050] For example, the speed threshold is 1.0 m / s for a longitudinal distance range of 0-20 m; 1.2 m / s for a longitudinal distance range of 20-40 m; 1.4 m / s for a longitudinal distance range of 40-60 m; 1.6 m / s for a longitudinal distance range of 60-80 m; 1.8 m / s for a longitudinal distance range of 80-100 m; 2.0 m / s for a longitudinal distance range of 100-120 m; 2.5 m / s for a longitudinal distance range of 120-140 m; and 3.0 m / s for a longitudinal distance range >140 m. For instance, when the longitudinal distance between the visual target and the vehicle is 45 m, the corresponding speed threshold is 1.4 m / s.
[0051] Similarly, the length and width settings for the position threshold can be found as described above, and will not be repeated here.
[0052] The aforementioned first correlation degree can be characterized as the correlation degree between the visual target and the first radar target, determined based on the longitudinal distance between the visual target and the vehicle, when the visual target and the first radar target are successfully associated within a historical time period, and the position information of the first radar target is within the aforementioned position association threshold of the visual target, and the speed information of the first radar target is within the aforementioned speed association threshold of the visual target.
[0053] The aforementioned first preset threshold is a pre-set correlation value between visual targets and radar targets. When the correlation value meets the first preset threshold, it indicates that the visual target and radar target are successfully correlated; otherwise, the correlation fails.
[0054] The aforementioned current association information can be characterized as follows: when the correlation between the visual target and the first radar target exceeds a first preset threshold at the current moment, the visual target and the first radar target are successfully associated.
[0055] Conversely, if the position information of the first radar target is not within the aforementioned position association threshold of the visual target, and the velocity information of the first radar target is not within the aforementioned velocity association threshold of the visual target, or if the correlation between the visual target and the first radar target at the current moment does not exceed the first preset threshold, the aforementioned current association information can also be characterized as the visual target and the first radar target failing to associate at the current moment, and it is necessary to re-determine other radar targets associated with the visual target.
[0056] To illustrate the process of determining the current associated information described above, Figure 3 This is an algorithm flowchart of a target association method according to an embodiment of the present invention. Figure 3 As shown:
[0057] S11, Input the trajectory information of each sensor. The target association algorithm requires the input of target trajectory information output by each sensor. This target trajectory information consists of the trajectory information of visual targets and the trajectory information of radar targets, specifically including target ID (i.e., the ID (Identity Document) of each sensor), target lateral distance, longitudinal distance, longitudinal relative velocity, lateral relative velocity, target category, azimuth angle, and other information;
[0058] S12, perform spatiotemporal synchronization of trajectory information from various sensors. Specifically, for the detected target trajectory information, based on the installation position, angle, and coordinate transformation formula of each sensor, the position information of the target trajectory is converted into two-dimensional coordinate position information in the vehicle coordinate system, so that the sensor data is synchronized in space; based on the timestamps of the received data from each sensor and the pre-set motion models (such as CV (Constant Velocity) and CA (Constant Acceleration)), the position information of the vision sensor and radar sensor is compensated, so that the sensor data is synchronized in time.
[0059] S13, with the visual target as the center, establish position association thresholds and speed association thresholds based on the longitudinal distance between the visual target and the vehicle;
[0060] S14, traverse all visual targets and determine whether there is a radar target associated with the visual target in the previous frame (i.e., within the historical time period).
[0061] S15, if there is a radar target associated with the visual target in the previous frame (i.e., within the historical time period), it can be determined whether the associated radar target at the current moment is within the velocity association threshold and position association threshold of the visual target.
[0062] S16, if the associated radar target at the current moment exists within the velocity association threshold and position association threshold of the visual target, further determine whether the correlation between the visual target and the first radar target (i.e. the associated radar target mentioned above) is greater than the first preset threshold 1.
[0063] S17, if the correlation between the visual target and the first radar target is greater than the first preset threshold 1, it can be determined that the current correlation information is that the visual target and the first radar target at the current moment are successfully correlated.
[0064] S18. If the position information of the first radar target is not within the position association threshold of the visual target, and the velocity information of the first radar target is not within the velocity association threshold of the visual target, or the correlation between the visual target and the first radar target at the current moment does not exceed the first preset threshold, the current association information is determined to be that the visual target and the first radar target at the current moment have failed to be associated, and other radar targets associated with the visual target need to be re-determined.
[0065] Optionally, in response to the position information in the second sensor data not being within the position association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold, the method further includes: determining whether there is a third target that meets preset conditions within the threshold corresponding to the first target, wherein the third target is used to characterize the target sensed by the second sensor; in response to the existence of a third target within the threshold corresponding to the first target, determining that the first target and the third target are successfully associated; in response to the absence of a third target within the threshold corresponding to the first target, determining that there is no associated target related to the first target at the current moment.
[0066] Specifically, the aforementioned third target, detected by radar sensors, is another radar target that is distinct from the first radar target.
[0067] The aforementioned preset conditions include, at a minimum, the radar target closest to the visual target in the visual target position association threshold, the radar target with the closest velocity to the visual target in the visual target velocity association threshold, and a radar target that has not been associated with the visual target.
[0068] like Figure 3 As shown, when redefining other radar targets associated with visual targets based on S18, this includes:
[0069] S181, determine whether there are other radar targets that meet the above preset conditions within the position association threshold and velocity association threshold of the visual target;
[0070] S182, if there are other radar targets that meet the above preset conditions within the position association threshold and velocity association threshold of the visual target, the radar target can be marked as the second radar target (that is, the third target mentioned above). At the same time, it can be determined that the visual target and the second radar target are successfully associated.
[0071] S183, if there are no other radar targets that meet the above preset conditions within the position association threshold and velocity association threshold of the visual target, it can be determined that there are no other radar targets associated with the visual target at the current moment.
[0072] Optionally, in response to the existence of a third target within a threshold corresponding to the first target, the method includes: determining a second correlation degree between the first target and the third target based on distance information in the first sensor data; determining whether there is a historical target identical to the third target in a target correlation information table, wherein the target correlation information table is used to store the correlation degree between the first target and associated targets within a historical time period, and the associated targets are used to characterize targets sensed by the second sensor and associated with the first target within the historical time period; in response to the absence of a historical target in the target correlation information table, updating the target correlation information table based on the third target and the second correlation degree; in response to the existence of a historical target in the target correlation information table, updating the historical correlation degree in the target correlation information table based on the second correlation degree, wherein the historical correlation degree is used to characterize the correlation degree between the first target and historical targets stored in the target correlation information table.
[0073] Specifically, the aforementioned second correlation degree, which characterizes the correlation value between the visual target and the second radar target, can be determined by the distance between the visual target and the longitudinal distance of the vehicle.
[0074] The aforementioned target association information table is used to store radar target information associated with visual targets within a historical time period, including at least visual target ID, radar target ID, and the association value between visual targets and radar targets.
[0075] Table 1 Target Association Information for Visual and Radar Targets
[0076] Visual target ID Radar Target ID correlation 1 2 15 2 40 12 3 52 5 … … … … … … 30 … …
[0077] Table 1 shows the target association information for visual and radar targets. As shown in Table 1, n represents the maximum number of targets output by the visual sensor. Generally, the ID of a visual target ranges from 1 to 30. The row number of each row represents the ID of the visual target, so the ID range is 1 to 30. The second column of the array stores the radar IDs associated with the visual targets, and the third column is their association degree. The default value for the entire array is 0 during initialization. Therefore, the target association information table is a 30*2 array. For example, the fourth row in Table 1 means that the current association degree between the visual target (ID 3) and the radar target (ID 52) is 5.
[0078] The aforementioned historical target is a radar target sensed by the radar sensor, and this radar target has the same ID as the second radar target (i.e., the aforementioned third target).
[0079] The aforementioned historical correlation degree represents the correlation between the visual targets stored in the target correlation information table and historical targets.
[0080] In one optional embodiment, in the process of determining the second correlation degree between the visual target and the second radar target, it can be first determined whether there is a historical target of the second radar target in the target correlation information table. If it does not exist, the ID of the second radar target and the second correlation degree between the visual target and the second radar target are updated in the target correlation information table; otherwise, if it exists, the historical correlation degree between the visual target and the historical target is directly updated according to the second correlation degree.
[0081] For example, after obtaining the ID of the second radar target, it is necessary to check whether there is a historical radar target with the same ID in the target association information table. If so, the historical association degree between the historical radar target and the corresponding visual target is obtained, and then the historical association degree is updated using the second association degree. Conversely, if no such target exists, it means that there is no second radar target associated with the visual target in the historical time period, and the ID and second association degree of the second radar target are directly updated in the target association information table.
[0082] Optionally, determining the second correlation degree between the first target and the third target includes: determining the distance information between the first target and the vehicle based on the distance information in the first sensor data; determining the target distance range to which the distance information belongs from multiple preset distance ranges; and determining the second correlation degree based on the target cumulative value corresponding to the target distance range, wherein the cumulative value corresponding to different preset distance ranges is different.
[0083] Specifically, the aforementioned distance information reflects the longitudinal distance between the visual target and the vehicle, and can be directly obtained from the data collected by the visual sensors.
[0084] The aforementioned preset distance ranges refer to the pre-defined longitudinal distance ranges between multiple visual targets and the vehicle.
[0085] The target distance range mentioned above reflects the target distance range to which the distance information belongs within multiple preset distance ranges.
[0086] The aforementioned target cumulative value is the cumulative correlation value corresponding to multiple preset distance ranges.
[0087] For example, the aforementioned preset distance ranges include: within 30m, within 30-50m, within 50-80m, and above 80m. When the visual target is within 30m of the vehicle, the cumulative value of one association between the visual target and the radar target is 4; within 30-50m, the cumulative value is 3; within 50-80m, the cumulative value is 2; above 80m, the cumulative value is 1, until the maximum association degree between the visual target and the radar target is 20.
[0088] Optionally, the method further includes: in response to the historical association information indicating that the association between the first target and the second target failed within a historical time period, determining whether there is a fourth target that meets preset conditions within a threshold corresponding to the first target, wherein the fourth target is used to characterize the target sensed by the second sensor; in response to the existence of a fourth target that meets preset conditions within the threshold corresponding to the first target, determining that the association between the first target and the fourth target is successful; in response to the absence of a fourth target that meets preset conditions within the threshold corresponding to the first target, determining that there is no fourth target associated with the first target at the current moment.
[0089] Specifically, the fourth target mentioned above is still a target detected by radar sensors, which can be understood as the third radar target.
[0090] like Figure 3 As shown, based on S14, if the visual target was not associated with any radar target in the previous frame (i.e., within the historical time period), the following process is also included:
[0091] S19, determine whether there is a third radar target that meets the preset conditions within the position association threshold and velocity association threshold corresponding to the visual target;
[0092] S20 (same as S182 above): If there is a third radar target that meets the preset conditions, it can be determined that the visual target and the third radar target are successfully associated.
[0093] S21 (same as S183 above): If there is no third radar target that meets the preset conditions, it can be determined that there is no third radar target associated with the visual target at the current moment.
[0094] S22, determine whether all visual targets have been traversed;
[0095] S23, If the traversal is complete, the algorithm ends;
[0096] If the traversal is not completed by S24, then jump to S14 to complete the traversal of all visual targets.
[0097] Example 2
[0098] According to an embodiment of the present invention, a target association device is also provided. This device can execute a target association method provided in Embodiment 1 above. The specific implementation method and preferred application scenario are the same as those in Embodiment 1 above, and will not be described in detail here.
[0099] Figure 4 This is a schematic diagram of a target association device according to an embodiment of the present invention, such as... Figure 4 As shown, the device includes:
[0100] The data acquisition module 402 is used to acquire first sensor data of a first target using a first sensor installed on the vehicle during vehicle operation, and to acquire second sensor data of a second target using a second sensor installed on the vehicle.
[0101] The historical association information determination module 404 is used to determine the historical association information of the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period;
[0102] The current association information determination module 406 is used to determine the current association information of the first target and the second target based on the first sensor data and the second sensor data in response to the historical association information indicating that the first target and the second target were successfully associated within a historical time period. The current association information is used to characterize whether the first target and the second target are successfully associated at the current moment.
[0103] Optionally, the current association information determination module 406 includes: a threshold determination module, used to determine a threshold corresponding to the first target based on the first sensor data, wherein the threshold is used to characterize the position association threshold and velocity association threshold of the associated targets associated with the first target; a first association degree determination module, used to determine a first association degree between the first target and the second target based on the distance information in the first sensor data, in response to the position information in the second sensor data being within the position association threshold and the velocity information in the second sensor data being within the velocity association threshold; a first associated target determination module, used to determine the current association information as the first target and the second target being successfully associated at the current moment, in response to the first association degree exceeding a first preset threshold; and a second associated target determination module, used to determine the current association information as the first target and the second target being unassociated at the current moment, in response to the position information in the second sensor data not being within the position association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold.
[0104] Optionally, the second associated target determination module includes: a first threshold target determination module, used to determine whether there is a third target that meets preset conditions within the threshold corresponding to the first target, wherein the third target is used to characterize the target sensed by the second sensor; a third associated target determination module, used to determine that the first target and the third target are successfully associated in response to the existence of a third target within the threshold corresponding to the first target; and a fourth associated target determination module, used to determine that there is no associated target related to the first target at the current time in response to the absence of a third target within the threshold corresponding to the first target.
[0105] Optionally, the third associated target determination module includes: a second correlation degree determination module, used to determine a second correlation degree between the first target and the third target based on distance information in the first sensor data; a historical target determination module, used to determine whether there is a historical target identical to the third target in the target association information table, wherein the target association information table is used to store the correlation degree between the first target and associated targets within a historical time period, and the associated targets are used to characterize targets sensed by the second sensor and associated with the first target within the historical time period; and a table update module, used to update the target association information table based on the third target and the second correlation degree in response to the absence of a historical target in the target association information table.
[0106] Furthermore, the device also includes a historical correlation update module, used to update the historical correlation in the target correlation information table based on the second correlation in response to the existence of historical targets in the target correlation information table, wherein the historical correlation is used to characterize the correlation between the first target stored in the target correlation information table and the historical targets.
[0107] Furthermore, the second correlation determination module includes: a distance determination unit, used to determine the distance information between the first target and the vehicle based on the distance information in the first sensor data; a target distance range determination unit, used to determine the target distance range to which the distance information belongs from multiple preset distance ranges; and a second correlation determination unit, used to determine the second correlation based on the target cumulative value corresponding to the target distance range, wherein the cumulative value corresponding to different preset distance ranges is different.
[0108] Furthermore, the device also includes: a second threshold target determination module, used to determine whether there is a fourth target that meets preset conditions within the threshold corresponding to the first target in response to the failure of association between the first target and the second target within a historical time period in the historical association information, wherein the fourth target is used to characterize the target sensed by the second sensor; a fifth associated target determination module, used to determine that the first target and the fourth target are successfully associated in response to the existence of a fourth target that meets preset conditions within the threshold corresponding to the first target; and a sixth associated target determination module, used to determine that there is no fourth target associated with the first target at the current time in response to the absence of a fourth target that meets preset conditions within the threshold corresponding to the first target.
[0109] Example 3
[0110] According to an embodiment of the present invention, a vehicle is also provided, including one or more processors and a storage device, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to perform the above-described target association method.
[0111] Example 4
[0112] According to an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to execute the above-described target association method.
[0113] Example 5
[0114] According to an embodiment of the present invention, a processor is also provided, which is used to run a program, wherein the program executes the target association method described above during runtime.
[0115] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0116] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0117] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.
[0118] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0119] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0120] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.
[0121] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.
Claims
1. A target association method, characterized by, The method includes: During vehicle operation, the first sensor data of the first target is collected using the first sensor installed on the vehicle, and the second sensor data of the second target is collected using the second sensor installed on the vehicle. Determine the historical association information between the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period; In response to the historical association information indicating that the first target and the second target were successfully associated within the historical time period, the current association information of the first target and the second target is determined based on the first sensor data and the second sensor data, wherein the current association information is used to characterize whether the first target and the second target are successfully associated at the current moment.
2. The method of claim 1, wherein, Based on the first sensor data and the second sensor data, the current association information between the first target and the second target is determined, including: Based on the first sensor data, a threshold corresponding to the first target is determined, wherein the threshold is used to characterize the position association threshold and velocity association threshold of the associated targets associated with the first target; In response to the location information in the second sensor data being within the location association threshold and the velocity information in the second sensor data being within the velocity association threshold, a first correlation degree between the first target and the second target is determined based on the distance information in the first sensor data; In response to the first correlation degree exceeding a first preset threshold, the current correlation information is determined to indicate that the first target and the second target are successfully associated at the current moment; In response to the location information in the second sensor data not being within the location association threshold, the speed information in the second sensor data not being within the speed association threshold, or the first association degree not exceeding the first preset threshold, the current association information is determined to be that the association between the first target and the second target at the current time has failed.
3. The method of claim 2, wherein, In response to the location information in the second sensor data not being within the location association threshold, the velocity information in the second sensor data not being within the velocity association threshold, or the first association degree not exceeding the first preset threshold, the method further includes: Determine whether there is a third target that meets preset conditions within the threshold corresponding to the first target, wherein the third target is used to characterize the target sensed by the second sensor; If the third target exists within the threshold corresponding to the first target, it is determined that the first target and the third target are successfully associated. In response to the absence of the third target within the threshold corresponding to the first target, it is determined that there is no associated target related to the first target at the current moment.
4. The method of claim 3, wherein, In response to the existence of the third target within a threshold corresponding to the first target, the method includes: Based on the distance information in the first sensor data, a second correlation degree between the first target and the third target is determined; Determine whether there is a historical target that is the same as the third target in the target association information table, wherein the target association information table is used to store the association degree between the first target and the associated targets within the historical time period, and the associated targets are used to characterize the targets sensed by the second sensor and associated with the first target within the historical time period; If the historical target is not found in the target association information table, the target association information table is updated based on the third target and the second association degree.
5. The method according to claim 4, characterized in that, The method includes: In response to the presence of historical targets in the target association information table, the historical association degree in the target association information table is updated based on the second association degree, wherein the historical association degree is used to characterize the association degree between the first target stored in the target association information table and the historical targets.
6. The method of claim 4, wherein, Determining the second correlation between the first target and the third target includes: Based on the distance information in the first sensor data, determine the distance information between the first target and the vehicle; From multiple preset distance ranges, determine the target distance range to which the distance information belongs; The second correlation degree is determined based on the cumulative target value corresponding to the target distance range, wherein the cumulative value is different for different preset distance ranges.
7. The method according to claim 1, characterized in that, The method further includes: In response to the historical association information indicating that the first target and the second target failed to associate within the historical time period, it is determined whether there is a fourth target that meets the preset conditions within the threshold corresponding to the first target, wherein the fourth target is used to characterize the target sensed by the second sensor; In response to the existence of a fourth target that satisfies the preset condition within the threshold corresponding to the first target, it is determined that the first target and the fourth target are successfully associated; In response to the absence of a fourth target satisfying the preset condition within the threshold corresponding to the first target, it is determined that there is no fourth target associated with the first target at the current moment.
8. A target association device, characterized in that, The device includes: The data acquisition module is used to acquire first sensor data of a first target using a first sensor installed on the vehicle, and to acquire second sensor data of a second target using a second sensor installed on the vehicle during vehicle operation. The historical association information determination module is used to determine the historical association information between the first target and the second target, wherein the historical association information is used to characterize whether the first target and the second target were successfully associated within a historical time period; The current association information determination module is used to determine the current association information of the first target and the second target based on the first sensor data and the second sensor data in response to the historical association information indicating that the first target and the second target were successfully associated within the historical time period. The current association information is used to characterize whether the first target and the second target are successfully associated at the current moment.
9. A non-volatile storage medium, comprising: The non-volatile storage medium includes a stored program, wherein, when the program is executed, the target association method of any one of claims 1-8 is executed in the processor of the device.
10. A vehicle, characterized in that, include: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors perform the target association method according to any one of claims 1 to 8.