Target tracking method and device, electronic equipment and storage medium

Abstract

The invention provides a target tracking method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining current feature data of each target around a vehicle based on a plurality of sensors installed on the vehicle; according to the current feature data, performing initial fusion on each target to obtain an initial fusion target set; acquiring historical feature data of each initial fusion target in the initial fusion target set based on a plurality of sensors; determining a first target similarity between the initial fusion targets based on the historical feature data; based on the first target similarity, adjusting a fusion relationship among the initial fusion targets to obtain a fusion target set; and performing target tracking on each fusion target in the fusion target set based on the current feature data. The method can improve the accuracy of target fusion and the stability of target tracking.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and more specifically, to a target tracking method, device, electronic device and storage medium in the field of data processing technology. Background Art

[0002] In the field of target tracking, due to various external factors such as performance defects of sensor detection, target occlusion, etc., it is often impossible to accurately track and identify the target in some cases. Therefore, multi-sensor data fusion technology is widely used in the field of target tracking. Multi-sensor data fusion technology eliminates the limitations of a single sensor, gives full play to the joint advantages of multiple sensors, and integrates the data resources provided by multiple similar or different sensors distributed in different locations, so that the system can obtain more sufficient information.

[0003] In the related art, the target is usually fused through the attribute information of the targets around the vehicle at the current time point detected by multiple sensors, and the fused target is tracked. If the attribute information of the target a1 detected by sensor a and the target b1 detected by sensor b at the current time point is different due to the difference in sensor performance, then a1 and b1 are considered to be two different targets, and the two targets are not fused. The two targets will be tracked separately in the future. But in fact, a1 and b1 are the same target, and the same target needs to be fused before the fused target can be tracked. Therefore, this method will lead to low accuracy of target fusion, which will affect the stability of target tracking. Summary of the invention

[0004] The present application provides a target tracking method, device, electronic device and storage medium, which can improve the accuracy of target fusion and the stability of target tracking.

[0005] In a first aspect, a target tracking method is provided, the method comprising:

[0006] Acquire current feature data of each target around the vehicle based on a plurality of sensors installed on the vehicle;

[0007] According to the current feature data, each target is initially fused to obtain an initial fusion target set;

[0008] Acquire historical feature data of each initial fusion target in the initial fusion target set based on multiple sensors;

[0009] Based on the historical feature data, determining the first target similarity between the initial fusion targets;

[0010] Based on the similarity of the first target, the fusion relationship between the initial fusion targets is adjusted to obtain a fusion target set;

[0011] Based on the current feature data, each fusion target in the fusion target set is tracked.

[0012] In the above technical solution, the current feature data of each target around the vehicle is obtained through multiple sensors on the vehicle, and each target is initially fused according to the current feature data of each target, and then the target similarity between each initial fusion target is determined according to the historical feature data of the initial fusion target, and the fusion relationship between each initial fusion target is adjusted according to the target similarity. According to the current feature data of each target, each target is initially fused to obtain the initial fusion target, so the initial fusion target reflects the difference in feature data of each target at the current time point, and because the historical feature data of each initial fusion target reflects the movement trend of each initial fusion target over a period of time, the first target similarity of each initial fusion target determined by the historical feature data can more accurately determine whether the two initial fusion targets are the same target, and then adjust the fusion relationship between the initial fusion targets to improve the accuracy of target fusion. The adjusted fusion target is accurately tracked in combination with the current feature data, which is conducive to improving the stability of target tracking.

[0013] In combination with the first aspect, in some possible implementations, the historical feature data of the initial fusion target includes attribute information of multiple dimensions of the initial fusion target within a preset time period before the current time point;

[0014] Based on the historical feature data, the first target similarity between the initial fusion targets is determined, including:

[0015] Based on the attribute information of multiple dimensions of each initial fusion target, determining the attribute similarity of each initial fusion target in the same dimension;

[0016] Based on the attribute similarity of each initial fusion target in the same dimension, the first target similarity between each initial fusion target is determined.

[0017] In combination with the first aspect and the above implementation, in some possible implementations, based on the attribute information of multiple dimensions of each initial fusion target, determining the attribute similarity of each initial fusion target in the same dimension includes:

[0018] For attribute information of any dimension among the attribute information of multiple dimensions, compare the differences between the attribute information of each initial fusion target at each time point within a preset time period;

[0019] Determine a first ratio of the length of time that the difference is within a preset difference to the length of time in the preset time period;

[0020] Based on the first ratio, the attribute similarity of each initial fusion target in the same dimension is determined.

[0021] In combination with the first aspect and the above implementations, in some possible implementations, the attribute information of multiple dimensions includes location information;

[0022] Based on the attribute information of multiple dimensions of each initial fusion target, the attribute similarity of each initial fusion target in the same dimension is determined, including:

[0023] Based on the position information of each initial fusion target within a preset time period, determining the position change trend of each initial fusion target;

[0024] Based on the position change trend, determine the movement direction of each initial fusion target;

[0025] Based on the motion direction, the attribute similarity of each initial fusion target in position information is determined.

[0026] In combination with the first aspect and the above implementation, in some possible implementations, based on target similarity, the fusion relationship between the initial fusion targets is adjusted to obtain a fusion target set, including:

[0027] If the target similarity between the first initial fusion target and the second initial fusion target is higher than a first threshold, the first initial fusion target and the second initial fusion target are determined to be the same target, and the first initial fusion target and the second initial fusion target are fused to obtain a fusion target set, wherein the first initial fusion target and the second initial fusion target are any two initial fusion targets in the initial fusion target set;

[0028] Based on the current feature data, target tracking is performed on each fusion target in the fusion target set, including:

[0029] Fusing the current feature data of the first initial fusion target and the current feature data of the second initial fusion target to obtain processed current feature data;

[0030] The same target is tracked based on the processed current feature data.

[0031] In combination with the first aspect and the above implementation manner, in some possible implementation manners, after initially fusing each target according to the current feature data to obtain an initial fusion target set, the method further includes:

[0032] If there is a first type of fusion target including at least two targets in the initial fusion target set, acquiring historical feature data of the at least two targets based on at least two sensors that detect the at least two targets;

[0033] Determining whether at least two targets in the first type of fusion targets are the same target based on historical feature data of at least two targets;

[0034] If at least two targets are the same target, it is determined that the first type of fusion target belongs to the correctly fused first type of fusion target;

[0035] If at least two targets are not the same target, it is determined that the first type of fusion target belongs to the initial fusion target of incorrect fusion, and the at least two targets are respectively used as separate second type of fusion targets;

[0036] According to the correctly fused first-category fusion targets and the separate second-category fusion targets, the initial fusion target set is corrected, and the corrected fusion target set is used as the initial fusion target set.

[0037] In combination with the first aspect and the above implementation manner, in some possible implementation manners, determining whether at least two targets in the first type of fusion targets are the same target according to historical feature data of at least two targets includes:

[0038] Determine, based on the historical feature data of the at least two targets and the historical feature data of the first type of fusion targets, second target similarities between the at least two targets and the first type of fusion targets respectively;

[0039] Acquire association states of at least two sensors and the first type of fusion target respectively within a preset time period;

[0040] Determine, according to the association states, the similarity of association states between at least two sensors and the first type of fusion target;

[0041] According to the second target similarity and the association state similarity, it is determined whether the at least two targets are the same target.

[0042] In combination with the first aspect and the above implementation, in some possible implementations, initial fusion is performed on each target according to the current feature data to obtain an initial fusion target set, including:

[0043] If the gap between the current feature data of the first target and the current feature data of the second target is less than the preset gap, the first target and the second target are initially fused to obtain an initial fused target set, where the first target is the target detected by the first sensor, the second target is the target detected by the second sensor, and the first sensor and the second sensor are any two sensors among the multiple sensors.

[0044] In combination with the first aspect and the above implementation, in some possible implementations, each sensor corresponds to n queues, each queue is used to store historical feature data of a target detected by the sensor, and n is the number of targets that each sensor can detect;

[0045] The historical feature data of each initial fusion target in the initial fusion target set is obtained based on multiple sensors, including:

[0046] Based on the historical feature data stored in the queues corresponding to the multiple sensors, the historical feature data of each initial fusion target in the initial fusion target set is obtained.

[0047] In a second aspect, a target tracking device is provided, the device comprising:

[0048] A first acquisition module, used for acquiring current feature data of each target around the vehicle based on a plurality of sensors installed on the vehicle;

[0049] A fusion module is used to initially fuse each target according to the current feature data to obtain an initial fusion target set;

[0050] A second acquisition module is used to acquire historical feature data of each initial fusion target in the initial fusion target set based on multiple sensors;

[0051] A determination module, used for determining the first target similarity between the initial fusion targets based on the historical feature data;

[0052] An adjustment module, used for adjusting the fusion relationship between the initial fusion targets based on the first target similarity to obtain a fusion target set;

[0053] The tracking module is used to track each fusion target in the fusion target set based on the current feature data.

[0054] In combination with the second aspect, in some possible implementations, the historical feature data of the initial fusion target includes attribute information of multiple dimensions of the initial fusion target within a preset time period before the current time point; the determination module is also used to: determine the attribute similarity of each initial fusion target in the same dimension based on the attribute information of multiple dimensions of each initial fusion target; determine the first target similarity between each initial fusion target based on the attribute similarity of each initial fusion target in the same dimension.

[0055] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the determination module is also used to: for the attribute information of any dimension in the attribute information of multiple dimensions, compare the differences between the attribute information of each initial fusion target at each time point within a preset time period; determine a first ratio of the time length of the difference within the preset difference to the time length of the preset time period; based on the first ratio, determine the attribute similarity of each initial fusion target in the same dimension.

[0056] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the attribute information of multiple dimensions includes location information; the determination module is also used to: determine the position change trend of each initial fusion target based on the location information of each initial fusion target within a preset time period; determine the movement direction of each initial fusion target based on the position change trend; determine the attribute similarity of each initial fusion target in the location information based on the movement direction.

[0057] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the adjustment module is also used to: if the target similarity between the first initial fusion target and the second initial fusion target is higher than a first threshold, determine that the first initial fusion target and the second initial fusion target are the same target, and fuse the first initial fusion target and the second initial fusion target to obtain a fusion target set, wherein the first initial fusion target and the second initial fusion target are any two initial fusion targets in the initial fusion target set; the tracking module is also used to: fuse the current feature data of the first initial fusion target and the current feature data of the second initial fusion target to obtain processed current feature data; and track the same target based on the processed current feature data.

[0058] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the fusion module is also used to: if there is a first-category fusion target including at least two targets in the initial fusion target set, obtain historical feature data of the at least two targets based on at least two sensors that detect the at least two targets; determine whether at least two targets in the first-category fusion target are the same target based on the historical feature data of the at least two targets; if at least two targets are the same target, determine that the first-category fusion target belongs to the correctly fused first-category fusion target; if at least two targets are not the same target, determine that the first-category fusion target belongs to the incorrectly fused initial fusion target, and use the at least two targets as separate second-category fusion targets respectively; correct the initial fusion target set based on the correctly fused first-category fusion target and the separate second-category fusion target, and use the corrected fusion target set as the initial fusion target set.

[0059] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the fusion module is also used to: determine the second target similarity between at least two targets and the first type of fusion target respectively based on the historical feature data of at least two targets and the historical feature data of the first type of fusion target; obtain the association status of at least two sensors with the first type of fusion target respectively within a preset time period; determine the association status similarity between at least two sensors and the first type of fusion target based on the association status; determine whether at least two targets are the same target based on the second target similarity and the association status similarity.

[0060] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, the fusion module is also used to: if the gap between the current feature data of the first target and the current feature data of the second target is less than a preset gap, the first target and the second target are initially fused to obtain an initial fused target set, where the first target is the target detected by the first sensor, the second target is the target detected by the second sensor, and the first sensor and the second sensor are any two sensors among multiple sensors.

[0061] In combination with the second aspect and the above-mentioned implementation methods, in some possible implementation methods, each sensor corresponds to n queues, each queue is used to store historical feature data of a target detected by the sensor, and n is the number of targets that each sensor can detect; the second acquisition module is also used to: based on the historical feature data stored in the queues corresponding to multiple sensors, obtain the historical feature data of each initial fusion target in the initial fusion target set.

[0062] In a third aspect, an electronic device is provided, comprising a memory and a processor. The memory is used to store executable program code, and the processor is used to call and run the executable program code from the memory, so that the electronic device executes the method in the first aspect or any possible implementation of the first aspect.

[0063] In a fourth aspect, a computer program product is provided, comprising: a computer program code, which, when executed on a computer, enables the computer to execute the method in the first aspect or any possible implementation of the first aspect.

[0064] In a fifth aspect, a computer-readable storage medium is provided, which stores a computer program code. When the computer program code runs on a computer, the computer executes the method in the above-mentioned first aspect or any possible implementation manner of the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 is a schematic flow chart of a target tracking method provided in an embodiment of the present application;

[0066] Figure 2 is a schematic diagram of sensor disassociation provided in an embodiment of the present application;

[0067] Figure 3 is a schematic diagram of sensor association provided in an embodiment of the present application;

[0068] Figure 4 is a structural schematic diagram of a target tracking device provided in an embodiment of the present application;

[0069] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] The following will be combined with the accompanying drawings to clearly and thoroughly describe the technical solutions in this application. In the description of the embodiments of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B: "and / or" in the text is only a description of the association state of the associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application, "multiple" means two or more than two.

[0071] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features.

[0072] Before explaining in detail the target tracking method provided in the embodiment of the present application, the application scenario provided in the embodiment of the present application is first introduced.

[0073] Multi-sensor data fusion technology is an information processing method proposed for the specific problem of using multiple sensors to achieve environmental perception in intelligent driving vehicle systems. It can give full play to the combined advantages of multiple sensors, eliminate the limitations of a single sensor, integrate the data resources provided by multiple sensors of the same or different types distributed in different locations, use computer technology to analyze and complement them, achieve the best synergy, obtain a consistent interpretation and description of the observed object, improve the fault tolerance of the system, and enable the system to obtain more sufficient information. Therefore, multi-sensor data fusion technology is applied to the field of target tracking so that the system can track the target based on the data collected by multiple sensors.

[0074] When using multiple sensors for target tracking, the target is usually fused through the attribute information of the target detected by multiple sensors at the current time, and the fused target is tracked. If sensor a detects target a1 and sensor b detects target b1 at a certain time point, and the attribute information of target a1 detected by sensor a is the same as the attribute information of target b1 detected by sensor b, then target a1 and target b1 are considered to be the same target, and target a1 and target b1 are fused, that is, sensor a and sensor b are associated, and the fused target is tracked according to the attribute information collected by the associated sensors. If the attribute information of target a1 detected by sensor a and the attribute information of target b1 detected by sensor b are different at the next time point, target a1 and target b1 are unfused. But in fact, if a1 and b1 are indeed the same target, unfusion of target a1 and target b1 will lead to inaccurate target fusion and unstable target tracking.

[0075] Based on this, an embodiment of the present application provides a target tracking method, which is applied to a computer device for tracking a target. The method can combine the current feature data and historical feature data of the target, accurately adjust the fusion relationship between the initial fusion targets, and track the adjusted fusion target, thereby improving the accuracy of target fusion and the stability of target tracking.

[0076] Figure 1 is a schematic flow chart of a target tracking method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0077] Step 101: Acquire current feature data of each target around the vehicle based on multiple sensors installed on the vehicle.

[0078] Among them, multiple sensors can be located at different positions of the vehicle to detect feature data of various targets around the vehicle. For example, the multiple sensors can include a front-view camera, a front radar, a front corner radar installed in front of the vehicle, a rear-view camera, a rear corner radar installed in the rear of the vehicle, etc., and the targets can include pedestrians, obstacles, guardrails, lamp poles, etc. around the vehicle.

[0079] The characteristic data of the target includes attribute information of multiple dimensions of the target at the current time point, and the attribute information of multiple dimensions may include, for example, the location information, speed information, motion state, motion direction, and other attribute information of the target. The location information may include lateral location information and longitudinal location information, and the speed information may include lateral speed information and longitudinal speed information.

[0080] Among them, the position information may include the relative position and absolute position of the target. The relative position indicates the position of the target relative to the sensor, and the absolute position indicates the actual position of the target. After the sensor acquires the relative position of the target, the absolute position of the target can be calculated based on the relative position, and then the target similarity of each target in the position information is calculated based on the absolute position of each target. For ease of description, the position information in the subsequent content refers to the absolute position of the target. The motion state may include the target motion state and the target stop state, and the motion direction may include east, west, south, north and other directions.

[0081] Among them, multiple sensors such as a front-view camera and a front radar are used to detect targets around the vehicle, and each sensor collects feature data of the target. For example, a front radar and a front-view camera can be deployed at the front of the vehicle, and both sensors are used to detect targets around the vehicle and collect current feature data of the targets. For example, sensor a detects that there are three targets a1, a2, and a3 around the vehicle, and obtains the current feature data of each target. Sensor b detects that there are five targets b1, b2, b3, b4, and b5 around the vehicle, and obtains the current feature data of each target. Since the detection performance of different sensors varies, and the detection range of the front-view camera and the front radar may vary, there may be differences between the feature data collected by multiple sensors about a certain target.

[0082] Step 102: Perform initial fusion on each target based on the current feature data to obtain an initial fusion target set.

[0083] After obtaining the current feature data of each target, the targets with smaller current feature data differences can be initially fused according to the current feature data of each target to obtain an initial fused target set.

[0084] In some embodiments, the implementation process of step 102 may be: if the gap between the current feature data of the first target and the current feature data of the second target is less than a preset gap, the first target and the second target are initially fused to obtain an initial fused target set, where the first target is the target detected by the first sensor, the second target is the target detected by the second sensor, and the first sensor and the second sensor are any two sensors among the multiple sensors.

[0085] The preset gap can be preset, and the embodiment of the present application does not limit this. The gap between the current feature data of the two targets can be the gap between the attribute information such as position information, speed information, motion state, motion direction, etc.

[0086] After obtaining the current feature data of each target collected by each sensor, the difference between the current feature data of each target collected by different sensors can be compared. If the difference between the current feature data of the first target collected by the first sensor and the current feature data of the second target collected by the second sensor is less than the preset difference, the first target and the second target are considered to be the same target, and the first target and the second target are initially fused to obtain an initial fused target, that is, the first sensor and the second sensor are associated. At this time, the current feature data of the fused target is the current feature data of the first target and the current feature data of the second target, and the association state of the fused target can also be recorded, that is, the fused target is associated through the first sensor and the second sensor at the current time point.

[0087] For example, the difference between the current position information of the first target collected by the first sensor and the current position information of the second target collected by the second sensor can be compared. If the difference between the current position information of the first target and the current position information of the second target is less than a preset difference, the first target and the second target are considered to be the same target, and the first target and the second target are initially fused to obtain an initial fused target.

[0088] For example, if sensor a collects 3 targets, namely a1, a2, and a3, and obtains the current feature data of each target, sensor b collects 4 targets, namely b1, b2, b3, and b4, and sensor d collects 2 targets, namely d1 and d2, and obtains the current feature data of each target. Then, the difference between the current feature data of each target collected by sensors a, b, and d is determined respectively. If the gap between the current feature data of targets a1, b2, and d2 is smaller than the preset gap, and the gap between the current feature data of target a3 and the current feature data of target b3 is smaller than the preset gap, then it is determined that targets a1, b2, and d2 are the same target, and targets a3 and b3 are the same target. Targets a1, b2, and d2 are initially fused to obtain an initial fused target c1, and targets a3 and b3 are initially fused to obtain an initial fused target c2. A2, b1, b4, and d1 are respectively used as initial fused targets. At this time, an initial fused target set including six initial fused targets c1, c2, a2, b1, b4, and d1 can be obtained. It should be noted that the above examples only take the possibility that the initial fusion target may be fused with 2 separate targets (initial fusion target c2) or 3 separate targets (initial fusion target c1) as an example. In specific implementation, 1 initial fusion target may be fused with a larger number of targets, for example, 1 initial fusion target may be fused with 5 targets, 7 targets, etc.

[0089] For ease of description, in this embodiment, the initial fusion target including at least two targets is recorded as the first type of fusion target, and the initial fusion target including one target is recorded as the second type of fusion target. For example, among the six initial fusion targets c1, c2, a2, b1, b4, and d1 in the above example, c1 and c2 belong to the first type of fusion targets, and a2, b1, and b4 belong to the second type of fusion targets.

[0090] After obtaining the initial fusion target set, in a possible implementation, the first type of fusion targets in the initial fusion target set may be further verified to determine whether at least two targets in the first type of fusion targets are indeed the same target.

[0091] In a possible implementation, if there is a first type of fusion target including at least two targets in the initial fusion target set, the following steps are performed:

[0092] Step 1: acquiring historical feature data of at least two targets based on at least two sensors that detect at least two targets;

[0093] Step 2: determining whether at least two targets in the first type of fusion targets are the same target based on historical feature data of at least two targets;

[0094] Step 3: If at least two targets are the same target, determine that the first type of fusion target belongs to the correctly fused first type of fusion target;

[0095] Step 4: If at least two targets are not the same target, it is determined that the first type of fusion target belongs to the initial fusion target of incorrect fusion, and the at least two targets are respectively used as separate second type of fusion targets;

[0096] Step 5: According to the correctly fused first-category fusion targets and the separate second-category fusion targets, the initial fusion target set is corrected, and the corrected fusion target set is used as the initial fusion target set.

[0097] That is to say, for the first type of fusion target including at least two targets, first obtain the historical feature data of at least two targets in the first type of fusion target, and determine whether the at least two targets are the same target according to the historical feature data of at least two targets. If they are the same target, it is determined that the first type of fusion target is correctly fused. If they are not the same target, the at least two targets are split and the at least two targets are respectively used as separate second type fusion targets to obtain the corrected initial fusion target set. Specifically, the first type of fusion target that is incorrectly fused can be split, and then each target in the first type of fusion target that is incorrectly fused can be used as a separate second type fusion target. For example, the first type of fusion target that is incorrectly fused includes the above-mentioned initial fusion target c1. Since c1 is fused with target a1, target b2 and target d2, c1 can be split so that target a1, target b2 and target d2 are all used as separate second type fusion targets. At this time, the corrected initial fusion target set includes: a1, b2, d2, c2, a2, b1, b4, d1.

[0098] The following describes steps 1 to 5 respectively:

[0099] In step 1, the historical feature data of the target is the feature data within a preset time period before the current time point. The preset time period can be pre-set, and the embodiment of the present application is not limited to this.

[0100] Each sensor corresponds to n queues, each queue is used to store historical feature data of a target detected by the sensor, and n is the number of targets that each sensor can detect.

[0101] The sensor collects the characteristic data of the target and can store the characteristic data in a queue, which is convenient for real-time updating and maintaining the historical trajectory of the target. Therefore, the historical characteristic data can be represented in the form of a queue, which is arranged in chronological order. Each data point in the queue of a certain target corresponds to the location information, speed information, motion state, motion direction and other attribute information of the target at a point in time. If the target disappears, its queue will be cleared. If a new target appears, a new queue will be established based on the newly appeared target to store the attribute information of the newly appeared target. Moreover, the queue is usually full of 10 frames. When the queue is full, the queue will automatically roll and update, and the earliest stored attribute information will be overwritten in sequence with the attribute information collected at the next time point.

[0102] After detecting a target, the sensor will continue to track the target, so the sensor can collect the attribute information of the target at each time point, and store the attribute information collected at each time point in the form of a queue, so that the historical feature data of each target can be obtained.

[0103] After obtaining the historical feature data of at least two targets, it can be determined based on step 2 whether at least two targets in the first type of fusion targets are the same target.

[0104] In a possible implementation, step 2 is the process of determining target similarity between track and sensor, where track refers to the fusion target and sensor refers to the target detected by the sensor.

[0105] The implementation process of step 2 may be the following steps 21 to 24:

[0106] Step 21: determining second target similarities between the at least two targets and the first type of fusion targets respectively according to the historical feature data of the at least two targets and the historical feature data of the first type of fusion targets;

[0107] Step 22: Obtaining the association status of at least two sensors with the first type of fusion target respectively within a preset time period;

[0108] Step 23: determining, according to the association states, the similarity of the association states of at least two sensors and the first type of fusion target;

[0109] Step 24: Determine whether at least two targets are the same target based on the second target similarity and the association state similarity.

[0110] The historical feature data of the first type of fusion target can be obtained based on the historical feature data of at least two targets included in the first type of fusion target.

[0111] For each of the at least two targets included in the first type of fusion targets, the first target similarity between each target and the first type of fusion targets is determined in the following manner. Therefore, the following is an example of determining the first target similarity between the first type of fusion targets and the first target, where the first target is any target included in the first type of fusion targets.

[0112] In some embodiments, the implementation process of step 21 can be divided into the following two steps, namely step 211 to step 212:

[0113] Step 211: Based on the attribute information of multiple dimensions of the first target and the attribute information of multiple dimensions of the first type of fused targets, determine the attribute similarity between the first target and the first type of fused targets in the same dimension.

[0114] Step 212: Determine the second target similarity between the first target and the first type of fusion targets based on the attribute similarity between the first target and the first type of fusion targets in the same dimension.

[0115] The multiple dimensions may include a position dimension, a speed dimension, a motion state dimension, a motion direction dimension, etc. Correspondingly, the attribute information of the multiple dimensions may include the target's position information, speed information, motion state, motion direction, and other attribute information.

[0116] Among them, historical feature data, i.e., attribute information of multiple dimensions, is stored in the form of a queue, and a queue is full of 10 frames. Therefore, each of the 10 frames of the first target includes the attribute information of the first target at the current time point, and each of the 10 frames of the first type of fusion target includes the attribute information of each target included in the first type of fusion target at the current time point.

[0117] In a scenario where the attribute information of multiple dimensions includes speed information, motion status, and motion direction, for example, the implementation process of step 211 may be: for the attribute information of any dimension in the attribute information of multiple dimensions, compare the difference between the attribute information of the first target and the first type of fusion target at each time point within a preset time period; determine a first ratio of the time length that the difference is within the preset difference to the time length of the preset time period; based on the first ratio, determine the attribute similarity of the first target and the first type of fusion target in the same dimension.

[0118] The preset difference may be set in advance, and when the dimensions are different, the preset difference may be the same or different.

[0119] Exemplarily, for attribute information that can be expressed by numerical values, the difference between the attribute information is also a numerical value, and the preset difference corresponding to the attribute information of this dimension can also be expressed by a numerical value. Speed ​​information is a numerical value used to characterize the movement speed of the target, and the preset difference corresponding to the speed dimension can be a preset speed difference value. Since there may be performance differences between multiple sensors, the speed values ​​collected by different sensors for the same target may also be different, but usually, the difference between the speed values ​​collected by these sensors will not be very large. Therefore, the difference between the sensors can be ignored, that is, two targets with very small speed differences can be ignored, and the situation where the difference between the speed information of the two targets is within the preset difference is regarded as the speed of the two targets being consistent. Therefore, when the attribute information of the dimension is speed information, the preset difference can be a very small difference, indicating that the speed information difference of the two targets is very small and can be regarded as consistent.

[0120] For example, for attribute information that is usually not expressed in numerical values, such as motion direction, motion state, etc., the difference between the attribute information can be divided into two cases: zero difference and difference. For example, if the motion states of two targets are consistent, it means that the difference between the two targets in the motion state dimension is zero difference. If the motion states of two targets are inconsistent, it means that the difference between the two targets in the motion state dimension is difference.

[0121] Specifically, when determining the attribute similarity between the first target and the first type of fusion target for a certain dimension, the differences between the attribute information at each time within the preset time period are compared respectively, and the proportion of time within the preset time period in which the difference is within the preset difference is determined, and based on the proportion, the attribute similarity between the first target and the first type of fusion target in this dimension is determined.

[0122] Among them, it is assumed that the queue used to store the historical feature data of the first target includes 10 frames, and the queue used to store the historical feature data of the first type of fusion target includes 10 frames, each of the 10 frames of the first target includes the attribute information of the first target at the current time point, and each of the 10 frames of the first type of fusion target includes the attribute information of each target included in the first type of fusion target at the current time point.

[0123] For example, when the attribute information is the motion state, the difference between the motion state of the first target and the first type of fusion target in each of the 10 frames is compared, that is, whether the motion state of the first target and the first type of fusion target is consistent in each frame is compared, and the proportion of the consistent motion state in these 10 frames is determined, and the proportion is used as the attribute similarity of the motion state of the first target and the first type of fusion target. If the motion state of the first target and the first type of fusion target is consistent in 4 frames out of 10 frames, that is, the proportion of the consistent motion state of the first target and the first type of fusion target is 4 / 10, then it can be considered that the attribute similarity of the first target and the first type of fusion target in the dimension of motion state is 0.4.

[0124] When the attribute information is speed information, the preset difference can be set to 1, and the difference between the speeds of the first target and the first type of fusion target in each of the 10 frames can be compared, that is, whether the speed difference between the first target and the first type of fusion target in each frame is within the preset difference, and the case within the preset difference is regarded as the first target and the first type of fusion target having the same speed in this frame. And determine the proportion of consistent speed information in these 10 frames, and use this proportion as the attribute similarity of the first target and the first type of fusion target in speed. If the speed information of the first target and the first type of fusion target is consistent in 5 of the 10 frames, that is, the proportion of the speed information of the first target and the first type of fusion target is consistent is 5 / 10, then it can be considered that the attribute similarity of the first target and the first type of fusion target in the dimension of speed is 0.5.

[0125] When the attribute information is position information, the similarity of the motion trends of the first target and the first type of fusion target can be determined based on the position changes of the first target and the first type of fusion target within a preset time period. Based on this, for example, the implementation process of step 211 can be: based on the position information of the first target and the first type of fusion target within a preset time period, determine the position change trend of the first target and the first type of fusion target; based on the position change trend, determine the movement direction of the first target and the first type of fusion target; based on the movement direction, determine the attribute similarity of the first target and the first type of fusion target in the dimension of position. Among them, the movement direction of the target represents the future direction of the target predicted based on the position change trend of the target.

[0126] Specifically, for any target, the change trend of the latest position of the target relative to the previous position can be determined based on the position information of the target at each time in the preset time period, and then the movement direction of the target can be determined. After the movement direction is obtained, the attribute similarity of the two targets in the position information can be determined by comparing whether the movement directions of the two targets are consistent. If the movement directions of the two targets are consistent, the attribute similarity of the two targets in the position information is 1, otherwise it is 0.

[0127] For example, according to the position information of the first target at each time point in the preset time period, it is determined that the position change trend of the first target is gradually moving eastward, so the movement direction of the first target can be determined to be in the direction toward the east. Assume that according to the position information of the first type of fusion target in the preset time period, it is determined that the previous positions of the first type of fusion target and the first target are the same, and the position change trend of the first type of fusion target is gradually moving eastward, and then the movement direction of the first type of fusion target is determined to be in the direction toward the east. Therefore, the movement directions of the first target and the first type of fusion target are consistent, and the attribute similarity of the first target and the first type of fusion target in the dimension of position is 1.

[0128] In addition, when determining the attribute similarity above, some problem scenarios can be specially processed, such as removing these targets before determining the target similarity. This can reduce some data processing work to improve efficiency. If the difference between the position information in the historical feature data collected by the two sensors is large, it can be considered that the two targets to which the historical feature data collected by the two sensors belong are not the same target, and the subsequent operation of determining the target similarity can be omitted, reducing the workflow. Therefore, it can be considered that the above operation of determining the target similarity is based on the fact that the difference between the position information in the historical feature data collected by each sensor is not large.

[0129] If the historical characteristic data of the target collected by the sensor shows that the target has been in a stopped state for a long time, such as the target is in a stopped state in 8 or 9 frames out of 10 frames, such a target is not referenceable and therefore can be left unprocessed, that is, no operation is performed to determine the target similarity.

[0130] After respectively obtaining the attribute similarities between the first target and the first type of fused targets in the speed dimension, the motion state dimension, the motion direction dimension, and the position dimension, the second target similarity between the first target and the first type of fused targets can be determined based on step 212 .

[0131] As an example, a weight can be set for each dimension according to the importance of each dimension, and then the similarity of the second target between the first target and the first type of fusion target is determined according to the attribute similarity of the first target and the first type of fusion target on the same dimension and the weight of the corresponding dimension. The higher the weight, the higher the importance of the dimension.

[0132] For example, the position dimension is the most important among multiple dimensions, so the weight of the position dimension can be set to 50%, the weight of the motion state dimension can be set to 10%, the weight of the speed dimension can be set to 20%, and the weight of the motion direction dimension can be set to 20%. The sum of the weights of these four dimensions is 100%.

[0133] Specifically, the attribute similarity of the first target and the first type of fusion target in the same dimension can be multiplied by the weight of the corresponding dimension, and the value obtained by summing up the multiplied values ​​is used as the second target similarity between the first target and the first type of fusion target. For example, the attribute similarity of the first target and the first type of fusion target in position is 1, the attribute similarity in motion state is 0.4, the attribute similarity in speed is 0.5, and the attribute similarity in motion direction is 0.4, then the second target similarity between the first target and the first type of fusion target can be determined as: position similarity × 50% + motion state similarity × 10% + speed similarity × 20% + motion direction similarity × 20% = 1 × 50% + 0.4 × 10% + 0.5 × 20% + 0.4 × 20% = 0.72.

[0134] As another example, it is also possible to assume that the importance of each dimension is the same, and directly sum up the attribute similarities of each dimension and take the average value, and use the average value as the second target similarity between the first target and the first type of fusion target, that is, the second target similarity = 1 / 4 (position information similarity + motion state similarity + speed information similarity + motion direction similarity).

[0135] After obtaining the second target similarity between the first target and the first type of fused target, the association state of the first sensor and the first type of fused target can be obtained based on step 22, wherein the first sensor is a sensor for detecting the first target.

[0136] If the first type of fusion target is obtained by fusing target 1 and target 2 at the current time point, target 1 is detected by sensor 1, and target 2 is detected by sensor 2, the association state of the first type of fusion target at the current time point can be obtained, that is, the first type of fusion target is associated through sensor 1 and sensor 2 at the current time point.

[0137] Therefore, the embodiment of the present application can also obtain the association status of the first type of fusion target and the sensor in each of the 10 frames of the first type of fusion target, and determine how many frames in the 10 frames of the first type of fusion target are associated with sensor 1, and accordingly, determine how many frames in the 10 frames of the first type of fusion target are associated with sensor 2, that is, obtain the association status of the first type of fusion target with sensor 1 and sensor 2 respectively. For example, 7 frames in the 10 frames of the first type of fusion target are associated with sensor 1, and 5 frames are associated with sensor 2.

[0138] When the association states of the at least two sensors and the first type of fusion target are obtained, the similarity of the association states of the at least two sensors and the first type of fusion target may be determined based on step 23 .

[0139] Specifically, the ratio of the number of frames in which the first sensor is associated with the first type of fusion target in a preset time period to 10 frames can be determined, thereby determining the similarity of the association state between the first sensor and the first type of fusion target. For example, if 7 out of 10 frames of the first type of fusion target are associated with the first sensor, it can be determined that the similarity of the association state between the first sensor and the first type of fusion target is 0.7.

[0140] For at least two targets included in the first type of fusion targets, the operations of steps 21 to 23 are performed, that is, the first target similarity between the at least two targets and the first type of fusion targets, and the association state similarity between the at least two sensors and the first type of fusion targets can be obtained. Then, based on step 24, it is determined whether the at least two targets are the same target according to the first target similarity and the association state similarity.

[0141] For example, a weight may be set for the association state, and the value obtained by multiplying the association state similarity by its weight is summed with the attribute similarity (second target similarity) to obtain the third target similarity between the first target and the first type of fusion target.

[0142] For example, the weight of the association state can be set to 20%, and the weights of each dimension are as in the above example. Then, the third target similarity between the first target and the first type of fusion target can be determined as: position information similarity × 50% + motion state similarity × 10% + speed information similarity × 20% + motion direction similarity × 20% + association state similarity × 20%. In this scenario, the sum of the weights of the association state and the other four dimensions is 120%. When the association state of the sensor is included, since the weight of the association state is 20%, the sum of the weights obtained by adding the association state is 120%. In this case, the full score of the third target similarity is 1.2.

[0143] In the case where the first category of fusion targets includes the first target and the second target, if the similarity of the third target between the first target and the second target and the first category of fusion targets is higher than the first threshold, that is, the difference between the first target and the second target and the first category of fusion targets is small, then it can be considered that the first target and the second target in the first category of fusion targets are indeed the same target. If the similarity of the third target between the first target and the second target and the first category of fusion targets is lower than the first threshold, that is, the difference between the first target and the second target and the first category of fusion targets is large, then it can be considered that the first target and the second target in the first category of fusion targets are not the same target. Among them, the first threshold can be set in advance, and the embodiments of the present application are not limited to this.

[0144] If the first target and the second target are the same target, the first type of fusion target is determined to be a correctly fused first type of fusion target. If the first target and the second target are not the same target, the first type of fusion target is determined to be an incorrectly fused initial fusion target, the first sensor detecting the first target and the second sensor detecting the second target are disassociated, and the first target and the second target are respectively used as separate second type of fusion targets.

[0145] Figure 2 FIG. 1 is a schematic diagram of a sensor de-association method provided in an embodiment of the present application. Figure 2As shown in the figure, the left figure shows two sensors (front radar and camera) at the current time point. Since the current feature data collected by the two sensors have a small gap before, the two sensors are in an associated state at the current time point. The dotted line connecting the front radar and the camera in the left figure indicates that the two sensors are associated. Therefore, it can be considered that the first type of fusion target includes target 1 detected by the front radar and target 2 detected by the camera. After adding the historical trajectory, that is, considering the historical feature data collected by the sensor, it is assumed that the third target similarity between target 1 and target 2 and the first type of fusion target is small, that is, target 1 and target 2 are different targets, so target 1 and target 2 can be defused, and target 1 and target 2 can be used as a second type of fusion target respectively, that is, the front radar and the camera are decoupled, as shown in the right figure. In the right figure, there is no dotted line connecting the front radar and the camera, which means that the two sensors are decoupled.

[0146] By performing the above steps 1 to 4 for each first-class fusion target, a correctly fused first-class fusion target and a separate second-class fusion target can be obtained, and then the corrected first-class fusion target and the separate second-class fusion target are used to correct the initial fusion target set, and the corrected fusion target set is used as the initial fusion target, so that the initial fusion target can be adjusted according to the historical feature data of each initial fusion target.

[0147] Step 103: Acquire historical feature data of each initial fusion target in the initial fusion target set based on multiple sensors.

[0148] As mentioned above, the historical feature data of the initial fusion target can be obtained based on the historical feature data of each target included in the initial fusion target. It should be understood that the initial fusion target here is the corrected initial fusion target. Therefore, the historical feature data of the initial fusion target is also obtained based on the historical feature data of each target included in the corrected initial fusion target. That is, based on the historical feature data stored in the queues corresponding to multiple sensors, the historical feature data of each initial fusion target in the initial fusion target set is obtained.

[0149] Step 104: Based on the historical feature data, determine the first target similarity between the initial fusion targets.

[0150] Among them, step 104 is the process of determining the target similarity of track-track. In some embodiments, the implementation process of step 104 can be divided into two steps:

[0151] Step 1041: determining the attribute similarity of each initial fusion target in the same dimension based on the attribute information of multiple dimensions of each initial fusion target;

[0152] Step 1042: Determine the first target similarity between the initial fusion targets based on the attribute similarity of the initial fusion targets in the same dimension.

[0153] In a scenario where the attribute information of multiple dimensions includes speed information, motion status, and motion direction, for example, the implementation process of step 1041 may be: for the attribute information of any dimension in the attribute information of multiple dimensions, compare the differences between the attribute information of each initial fusion target at each time point within a preset time period; determine a first ratio of the time length of the difference within the preset difference to the time length of the preset time period; based on the first ratio, determine the attribute similarity of each initial fusion target in the same dimension.

[0154] Specifically, for two initial fusion targets, when determining the attribute similarity of the two initial fusion targets for a certain dimension, the differences between the attribute information at each time within the preset time period are compared respectively, and the proportion of the time within the preset time period that the difference is within the preset difference is determined, and based on the proportion, the attribute similarity of the two initial fusion targets in this dimension is determined.

[0155] When the attribute information is location information, for example, the implementation process of step 1041 may be: based on the location information of each initial fusion target within a preset time period, determine the location change trend of each initial fusion target; based on the location change trend, determine the movement direction of each initial fusion target; based on the movement direction, determine the attribute similarity of each initial fusion target in the location dimension. The movement direction of the initial fusion target indicates the future direction of the target predicted according to the location change trend of the initial fusion target.

[0156] Specifically, the implementation process of the above steps can refer to the relevant content of the above step 21 when the attribute information is location information, which will not be repeated here.

[0157] After obtaining the attribute similarities of each initial fusion target in the speed dimension, motion state dimension, motion direction dimension, and position dimension, the first target similarity between each initial fusion target can be determined based on step 1042. The sum of the weights of the four dimensions of speed information, motion state, motion direction, and position information is 100%, so in this case, the first target similarity has a full score of 1. The first target similarity here is equivalent to the second target similarity in the above step 102, so the implementation process of determining the first target similarity in step 104 can refer to the relevant content of determining the second target similarity in the above step 102.

[0158] Step 105: Based on the first target similarity, the fusion relationship between the initial fusion targets is adjusted to obtain a fusion target set.

[0159] Since the historical feature data of the initial fusion target reflects the historical trajectory of the initial fusion target, and the first target similarity is obtained based on the historical feature data, the target similarity reflects the similarity between the historical trajectories of the initial fusion targets. Therefore, combined with the first target similarity, at least two sensors that detect the same initial fusion target can be more accurately associated.

[0160] In the case where the initial fusion target set includes a first initial fusion target and a second initial fusion target, and the first initial fusion target and the second initial fusion target are any two initial fusion targets in the initial fusion target set, if the first target similarity between the first initial fusion target and the second initial fusion target is higher than a first threshold, then it is determined that the first initial fusion target and the second initial fusion target are the same target, and the first initial fusion target and the second initial fusion target are fused to obtain the fusion target set.

[0161] That is to say, when the current feature data shows that there is a large difference between the two initial fusion targets and they may be two independent initial fusion targets, if the first target similarity between the two initial fusion targets is high, then the two initial fusion targets are considered to be the same target. Therefore, the two initial fusion targets can be fused, that is, the multiple sensors used to detect the two initial fusion targets can be associated.

[0162] Figure 3 is a schematic diagram of a sensor association provided in an embodiment of the present application. Figure 3 As shown in the figure on the left, three sensors (front radar, front corner radar and camera) are shown at the current time point. Among them, the front radar detects target 1, the front corner radar detects target 2, and the camera detects target 3. At the current time point, the difference between the current feature data of target 1 and the current feature data of target 2 is small, and the difference between the current feature data of target 1 and the current feature data of target 2 and the current feature data of target 3 is large. Therefore, target 1 and target 2 are fused to obtain an initial fused target A, that is, the front radar and the front corner radar are associated, and target 3 is taken as an initial fused target B. As shown in the figure on the left, the dotted line connecting the front radar and the front corner radar indicates that the two sensors are associated. Therefore, Figure 3 As can be seen in the left figure, the front radar and the front corner radar are associated, and these two radars are not associated with the camera. After adding the historical trajectory, that is, considering the historical feature data collected by the sensor, it is assumed that the first target similarity between the two initial fusion targets is high, so the initial fusion target A and the initial fusion target B are fused, that is, the two sensors (front radar and front corner radar) are associated with the camera, as shown in the right figure. The dotted line connecting the front radar, front corner radar and camera in the right figure indicates that the two sensors are associated with the camera.

[0163] Step 106: Based on the current feature data, track each fusion target in the fusion target set.

[0164] In the scenario where the first initial fusion target and the second initial fusion target belonging to the same target are fused based on step 105, in some embodiments, the implementation process of step 106 may be: fusing the current feature data of the first initial fusion target and the current feature data of the second initial fusion target to obtain processed current feature data; and tracking the same target based on the processed current feature data.

[0165] Among them, step 105 fuses the first initial fusion target and the second initial fusion target, that is, the first sensor detecting the first initial fusion target and the second sensor detecting the second initial fusion target are associated, so that the same target is tracked based on the current feature data after fusion processing, which can improve the stability of target tracking.

[0166] For example, the way of fusing the current feature data of the first initial fusion target and the current feature data of the second initial fusion target may be: determining an average value of the two current feature data, and using the average value as the current feature data after fusion processing.

[0167] Optionally, the method of fusing the current feature data of the first initial fusion target and the current feature data of the second initial fusion target is not limited to the above-mentioned method, and other processing methods may also be adopted, such as taking the maximum value of the current feature data of the two initial fusion targets as the processed current feature data, or taking the minimum value of the current feature data of the two initial fusion targets as the processed current feature data, etc. The embodiments of the present application are not limited to this.

[0168] In an embodiment of the present application, the current feature data of each target around the vehicle is obtained by multiple sensors on the vehicle, and each target is initially fused according to the current feature data of each target, and then the target similarity between each initial fusion target is determined according to the historical feature data of the initial fusion target, and the fusion relationship between each initial fusion target is adjusted according to the target similarity. Since the historical feature data of each initial fusion target reflects the movement trend of each initial fusion target over a period of time, it is possible to more accurately judge whether two initial fusion targets are the same target in combination with the target similarity determined by the historical feature data, and then adjust the fusion relationship between the initial fusion targets to improve the accuracy of target fusion. The adjusted fusion target is accurately tracked in combination with the current feature data, which is conducive to improving the stability of target tracking. Moreover, the embodiment of the present application matches and tracks the target based on the historical trajectory of the target, which can improve the globality and robustness of target detection and recognition. In addition, the target tracking method of the embodiment of the present application can be applied to various computer vision scenarios, such as intelligent monitoring, automatic driving and other scenarios. Moreover, accurate tracking of the target can improve the safety level of these scenarios and can more realistically reflect the reality of these scenarios.

[0169] All the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and the embodiments of the present application will not be described in detail one by one.

[0170] Figure 4 is a schematic diagram of the structure of a target tracking device provided in an embodiment of the present application. Figure 4 As shown, the device 400 includes:

[0171] A first acquisition module 401 is used to acquire current feature data of each target around the vehicle based on multiple sensors installed on the vehicle;

[0172] A fusion module 402 is used to initially fuse each target according to the current feature data to obtain an initial fusion target set;

[0173] The second acquisition module 403 is used to acquire historical feature data of each initial fusion target in the initial fusion target set based on multiple sensors;

[0174] A determination module 404, configured to determine the first target similarity between the initial fusion targets based on the historical feature data;

[0175] An adjustment module 405 is used to adjust the fusion relationship between the initial fusion targets based on the first target similarity to obtain a fusion target set;

[0176] The tracking module 406 is used to track each fusion target in the fusion target set based on the current feature data.

[0177] Optionally, the historical feature data of the initial fusion target includes attribute information of multiple dimensions of the initial fusion target within a preset time period before the current time point; the determination module 404 is also used to: determine the attribute similarity of each initial fusion target in the same dimension based on the attribute information of multiple dimensions of each initial fusion target; determine the first target similarity between each initial fusion target based on the attribute similarity of each initial fusion target in the same dimension.

[0178] Optionally, the determination module 404 is also used to: compare the differences between the attribute information of each initial fusion target at each time point within a preset time period for the attribute information of any dimension among the attribute information of multiple dimensions; determine a first ratio of the time length of the difference within the preset difference to the time length of the preset time period; based on the first ratio, determine the attribute similarity of each initial fusion target in the same dimension.

[0179] Optionally, the attribute information of multiple dimensions includes location information; the determination module 404 is also used to: determine the location change trend of each initial fusion target based on the location information of each initial fusion target within a preset time period; determine the movement direction of each initial fusion target based on the location change trend; determine the attribute similarity of each initial fusion target in the location information based on the movement direction.

[0180] Optionally, the adjustment module 405 is also used to: if the target similarity between the first initial fusion target and the second initial fusion target is higher than a first threshold, determine that the first initial fusion target and the second initial fusion target are the same target, and fuse the first initial fusion target and the second initial fusion target to obtain a fusion target set, wherein the first initial fusion target and the second initial fusion target are any two initial fusion targets in the initial fusion target set; the tracking module 406 is also used to: fuse the current feature data of the first initial fusion target and the current feature data of the second initial fusion target to obtain processed current feature data; and track the same target based on the processed current feature data.

[0181] Optionally, the fusion module 402 is also used for: if there is a first type of fusion target including at least two targets in the initial fusion target set, obtaining historical feature data of the at least two targets based on at least two sensors that detect the at least two targets; determining whether at least two targets in the first type of fusion target are the same target based on the historical feature data of the at least two targets; if at least two targets are the same target, determining that the first type of fusion target belongs to the correctly fused first type of fusion target; if at least two targets are not the same target, determining that the first type of fusion target belongs to the incorrectly fused initial fusion target, and treating the at least two targets as separate second type fusion targets respectively; correcting the initial fusion target set based on the correctly fused first type fusion target and the separate second type fusion target, and using the corrected fusion target set as the initial fusion target set.

[0182] Optionally, the fusion module 402 is also used to: determine the second target similarity between the at least two targets and the first type of fusion target respectively based on the historical feature data of the at least two targets and the historical feature data of the first type of fusion target; obtain the association status of the at least two sensors and the first type of fusion target respectively within a preset time period; determine the association status similarity between the at least two sensors and the first type of fusion target based on the association status; determine whether the at least two targets are the same target based on the second target similarity and the association status similarity.

[0183] Optionally, the fusion module 402 is also used to: if the gap between the current feature data of the first target and the current feature data of the second target is smaller than a preset gap, initially fuse the first target and the second target to obtain an initial fused target set, where the first target is the target detected by the first sensor, the second target is the target detected by the second sensor, and the first sensor and the second sensor are any two sensors among the multiple sensors.

[0184] Optionally, each sensor corresponds to n queues, each queue is used to store historical feature data of a target detected by the sensor, and n is the number of targets that each sensor can detect; the second acquisition module 403 is also used to: based on the historical feature data stored in the queues corresponding to multiple sensors, obtain the historical feature data of each initial fusion target in the initial fusion target set.

[0185] Figure 5 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0186] For example, Figure 5 As shown, the electronic device 500 includes: a memory 501 and a processor 502, wherein the memory 501 stores an executable program code 5011, and the processor 502 is used to call and execute the executable program code 5011 to perform a target tracking method.

[0187] In this embodiment, the electronic device can be divided into functional modules according to the above method example. For example, each functional module can be corresponded to, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0188] In the case of dividing each functional module according to each function, the electronic device may include: a first acquisition module, a fusion module, a second acquisition module, a determination module, an adjustment module, and a tracking module. It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0189] The electronic device provided in this embodiment is used to execute the above-mentioned target tracking method, and thus can achieve the same effect as the above-mentioned implementation method.

[0190] In the case of an integrated unit, the electronic device may include a processing module and a storage module. The processing module may be used to control and manage the actions of the electronic device. The storage module may be used to support the electronic device in executing mutual program codes and data.

[0191] The processing module may be a processor or a controller, which may implement or execute various exemplary logic blocks, modules and circuits disclosed in the present application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and a microprocessor, etc. The storage module may be a memory.

[0192] This embodiment also provides a computer-readable storage medium, in which a computer program code is stored. When the computer program code is executed on a computer, the computer executes the above-mentioned related method steps to implement a target tracking method in the above-mentioned embodiment.

[0193] This embodiment also provides a computer program product. When the computer program product runs on a computer, it enables the computer to execute the above-mentioned related steps to implement a target tracking method in the above-mentioned embodiment.

[0194] In addition, the electronic device provided in the embodiments of the present application may include a connected processor and a memory; wherein the memory is used to store instructions, and when the electronic device is running, the processor may call and execute the instructions so that the electronic device performs a target tracking method in the above-mentioned embodiments.

[0195] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0196] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0197] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0198] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A target tracking method, characterized in that: The method comprises: Acquire current feature data of each target around the vehicle based on a plurality of sensors installed on the vehicle; According to the current feature data, initially fusing each of the targets to obtain an initial fused target set; Acquire historical feature data of each of the initial fusion targets in the initial fusion target set based on the multiple sensors; Based on the historical feature data, determining the first target similarity between the initial fusion targets; Based on the first target similarity, adjusting the fusion relationship between the initial fusion targets to obtain a fusion target set; Based on the current feature data, target tracking is performed on each fusion target in the fusion target set.

2. The method according to claim 1, characterized in that The historical feature data of the initial fusion target includes attribute information of multiple dimensions of the initial fusion target within a preset time period before the current time point; The determining, based on the historical feature data, the first target similarity between the initial fusion targets includes: Based on the attribute information of multiple dimensions of each of the initial fusion targets, determining the attribute similarity of each of the initial fusion targets in the same dimension; Based on the attribute similarity of each of the initial fusion targets in the same dimension, the first target similarity between each of the initial fusion targets is determined.

3. The method according to claim 2, characterized in that The determining, based on the attribute information of multiple dimensions of each of the initial fusion targets, the attribute similarity of each of the initial fusion targets in the same dimension includes: For the attribute information of any dimension among the attribute information of the multiple dimensions, comparing the differences between the attribute information of each of the initial fusion targets at each time point within the preset time period; Determine a first ratio of the length of time that the difference is within a preset difference to the length of time of the preset time period; Based on the first ratio, attribute similarity of each of the initial fusion targets in the same dimension is determined.

4. The method according to claim 2, characterized in that: The attribute information of the multiple dimensions includes location information; The determining, based on the attribute information of multiple dimensions of each of the initial fusion targets, the attribute similarity of each of the initial fusion targets in the same dimension includes: Based on the position information of each of the initial fusion targets within the preset time period, determining the position change trend of each of the initial fusion targets; Based on the position change trend, determining the movement direction of each of the initial fusion targets; Based on the movement direction, attribute similarity of each of the initial fusion targets on the position information is determined.

5. The method according to any one of claims 1 to 4, characterized in that: The step of adjusting the fusion relationship between the initial fusion targets based on the first target similarity to obtain a fusion target set includes: If the first target similarity between the first initial fusion target and the second initial fusion target is higher than a first threshold, determining that the first initial fusion target and the second initial fusion target are the same target, and fusing the first initial fusion target and the second initial fusion target to obtain the fusion target set, wherein the first initial fusion target and the second initial fusion target are any two initial fusion targets in the initial fusion target set; The tracking of each fusion target in the fusion target set based on the current feature data includes: fusing the current feature data of the first initial fusion target and the current feature data of the second initial fusion target to obtain processed current feature data; The same target is tracked based on the processed current feature data.

6. The method according to claim 1, characterized in that After initially fusing each of the targets according to the current feature data to obtain an initial fusion target set, the method further includes: If there are first-category fusion targets including at least two targets in the initial fusion target set, acquiring historical feature data of the at least two targets based on at least two sensors that detect the at least two targets; Determining whether at least two targets in the first type of fusion targets are the same target according to the historical feature data of the at least two targets; If the at least two targets are the same target, determining that the first type of fusion target belongs to the correctly fused first type of fusion target; If the at least two targets are not the same target, determining that the first type of fusion target belongs to an initial fusion target of incorrect fusion, and using the at least two targets as separate second type of fusion targets respectively; The initial fusion target set is corrected according to the correctly fused first-category fusion targets and the separate second-category fusion targets, and the corrected fusion target set is used as the initial fusion target set.

7. The method according to claim 6, characterized in that The determining, based on the historical feature data of the at least two targets, whether at least two targets in the first type of fusion targets are the same target includes: Determining second target similarities between the at least two targets and the first type of fusion targets respectively according to the historical feature data of the at least two targets and the historical feature data of the first type of fusion targets; Acquire the association status of the at least two sensors and the first type of fusion target respectively within a preset time period; Determining, according to the association state, the similarity of the association states of the at least two sensors and the first type of fusion target; According to the second target similarity and the association state similarity, it is determined whether the at least two targets are the same target.

8. The method according to claim 1, characterized in that: The initial fusion of each target is performed according to the current feature data to obtain an initial fusion target set, including: If the gap between the current feature data of the first target and the current feature data of the second target is less than a preset gap, the first target and the second target are initially fused to obtain the initial fused target set, where the first target is a target detected by the first sensor, the second target is a target detected by the second sensor, and the first sensor and the second sensor are any two sensors among the multiple sensors.

9. The method according to claim 1, characterized in that: Each of the sensors has n queues corresponding to it, and each of the queues is used to store historical feature data of a target detected by the sensor, where n is the number of targets that each sensor can detect; The acquiring, based on the multiple sensors, the historical feature data of each of the initial fusion targets in the initial fusion target set includes: Based on the historical feature data stored in the queues corresponding to the multiple sensors, the historical feature data of each of the initial fusion targets in the initial fusion target set is obtained.

10. A target tracking device, characterized in that: The device comprises: A first acquisition module, configured to acquire current feature data of each target around the vehicle based on a plurality of sensors installed on the vehicle; A fusion module, used to perform initial fusion on each of the targets according to the current feature data to obtain an initial fusion target set; A second acquisition module, configured to acquire historical feature data of each of the initial fusion targets in the initial fusion target set based on the multiple sensors; A determination module, configured to determine the first target similarity between the initial fusion targets based on the historical feature data; An adjustment module, configured to adjust the fusion relationship between the initial fusion targets based on the first target similarity to obtain a fusion target set; A tracking module is used to track each fusion target in the fusion target set based on the current feature data.

11. An electronic device, characterized in that: The electronic device comprises: A memory for storing executable program codes; A processor, configured to call and run the executable program code from the memory, so that the electronic device executes the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claims 1 to 9 is implemented.

Images