Track management method and device, computer equipment, readable storage medium and program product

Abstract

The invention relates to a track management method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring camera data and radar data at the current moment every preset time; based on the camera data and the radar data, updating a multi-sensor fusion track at the previous moment to obtain an updated track; determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment; and based on a flight path fusion strategy, performing flight path fusion on the camera target for fusion and the radar target for fusion, and obtaining a multi-sensor fusion flight path at the current moment, a camera flight path at the current moment and a radar flight path at the current moment in combination with the updated flight path. By adopting the method, a more accurate multi-sensor fusion track can be obtained.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a track management method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Art

[0002] As a core component of the intelligent driving technology system, perception fusion plays a vital role. It is not only a means of data processing, but also a key bridge to achieve highly automated driving. Specifically, perception fusion can cleverly integrate and analyze the raw data from multiple sensors based on the feature information captured by different sensors to generate a more comprehensive, stable and accurate road environment model.

[0003] The track fusion results generated by existing machine learning-based methods are uncertain, and the learning results are limited due to insufficient data sets, resulting in low accuracy of the fused tracks. Summary of the invention

[0004] Based on this, it is necessary to provide a track management method, device, computer equipment, computer-readable storage medium and computer program product that can more accurately obtain multi-sensor fusion tracks in response to the above technical problems.

[0005] In a first aspect, the present application provides a track management method, comprising:

[0006] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0007] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0008] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0009] In one embodiment, the tracking fusion strategy is based on which the camera target for fusion and the radar target for fusion are track-fused, including:

[0010] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0011] In one embodiment, updating the multi-sensor fusion track at a previous moment based on the camera data and the radar data to obtain an updated track includes:

[0012] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0013] In one embodiment, the fusion threshold interval is generated as follows:

[0014] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0015] In one embodiment, the combining and updating the track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment includes:

[0016] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

[0017] In one of the embodiments, the determining the camera target for fusion based on the camera data and the camera track at the previous moment includes: obtaining a camera target newly appearing at the current moment from the camera data, obtaining a camera target of the camera track at the previous moment, and using the camera target newly appearing at the current moment and the camera target of the camera track at the previous moment as the camera target for fusion;

[0018] The determining of the radar target for fusion based on the radar data and the radar track at the previous moment includes: acquiring a radar target newly appearing at the current moment from the radar data, acquiring a radar target of the radar track at the previous moment, and using the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment as the radar targets for fusion.

[0019] In a second aspect, the present application also provides a track management device, including:

[0020] An updating module, used for acquiring the camera data and the radar data at the current moment at a preset time interval; based on the camera data and the radar data, updating the multi-sensor fusion track at the previous moment to obtain an updated track;

[0021] An acquisition module, used to determine a camera target for fusion based on the camera data and the camera track at the previous moment; and to determine a radar target for fusion based on the radar data and the radar track at the previous moment;

[0022] The fusion module is used to fuse the camera target for fusion and the radar target for fusion based on the track fusion strategy, and combine the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0023] In a third aspect, the present application further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0024] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0025] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0026] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0027] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the following steps are implemented:

[0028] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0029] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0030] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0032] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0033] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0034] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0035] The above-mentioned track management method, device, computer equipment, computer-readable storage medium and computer program product obtain the camera data and radar data at the current moment at a preset time interval; based on the camera data and the radar data, update the multi-sensor fusion track at the previous moment to obtain an updated track; based on the camera data and the camera track at the previous moment, determine the camera target for fusion; based on the radar data and the radar track at the previous moment, determine the radar target for fusion; based on the track fusion strategy, fuse the camera target for fusion and the radar target for fusion, and combine the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment and the radar track at the current moment. Generating a multi-sensor fusion track based on a unique fusion strategy can make the fused multi-sensor fusion track more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the drawings required for use in the embodiments of the present application or related technical descriptions will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a flow chart of a track management method in one embodiment;

[0038] Figure 2 The figure is a schematic diagram of a process of performing track fusion on a camera target and a radar target in one embodiment;

[0039] Figure 3 A schematic diagram of a process for updating a multi-sensor fusion track in one embodiment;

[0040] Figure 4A schematic diagram of a flow chart for determining a fusion threshold interval in one embodiment;

[0041] Figure 5 A schematic diagram of a process for screening camera targets in one embodiment;

[0042] Figure 6 is a structural block diagram of a track fusion device in an embodiment;

[0043] Figure 7 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In one embodiment, Figure 1 As shown, a track management method is provided. This embodiment uses the method applied to a terminal as an example for illustration. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0046] Step 102, obtaining the camera data and radar data at the current moment at a preset time interval; based on the camera data and the radar data, updating the multi-sensor fusion track at the previous moment to obtain an updated track.

[0047] Optionally, the preset time may be 50 ms.

[0048] Exemplarily, every 50 ms, the camera data and radar data of the current moment are obtained through the camera sensor and the radar sensor; based on the camera data and the radar data, the camera target and radar target of the current moment are determined; based on the camera target and radar target of the current moment, the multi-sensor fusion track of the previous moment is updated to obtain an updated track.

[0049] Step 104, determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment.

[0050] The camera track at the previous moment is the track of the camera target that has not been track-fused with the radar target among the camera targets at the previous moment. Correspondingly, the radar track at the previous moment is the track of the radar target that has not been track-fused with the camera target among the radar targets at the previous moment.

[0051] Exemplarily, based on the camera data and the camera track at the previous moment, all camera targets whose tracks have not been fused with the radar targets are determined as camera targets for fusion; based on the radar data and the radar track at the previous moment, all radar targets whose tracks have not been fused with the camera targets are determined as radar targets for fusion.

[0052] Step 106, based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0053] Among them, the track fusion strategy is to use the camera target as a benchmark, find the radar target corresponding to the camera target from all radar targets and perform track fusion.

[0054] Exemplarily, based on the track fusion strategy, each camera target used for fusion is used as a benchmark, each camera target used for fusion and all radar targets used for fusion are compared and track fusion is performed, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0055] The above-mentioned track management method, device, computer equipment, computer-readable storage medium and computer program product obtain the camera data and radar data at the current moment at a preset time interval; based on the camera data and the radar data, update the multi-sensor fusion track at the previous moment to obtain an updated track; based on the camera data and the camera track at the previous moment, determine the camera target for fusion; based on the radar data and the radar track at the previous moment, determine the radar target for fusion; based on the track fusion strategy, fuse the camera target for fusion and the radar target for fusion, and combine the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment and the radar track at the current moment. Generating a multi-sensor fusion track based on a unique fusion strategy can make the fused multi-sensor fusion track more accurate.

[0056] In an exemplary embodiment, the method of performing track fusion on the camera target for fusion and the radar target for fusion based on the track fusion strategy includes:

[0057] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0058] Optional, up to 10 camera targets.

[0059] For example, Figure 2 As shown, starting from the first camera target for fusion, 10 camera targets for fusion and radar targets for fusion are fused in turn. The specific fusion process is: for each camera target for fusion, first traverse the radar target list, and compare the current camera target with all radar targets based on the fusion threshold interval; if there are multiple radar targets and the current camera target within the fusion threshold interval, then select the radar target with the smallest longitudinal distance difference with the current camera target from the multiple radar targets and fuse the track with the current camera target to obtain the multi-sensor fusion track corresponding to the camera, and delete the corresponding radar target from the radar target list. If there is only one radar target and the current camera target within the fusion threshold interval, then the radar target and the current camera target are fused to obtain the multi-sensor fusion track corresponding to the camera, and the corresponding radar target is deleted from the radar target list. If there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0060] In this embodiment, through the track fusion strategy and the fusion threshold interval, the camera target and the radar target can be accurately matched to obtain a multi-sensor fusion track.

[0061] In an exemplary embodiment, updating the multi-sensor fusion track at a previous moment based on the camera data and the radar data to obtain an updated track includes:

[0062] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0063] Optionally, the first preset value may be 1, the second preset value may be 5, the count is the cumulative number of times i, and the initial value is 0, that is, when counting for the first time, the cumulative number of times i is 0 plus 1, which is equal to 1.

[0064] Among them, first find the multi-sensor fusion track, check whether the camera target and radar target in the track exist, if neither exists, delete the track; if only one side exists, the fusion track automatically follows the existing target as a single sensor fusion track; if both sides exist, check whether the difference between the camera target and the radar target in the track is within the disconnection threshold interval, if it is within the disconnection threshold interval, the track is maintained; if one or more items are not within the disconnection threshold interval, it is accumulated once, if one or more items are not within the disconnection threshold interval for 5 consecutive times, two single sensor fusion tracks are generated; if the difference between the two targets is within the disconnection threshold interval before 5 consecutive accumulations, the accumulated times are reset to zero, and the multi-sensor fusion track state continues to be maintained.

[0065] For example, Figure 3As shown, all camera targets and all radar targets at the current moment are obtained based on camera data and radar data; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, it is determined whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar target exists but the camera target does not exist, the multi-sensor fusion track at the previous moment corresponding to the radar target is converted into the radar track at the previous moment; if only the camera target exists but the radar target does not exist If it exists, the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the cumulative number i of the multi-sensor fusion track at the previous moment is obtained plus 1, and the cumulative number i at the current moment is obtained. If the cumulative number i at the current moment is less than 5, the multi-sensor fusion track is maintained. If the cumulative number i at the current moment is not less than 5, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0066] In this embodiment, by disconnecting the threshold interval to determine whether the camera target and the radar target are the same target, and updating the multi-sensor fusion track, a more accurate multi-sensor fusion track can be obtained.

[0067] In an exemplary embodiment, the fusion threshold interval is generated as follows:

[0068] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0069] The collected data include: obstacle ID (value range: 1~255), obstacle longitudinal position (value range: 0 m ~250 m), obstacle lateral position (value range: -10 m ~10 m), obstacle longitudinal speed (value range: 50 ~50 ), obstacle lateral speed (value range: 50 ~50 ), obstacle longitudinal acceleration (value range: -15 ~15 ) m / s2-15~15 lateral acceleration of obstacle (value range: -15 ~15 ), obstacle existence probability (value range: 1%~100%), obstacle type, including: 1: small vehicles (sedans, SUVs), 2: large vehicles (buses, trucks, tractors, 3: pedestrians.

[0070] Optionally, the preset sampling points may be 5m, 30m, 60m, 90m, 120m and 150m respectively.

[0071] Among them, the generation process of disconnection threshold interval and fusion threshold interval is similar. The radar data is collected based on the longitudinal distance, with 5m, 30m, 60m, 90m, 120m and 150m as data sampling points, and the data near the sampling points (±1m) are screened out; the corresponding camera data is screened out according to the timestamp of the radar data that has been screened out; the position information and speed information of the camera and radar at the same timestamp are subtracted to obtain the difference data between the two sensors at different sampling points; the difference data of the same sampling point are placed in the matlab tool for normal distribution analysis, and (μ-2σ, μ+2σ) is taken as the corresponding sampling point fusion threshold interval, and (μ-3σ, μ+3σ) is taken as the corresponding sampling point disconnection threshold interval; the threshold intervals between adjacent sampling points are linearly related, so the threshold interval of the non-sampling point is determined by the position of the non-sampling point between the two sampling points and the threshold interval of the two sampling points.

[0072] Optionally, taking the radar longitudinal distance of 5 meters as an example, the fusion threshold intervals include: longitudinal distance fusion threshold interval (-2 meters, 2 meters), lateral distance fusion threshold interval (-1 meter, 1 meter), longitudinal speed fusion threshold interval (-1 meter / second, 1 meter / second) and lateral speed fusion threshold interval (-0.5 meter / second, 0.5 meter / second); the disconnection threshold intervals include: longitudinal distance disconnection threshold interval (-3 meters, 3 meters), lateral distance disconnection threshold interval (-1.5 meters, 1.5 meters), longitudinal speed disconnection threshold interval (-1.5 meters / second, 1.5 meters / second) and lateral speed disconnection threshold interval (-0.8 meters / second, 0.8 meters / second).

[0073] For example, Figure 4 As shown, based on the camera sensor and the radar sensor, the camera data and radar data of 5m, 30m, 60m, 90m, 120m and 150m are obtained respectively; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at 5m, 30m, 60m, 90m, 120m and 150m are determined; for each of the difference data, a normal distribution analysis is performed to obtain the fusion threshold interval and the disconnection threshold interval.

[0074] In this embodiment, by obtaining the fusion threshold interval and the disconnection threshold interval, it is possible to accurately determine whether the camera target and the radar target are the same target.

[0075] In an exemplary embodiment, the combined updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment includes:

[0076] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been track-fused with the radar target is used as the camera track at the current moment; and the radar target that has not been track-fused with any of the camera targets used for fusion is determined among the radar targets used for fusion, and the track of the radar target is used as the radar track at the current moment.

[0077] Exemplarily, the multi-sensor fusion track and the updated track corresponding to each camera target are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been track fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been track fused with the camera target is used as the radar track at the current moment; the corresponding radar track of each radar target used for fusion that has not been track fused with the camera target is used as the radar track at the current moment.

[0078] In this embodiment, by producing multi-sensor fusion tracks, camera tracks and radar tracks, more comprehensive track information can be obtained, which is conducive to the automatic driving to execute corresponding decisions.

[0079] In an exemplary embodiment, determining a camera target for fusion based on the camera data and the camera track at a previous moment includes:

[0080] Acquire a camera target that appears newly at the current moment from the camera data, acquire a camera target of the camera track at the previous moment, and use the camera target that appears newly at the current moment and the camera target of the camera track at the previous moment as camera targets for fusion;

[0081] The determining of the radar target for fusion based on the radar data and the radar track at the previous moment includes: acquiring a radar target newly appearing at the current moment from the radar data, acquiring a radar target of the radar track at the previous moment, and using the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment as the radar targets for fusion.

[0082] Among them, when the camera identifies a large vehicle, it will identify a large vehicle as two targets with similar distances. At this time, there are redundant targets in the camera data. Therefore, the following screening strategy is formulated: when there are two target points, the longitudinal distance difference does not exceed 5m, the lateral distance difference does not exceed 1m, and the obstacle types of the two target points are both large vehicles, according to the principle that the closer the longitudinal distance, the greater the threat, the target points with closer longitudinal distance are retained, and the target points with farther longitudinal distance are deleted.

[0083] For example, Figure 5 As shown, the newly appeared camera target at the current moment is obtained from the camera data, and the camera target of the camera track at the previous moment is obtained. After obtaining these camera targets, the camera targets will be screened, including: first starting from the i=1th camera target, determine whether the first camera target is a large vehicle. If not, then determine the i+1th camera target, that is, the second camera target; if the 2nd-10th camera targets are not large vehicles, then determine these 10 camera targets. If the i-th camera target is a large vehicle, then determine whether the j=i+1th camera target, that is, the i+1th target, is also a large vehicle. If not, then determine whether the j=i+2th camera target is a large vehicle. If there is still no large vehicle until j is greater than 10, then determine that the target is a large vehicle. If the i-th camera target is a large vehicle, and the j-th camera target is also a large vehicle, then determine the longitudinal distance difference and lateral distance difference between the i-th camera target and the j-th camera target. If the longitudinal distance difference is less than 5 meters and the lateral distance difference is less than 1 meter, then determine that the i-th camera target and the j-th camera target are the same large vehicle. At this time, retain the camera target with a longer longitudinal distance from the camera sensor and delete the closer target. Use the screened camera targets as camera targets for fusion. Obtain the newly appeared radar target at the current moment from the radar data, obtain the radar target of the radar track at the previous moment, and use the newly appeared radar target at the current moment and the radar target of the radar track at the previous moment as radar targets for fusion.

[0084] In this embodiment, by grouping taxis with similar distances as one target, the problem of a taxi being mistakenly identified as two targets due to its large size can be solved.

[0085] In an exemplary embodiment, a track management method includes:

[0086] Get the current camera data and radar data every 50ms; get all camera targets and all radar targets at the current moment based on the camera data and radar data; get the radar targets and camera targets in the multi-sensor fusion track at the previous moment; based on all camera targets and all radar targets at the current moment, determine whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if neither the radar target nor the camera target exists, delete the multi-sensor fusion track corresponding to the radar target and the camera target; if only the radar target exists but the camera target does not exist, convert the multi-sensor fusion track corresponding to the radar target at the previous moment into the radar track at the previous moment; if only the camera target exists If the head target exists but the radar target does not, the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the cumulative number i of the multi-sensor fusion track at the previous moment is obtained plus 1 to obtain the cumulative number i at the current moment. If the cumulative number i at the current moment is less than 5, the multi-sensor fusion track is maintained; if the cumulative number i at the current moment is not less than 5, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively. The newly appeared camera target at the current moment is obtained from the camera data, and the camera target of the camera track at the previous moment is obtained. After obtaining these camera targets, the camera targets will be screened, including: first starting from the i=1th camera target, determine whether the first camera target is a large vehicle. If not, determine the i+1th camera target, that is, the second camera target; if the 2nd-10th camera targets are not large vehicles, then determine these 10 camera targets. If the i-th camera target is a large vehicle, determine whether the j=i+1th camera target, that is, the i+1th target is also a large vehicle. If not, determine whether the j=i+2th camera target is a large vehicle. If there is still no large vehicle until j is greater than 10, then determine that the target is a large vehicle. If the i-th camera target is a large vehicle and the j-th camera target is also a large vehicle, the longitudinal distance difference and lateral distance difference between the i-th camera target and the j-th camera target are determined. If the longitudinal distance difference is less than 5 meters and the lateral distance difference is less than 1 meter, the i-th camera target and the j-th camera target are determined to be the same large vehicle. At this time, the camera target with a longer longitudinal distance from the camera sensor is retained and the closer target is deleted. The screened camera targets are used as camera targets for fusion.The radar target newly appeared at the current moment is obtained from the radar data, and the radar target of the radar track at the previous moment is obtained, and the radar target newly appeared at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion. Starting from the first camera target for fusion, 10 camera targets for fusion and radar targets for fusion are fused in sequence. The specific fusion process is: for each camera target for fusion, the radar target list is first traversed, and the current camera target is compared with all radar targets based on the fusion threshold interval; if there are multiple radar targets and the current camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference with the current camera target is selected from the multiple radar targets and the current camera target is fused with the track to obtain the multi-sensor fusion track corresponding to the camera, and the corresponding radar target is deleted from the radar target list. If there is only one radar target and the current camera target within the fusion threshold interval, the radar target and the current camera target are fused with the track to obtain the multi-sensor fusion track corresponding to the camera, and the corresponding radar target is deleted from the radar target list. If there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track. The multi-sensor fusion track and the updated track corresponding to each camera target are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment. Among them, the fusion threshold interval and the disconnection threshold interval are generated in the following manner: based on the camera sensor and the radar sensor, the camera data and the radar data of 5m, 30m, 60m, 90m, 120m and 150m are respectively obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at 5m, 30m, 60m, 90m, 120m and 150m are determined; for each of the difference data, a normal distribution analysis is performed to obtain the fusion threshold interval and the disconnection threshold interval.

[0087] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0088] In an exemplary embodiment, Figure 6 As shown, a track management device is provided, including: an updating module 601, an acquiring module 602 and a fusion module 603, wherein:

[0089] An updating module, used for acquiring the camera data and the radar data at the current moment at a preset time interval; based on the camera data and the radar data, updating the multi-sensor fusion track at the previous moment to obtain an updated track;

[0090] An acquisition module, used to determine a camera target for fusion based on the camera data and the camera track at the previous moment; and to determine a radar target for fusion based on the radar data and the radar track at the previous moment;

[0091] The fusion module is used to fuse the camera target for fusion and the radar target for fusion based on the track fusion strategy, and combine the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0092] In one embodiment, the fusion module is further configured to:

[0093] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0094] In one embodiment, the update module is further used to:

[0095] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0096] In one embodiment, the present invention further comprises: a threshold interval generating module, which is used to:

[0097] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0098] In one embodiment, the fusion module is further configured to:

[0099] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

[0100] In one embodiment, the acquisition module is further used to:

[0101] A camera target newly appearing at the current moment is obtained from the camera data, a camera target of the camera track at the previous moment is obtained, and the camera target newly appearing at the current moment and the camera target of the camera track at the previous moment are used as camera targets for fusion; a radar target newly appearing at the current moment is obtained from the radar data, a radar target of the radar track at the previous moment is obtained, and the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion.

[0102] Each module in the above-mentioned track fusion device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0103] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store track data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a track management method is implemented.

[0104] Those skilled in the art will understand that Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0106] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0107] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0108] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0109] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0110] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0111] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0112] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0113] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0114] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0115] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0116] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

[0117] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0118] A camera target newly appearing at the current moment is obtained from the camera data, a camera target of the camera track at the previous moment is obtained, and the camera target newly appearing at the current moment and the camera target of the camera track at the previous moment are used as camera targets for fusion; a radar target newly appearing at the current moment is obtained from the radar data, a radar target of the radar track at the previous moment is obtained, and the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion.

[0119] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0120] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0121] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0122] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0123] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0124] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0125] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0126] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0127] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0128] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0129] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0130] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

[0131] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0132] A camera target newly appearing at the current moment is obtained from the camera data, a camera target of the camera track at the previous moment is obtained, and the camera target newly appearing at the current moment and the camera target of the camera track at the previous moment are used as camera targets for fusion; a radar target newly appearing at the current moment is obtained from the radar data, a radar target of the radar track at the previous moment is obtained, and the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion.

[0133] In one embodiment, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the following steps:

[0134] Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track;

[0135] Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment;

[0136] Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

[0137] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0138] For each camera target used for fusion, the current camera target is compared with all the radar targets based on the fusion threshold interval; if the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold interval, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is a radar target and the camera target within the fusion threshold interval, the radar target and the camera target are track fused to obtain the multi-sensor fusion track corresponding to the current camera target; if the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

[0139] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0140] Based on the camera data and the radar data, all camera targets and all radar targets at the current moment are obtained; radar targets and camera targets in the multi-sensor fusion track at the previous moment are obtained; based on all camera targets and all radar targets at the current moment, whether the radar targets and camera targets in the multi-sensor fusion track at the previous moment still exist at the current moment; if both the radar targets and the camera targets do not exist, the multi-sensor fusion track corresponding to the radar targets and the camera targets is deleted; if only the radar targets exist but the camera targets do not exist, the multi-sensor fusion track at the previous moment corresponding to the radar targets is converted into the radar track at the previous moment; if only the camera targets exist but the radar targets do not exist , the multi-sensor fusion track corresponding to the camera target at the previous moment is converted into the camera track at the previous moment; if both the radar target and the camera target exist, it is determined whether the radar target and the camera target are within the disconnection threshold interval; if so, the multi-sensor fusion track corresponding to the radar target and the camera target is maintained; if not, the count of the multi-sensor fusion track at the previous moment is obtained and added with a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, the multi-sensor fusion track is maintained, and if the count at the current moment is not less than the second preset value, the radar track at the previous moment and the camera track at the previous moment are generated based on the radar target and the camera target respectively.

[0141] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0142] Based on the camera sensor and the radar sensor, the camera data and the radar data of the preset sampling point are obtained; based on the camera data and the radar data, the difference data of the camera sensor and the radar sensor at the preset sampling point are determined; and a normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

[0143] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0144] The multi-sensor fusion track corresponding to each camera target and the updated track are combined into the multi-sensor fusion track at the current moment; the corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; the corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

[0145] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0146] A camera target newly appearing at the current moment is obtained from the camera data, a camera target of the camera track at the previous moment is obtained, and the camera target newly appearing at the current moment and the camera target of the camera track at the previous moment are used as camera targets for fusion; a radar target newly appearing at the current moment is obtained from the radar data, a radar target of the radar track at the previous moment is obtained, and the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion.

[0147] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0148] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0149] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be construed as limiting the scope of the present application. It should be noted that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A track management method, characterized in that: The method comprises: Obtaining the camera data and radar data at the current moment at a preset time interval; updating the multi-sensor fusion track at the previous moment based on the camera data and the radar data to obtain an updated track; Determining a camera target for fusion based on the camera data and the camera track at the previous moment; determining a radar target for fusion based on the radar data and the radar track at the previous moment; Based on the track fusion strategy, the camera target for fusion and the radar target for fusion are track-fused, and combined with the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

2. The method according to claim 1, characterized in that The method of performing track fusion on the camera target for fusion and the radar target for fusion based on the track fusion strategy includes: For each camera target used for fusion, the current camera target is compared with all radar targets based on the fusion threshold interval; If the comparison result shows that there are multiple radar targets and the camera target within the fusion threshold range, the radar target with the smallest longitudinal distance difference and the current camera target are selected for track fusion to obtain the multi-sensor fusion track corresponding to the current camera target; If the comparison result shows that there is a radar target and the camera target within the fusion threshold range, the radar target and the camera target are track-fused to obtain a multi-sensor fusion track corresponding to the current camera target; If the comparison result shows that there is no radar target and the current camera target within the fusion threshold interval, the track of the current camera target is used as the camera track.

3. The method according to claim 1, characterized in that The updating of the multi-sensor fusion track at a previous moment based on the camera data and the radar data to obtain an updated track includes: Acquire all camera targets and all radar targets at the current moment based on the camera data and the radar data; Acquire the radar target and the camera target in the multi-sensor fusion track at the previous moment; based on all the camera targets and all the radar targets at the current moment, determine whether the radar target and the camera target in the multi-sensor fusion track at the previous moment still exist at the current moment; If neither the radar target nor the camera target exists, the multi-sensor fusion track corresponding to the radar target and the camera target is deleted; If only the radar target exists but the camera target does not exist, converting the multi-sensor fusion track at the previous moment corresponding to the radar target into the radar track at the previous moment; If only the camera target exists but the radar target does not exist, converting the multi-sensor fusion track at the previous moment corresponding to the camera target into the camera track at the previous moment; If both the radar target and the camera target exist, determine whether the radar target and the camera target are within the disconnection threshold interval; if so, maintain the multi-sensor fusion tracks corresponding to the radar target and the camera target; if not, obtain the count of the multi-sensor fusion track at the previous moment and add a first preset value to obtain the count at the current moment, if the count at the current moment is less than the second preset value, maintain the multi-sensor fusion track, if the count at the current moment is not less than the second preset value, generate the radar track at the previous moment and the camera track at the previous moment based on the radar target and the camera target, respectively.

4. The method according to claim 2, characterized in that: The fusion threshold interval is generated as follows: Based on the camera sensor and the radar sensor, obtain the camera data and radar data of the preset sampling points; Based on the camera data and the radar data, determining difference data between the camera sensor and the radar sensor at a preset sampling point; A normal distribution analysis is performed on each of the difference data to obtain a fusion threshold interval.

5. The method according to claim 1, characterized in that The combined updated track is used to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment, including: Combining the multi-sensor fusion track corresponding to each camera target and the updated track into the multi-sensor fusion track at the current moment; The corresponding camera track of each camera target used for fusion that has not been fused with the radar target is used as the camera track at the current moment; The corresponding radar track of each radar target used for fusion that has not been fused with the camera target is used as the radar track at the current moment.

6. The method according to claim 1, characterized in that The step of determining a camera target for fusion based on the camera data and the camera track at a previous moment comprises: Acquire a camera target that appears newly at the current moment from the camera data, acquire a camera target of the camera track at the previous moment, and use the camera target that appears newly at the current moment and the camera target of the camera track at the previous moment as camera targets for fusion; The step of determining a radar target for fusion based on the radar data and the radar track at a previous moment includes: A radar target newly appearing at the current moment is obtained from the radar data, a radar target of the radar track at the previous moment is obtained, and the radar target newly appearing at the current moment and the radar target of the radar track at the previous moment are used as radar targets for fusion.

7. A track management device, characterized in that: The device comprises: An updating module, used for acquiring the camera data and the radar data at the current moment at a preset time interval; based on the camera data and the radar data, updating the multi-sensor fusion track at the previous moment to obtain an updated track; An acquisition module, used to determine a camera target for fusion based on the camera data and the camera track at the previous moment; and to determine a radar target for fusion based on the radar data and the radar track at the previous moment; The fusion module is used to fuse the camera target for fusion and the radar target for fusion based on the track fusion strategy, and combine the updated track to obtain the multi-sensor fusion track at the current moment, the camera track at the current moment, and the radar track at the current moment.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images