Distributed track management method, system and storage medium

Abstract

The present invention discloses a distributed track management method based on multi-sensor fusion, including predicting the historical track based on camera data to obtain the first extrapolated track; firstly associating multiple camera targets with the first extrapolated track in sequence; taking the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, and secondly associating multiple radar targets with the second extrapolated track in sequence; and thirdly associating the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar according to the status mark of each sensor, and if the third association is successful, a new merged track is created, the extrapolated track whose information source is the newly created merged track is deleted, and each sensor status mark is updated for the third time. The present invention greatly improves the confidence of the output track, enhances the accuracy of track perception and enhances the perception ability of vehicle automatic driving.

Description

Technical Field

[0001] The present invention relates to the automotive field, and in particular to a distributed track management method based on multi-sensor fusion, a distributed track management system based on multi-sensor fusion, and a computer-readable storage medium for executing steps in the distributed track management method based on multi-sensor fusion. Background Art

[0002] Autonomous driving has become a research hotspot in the automotive field, and environmental perception, as the basis for the realization of autonomous driving functions, is a very challenging research direction. The multi-sensor fusion system combines the position, speed, shape and other information of the front target in this cycle obtained in real time by the camera and radar with the front target information predicted by the vehicle kinematic equation according to a certain algorithm, and can calculate the most reliable front target information, that is, the track. The multi-sensor fusion system outputs the track to the planning and control module to realize the autonomous driving function.

[0003] At present, the management method of multi-sensor fusion for tracks is generally: if the camera target and the radar target are successfully associated with the track, the successfully associated track is updated; if the camera target and the radar target fail to be associated with the track, the track is newly created using the target information of the failed association. However, this management method will cause the following problems: the confidence of some information detected by the camera is lower than that of the radar, and similarly, the confidence of some information detected by the radar is lower than that of the camera; therefore, if a track is only updated by the camera or the radar information, the confidence of the track is difficult to be convincing. Summary of the invention

[0004] A series of simplified concepts are introduced in the Summary of the Invention, which are all simplifications of the prior art in the field, and will be further described in detail in the Detailed Description of the Invention. The Summary of the Invention of the present invention does not mean to attempt to define the key features and essential technical features of the claimed technical solution, nor does it mean to attempt to determine the scope of protection of the claimed technical solution.

[0005] The technical problem to be solved by the present invention is to provide a distributed track management method based on multi-sensor fusion that can overcome the defects of the prior art, improve the track confidence, and enhance the accuracy of track perception. A distributed track management system based on multi-sensor fusion that can overcome the defects of the prior art, improve the track confidence, and enhance the accuracy of track perception, and a computer-readable storage medium for executing the steps in the distributed track management method based on multi-sensor fusion.

[0006] In order to solve the above technical problems, the present invention provides a distributed track management method based on multi-sensor fusion, comprising the following steps:

[0007] S1, based on the camera data, the historical track is predicted to obtain the first extrapolated track;

[0008] S2, first associating the multiple camera targets with the first extrapolated track in sequence, and if the first association is successful, updating the first extrapolated track; if the first association fails, creating a new first extrapolated track for the target, and updating the status flags of each sensor for the first time;

[0009] S3, taking the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, and sequentially associating the multiple radar targets with the second extrapolated track for the second time, if the second association is successful, updating the second extrapolated track, if the second association fails, creating a new second extrapolated track for the target, and updating the status flags of each sensor for the second time;

[0010] S4, according to the status mark of each sensor, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are associated for the third time. If the third association is successful, a new merged track is created, the extrapolated track whose extrapolated track information source is the new merged track is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

[0011] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the historical track is predicted in the following manner;

[0012] x_T_pre=0.5*ax_T_old*dt 2 +vx_T_old*dt+x_T_old;

[0013] y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old;

[0014] vx_T_pre=ax_T_old*dt+vx_T_old;

[0015] vy_T_pre=ay_T_old*dt+vy_T_old;

[0016] ax_T_pre = ax_T_old;

[0017] ay_T_pre = ay_T_old;

[0018] x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track is predicted respectively.

[0019] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and in step S2, the first extrapolated track is updated by using Kalman filtering;

[0020] state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C);

[0021] K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise;

[0022] state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0023] Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0024] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0025] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and in step S2, a first extrapolated track is newly created in the following manner;

[0026] state_T_new1 = state_C;

[0027] Wherein, state_T_new1 is the state quantity of the newly created first extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0028] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0029] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the first update of each sensor status mark includes:

[0030] If the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark.

[0031] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and in step S3, the second extrapolated track is updated by using Kalman filtering;

[0032] state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R);

[0033] K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise;

[0034] state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0035] Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0036] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0037] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and in step S3, a second extrapolated track is newly created in the following manner;

[0038] state_T_new2 = state_R;

[0039] Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0040] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0041] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the second updating of each sensor status mark includes:

[0042] If the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept set to the second mark.

[0043] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the sensor state mark is a first mark, and it is determined that the source of the extrapolated track information is only the camera;

[0044] If the sensor status mark is the second mark, it is determined that the source of the extrapolated track information is only the radar.

[0045] Optionally, the distributed track management method based on multi-sensor fusion is further improved. If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is less than a specified threshold, it is determined that the two tracks are successfully associated.

[0046] Optionally, the distributed track management method based on multi-sensor fusion is further improved. If the extrapolated track information source is only the extrapolated track of the camera and the extrapolated track information source is only the extrapolated track of the radar, the distance between them is calculated using the following formula:

[0047] {(x_C_only-x_R_only) 2 +(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2} 0.5 ;

[0048] x_C_only, y_C_only and vx_C_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera;

[0049] x_R_only, y_R_only and vx_R_only are divided into the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track when the only source of the extrapolated track information is the radar.

[0050] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the longitudinal information of the merged track is selected from the extrapolated track whose only source of the extrapolated track information is the radar, and the lateral information of the merged track is selected from the extrapolated track whose only source of the extrapolated track information is the camera.

[0051] Optionally, the distributed track management method based on multi-sensor fusion is further improved, and the third updating of each sensor status mark includes setting the sensor status mark to a third mark if it is a merged track.

[0052] In order to solve the above technical problems, the present invention provides a computer-readable storage medium for executing the steps in any one of the above-mentioned distributed track management methods based on multi-sensor fusion.

[0053] In order to solve the above technical problems, the present invention provides a distributed track management system based on multi-sensor fusion, comprising:

[0054] A receiving module, which is used to receive track information from cameras and radars;

[0055] An extrapolation module predicts the historical track to obtain a first extrapolated track;

[0056] A first track management module, which sequentially associates multiple camera targets with the first extrapolated track for the first time, updates the first extrapolated track if the first association succeeds, and creates a new first extrapolated track for the target if the first association fails, and updates each sensor status mark for the first time;

[0057] a second track management module, which uses the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, associates the multiple radar targets with the second extrapolated track in sequence, updates the second extrapolated track if the second association succeeds, and creates a new second extrapolated track for the target if the second association fails, and updates the status flags of each sensor for the second time;

[0058] A track selection module, which selects the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar according to the status mark of each sensor;

[0059] The third track management module performs a third association on the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar. If the third association is successful, a new merged track is created, the extrapolated track of the new merged track information source is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

[0060] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the extrapolation module predicts the historical track in the following manner;

[0061] x_T_pre=0.5*ax_T_old*dt 2 +vx_T_old*dt+x_T_old;

[0062] y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old;

[0063] vx_T_pre=ax_T_old*dt+vx_T_old;

[0064] vy_T_pre=ay_T_old*dt+vy_T_old;

[0065] ax_T_pre = ax_T_old;

[0066] ay_T_pre = ay_T_old;

[0067] x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track is predicted respectively.

[0068] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the first extrapolated track is updated using Kalman filtering;

[0069] state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C);

[0070] K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise;

[0071] state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0072] Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0073] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0074] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and a first extrapolated track is newly created in the following manner;

[0075] state_T_new1 = state_C;

[0076] Wherein, state_T_new1 is the state quantity of the newly created first extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0077] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0078] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the first update of each sensor status mark includes:

[0079] If the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark.

[0080] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the second extrapolated track is updated by using Kalman filtering;

[0081] state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R);

[0082] K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise;

[0083] state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0084] Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0085] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0086] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and a second extrapolated track is newly created in the following manner;

[0087] state_T_new2 = state_R;

[0088] Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0089] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

[0090] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the second update of each sensor status mark includes:

[0091] If the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept set to the second mark.

[0092] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the sensor state mark is a first mark, then it is determined that the source of the extrapolated track information is only the camera;

[0093] If the sensor status mark is the second mark, it is determined that the source of the extrapolated track information is only the radar.

[0094] Optionally, the distributed track management system based on multi-sensor fusion is further improved. If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is less than a specified threshold, it is determined that the two tracks are successfully associated.

[0095] Optionally, the distributed track management system based on multi-sensor fusion is further improved. If the extrapolated track information source is only the camera's extrapolated track and the extrapolated track information source is only the radar's extrapolated track, the distance between them is calculated using the following formula:

[0096] {(x_C_only-x_R_only) 2 +(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2} 0.5 ;

[0097] x_C_only, y_C_only and vx_C_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera;

[0098] x_R_only, y_R_only and vx_R_only are divided into the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track when the only source of the extrapolated track information is the radar.

[0099] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the longitudinal information of the merged track is selected from the extrapolated track whose only source of the extrapolated track information is the radar, and the lateral information of the merged track is selected from the extrapolated track whose only source of the extrapolated track information is the camera.

[0100] Optionally, the distributed track management system based on multi-sensor fusion is further improved, and the third updating of each sensor status mark includes setting the sensor status mark to a third mark if it is a merged track.

[0101] The present invention creates and updates a track according to target information detected by a camera or a radar. Compared with a traditional track management method, the present invention adds sensor marking information to the track, and divides the track into a general track, an extrapolated track whose extrapolated track information source is only the radar, and an extrapolated track whose extrapolated track information source is only the camera. In view of the low confidence of the extrapolated track whose extrapolated track information source is only the radar and the extrapolated track whose extrapolated track information source is only the camera, the two are associated and updated, which greatly improves the confidence of the output track, enhances the accuracy of track perception, and enhances the perception ability of vehicle automatic driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] The drawings of the present invention are intended to illustrate the general characteristics of the methods, structures and / or materials used in specific exemplary embodiments of the present invention, and to supplement the description in the specification. However, the drawings of the present invention are schematic diagrams not drawn to scale, and thus may not accurately reflect the precise structure or performance characteristics of any given embodiment, and the drawings of the present invention should not be interpreted as limiting or restricting the range of values ​​or properties covered by the exemplary embodiments of the present invention. The present invention is further described in detail below in conjunction with the drawings and specific embodiments:

[0103] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0104] The following is an explanation of the embodiments of the present invention by specific specific embodiments, and those skilled in the art can fully understand other advantages and technical effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through different specific embodiments, and the details in this specification can also be applied based on different viewpoints, and various modifications or changes can be made without deviating from the general design concept of the invention. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. The following exemplary embodiments of the present invention can be implemented in a variety of different forms and should not be interpreted as being limited to the specific embodiments described here. It should be understood that these embodiments are provided to make the disclosure of the present invention thorough and complete, and to fully convey the technical solutions of these exemplary specific embodiments to those skilled in the art. It should be understood that when an element is referred to as "connected" or "coupled" to another element, the element can be directly connected or coupled to another element, or there can be an intermediate element. The difference is that when an element is referred to as "directly connected" or "directly coupled" to another element, there is no intermediate element. In all the drawings, the same reference numerals always represent the same elements.

[0105] First embodiment;

[0106] The present invention provides a distributed track management method based on multi-sensor fusion, comprising the following steps:

[0107] S1, based on the camera data, the historical track is predicted to obtain the first extrapolated track;

[0108] S2, first associating the multiple camera targets with the first extrapolated track in sequence, and if the first association is successful, updating the first extrapolated track; if the first association fails, creating a new first extrapolated track for the target, and updating the status flags of each sensor for the first time;

[0109] S3, taking the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, and sequentially associating the multiple radar targets with the second extrapolated track for the second time, if the second association is successful, updating the second extrapolated track, if the second association fails, creating a new second extrapolated track for the target, and updating the status flags of each sensor for the second time;

[0110] S4, according to the status mark of each sensor, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are associated for the third time. If the third association is successful, a new merged track is created, the extrapolated track whose extrapolated track information source is the new merged track is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

[0111] In addition, it should be understood that, although the terms "first", "second", etc. may be used herein to describe different elements, parameters, components, regions, layers and / or parts, these elements, parameters, components, regions, layers and / or parts should not be limited by these terms. These terms are only used to distinguish one element, parameter, component, region, layer or part from another element, parameter, component, region, layer or part. Therefore, without departing from the teachings of exemplary embodiments of the present invention, the first element, parameter, component, region, layer or part discussed below may also be referred to as the second element, parameter, component, region, layer or part.

[0112] Second embodiment;

[0113] like Figure 1 As shown, the present invention provides a distributed track management method based on multi-sensor fusion, comprising the following steps:

[0114] S1, based on the camera data, predict the historical track to obtain the first extrapolated track; the historical track is predicted in the following manner;

[0115] The track array output by the multi-sensor fusion system in the previous cycle is called the historical track; in each operation cycle, before executing the creation and update of two tracks, the historical track will be predicted, and the predicted track is called the extrapolated track; in each operation cycle, predicting the historical track means: assuming that the historical track performs uniform acceleration motion within an operation cycle time, then the changes in the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity, and lateral acceleration of the track during the motion process are updated;

[0116] x_T_pre=0.5*ax_T_old*dt 2+vx_T_old*dt+x_T_old;

[0117] y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old;

[0118] vx_T_pre=ax_T_old*dt+vx_T_old;

[0119] vy_T_pre=ay_T_old*dt+vy_T_old;

[0120] ax_T_pre = ax_T_old;

[0121] ay_T_pre = ay_T_old;

[0122] x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track prediction respectively;

[0123] S2, firstly associating multiple camera targets with the first extrapolated track in sequence, and updating the first extrapolated track if the first association is successful, and creating a new first extrapolated track for the target if the first association fails, and updating the status flags of each sensor for the first time; the first extrapolated track is updated by using Kalman filtering;

[0124] state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C);

[0125] K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise;

[0126] state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0127] Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0128] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0129] S3, taking the newly created first extrapolated track, the updated first extrapolated track and the unupdated first extrapolated track as the second extrapolated track, sequentially associating the multiple radar targets with the second extrapolated track for the second time, if the second association is successful, updating the second extrapolated track, if the second association fails, creating a new second extrapolated track for the target, and updating the status flags of each sensor for the second time; the second extrapolated track is updated by using Kalman filtering;

[0130] state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R);

[0131] K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise;

[0132] state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0133] Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0134] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0135] Create a new second extrapolated track in the following way;

[0136] state_T_new2 = state_R;

[0137] Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0138] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0139] S4, according to the status marks of each sensor, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are thirdly associated, if the third association is successful, a new merged track is created, the extrapolated track whose new merged track information source is deleted, and the status marks of each sensor are updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged, the longitudinal information of the merged track selects the longitudinal information of the extrapolated track whose extrapolated track information source is only the radar, and the lateral information of the merged track selects the lateral information of the extrapolated track whose extrapolated track information source is only the camera;

[0140] Wherein, the first updating of each sensor status mark includes: if the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark is kept unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark;

[0141] The second updating of each sensor status mark includes: if the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark is kept unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept as the second mark;

[0142] If the sensor status mark is the first mark (e.g., 1), it is determined that the extrapolated track information comes from only the camera;

[0143] If the sensor status mark is the second mark (eg, 2), it is determined that the extrapolated track information comes from only the radar.

[0144] The third updating of each sensor status mark includes, if it is a merged track, setting the sensor status mark to a third mark (for example, 3);

[0145] If the distance between the extrapolated track whose information source is only the camera and the extrapolated track whose information source is only the radar is less than the specified threshold, it is determined that the two tracks are successfully associated;

[0146] If the extrapolated track information source is only the camera's extrapolated track and the extrapolated track information source is only the radar's extrapolated track, the distance between them is calculated using the following formula:

[0147] {(x_C_only-x_R_only) 2+(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2} 0.5 ;

[0148] x_C_only, y_C_only and vx_C_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera;

[0149] x_R_only, y_R_only and vx_R_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only radar;

[0150] In each operation cycle, the camera detection information and the radar detection information will be received at the same time, so the creation and update of the track will be performed twice, and the merging of the track will be performed once. Therefore, the track management in each operation cycle of the present invention can be divided into three steps. The first step is to create and update the track according to the information detected by the camera, the second step is to create and update the track according to the information detected by the radar, and the third step is to merge the updated tracks. These three steps of track management are serial in time.

[0151] Third embodiment;

[0152] The present invention provides a computer-readable storage medium for executing the steps in the distributed track management method based on multi-sensor fusion described in any one of the first embodiment or the second embodiment.

[0153] Fourth embodiment;

[0154] The present invention provides a distributed track management system based on multi-sensor fusion, comprising:

[0155] A receiving module, which is used to receive track information from cameras and radars;

[0156] An extrapolation module predicts the historical track to obtain a first extrapolated track;

[0157] A first track management module, which sequentially associates multiple camera targets with the first extrapolated track for the first time, updates the first extrapolated track if the first association succeeds, and creates a new first extrapolated track for the target if the first association fails, and updates each sensor status mark for the first time;

[0158] a second track management module, which uses the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, associates the multiple radar targets with the second extrapolated track in sequence, updates the second extrapolated track if the second association succeeds, and creates a new second extrapolated track for the target if the second association fails, and updates the status flags of each sensor for the second time;

[0159] A track selection module, which selects the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar according to the status mark of each sensor;

[0160] The third track management module performs a third association on the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar. If the third association is successful, a new merged track is created, the extrapolated track of the new merged track information source is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

[0161] Fifth embodiment;

[0162] The present invention provides a distributed track management system based on multi-sensor fusion, comprising:

[0163] An exemplary camera installation position is the upper middle position of the front windshield, and a radar installation position is the middle position of the front bumper. The information detected by the camera includes the longitudinal position, longitudinal speed, longitudinal acceleration, lateral position, lateral speed, and lateral acceleration of the camera target; the information detected by the radar includes the longitudinal position, longitudinal speed, longitudinal acceleration, lateral position, lateral speed, and lateral acceleration of the radar target; and the information contained in each track in the track array is: the longitudinal position, longitudinal speed, longitudinal acceleration, lateral position, lateral speed, lateral acceleration, and sensor tag of the track;

[0164] A receiving module, which is used to receive track information from cameras and radars;

[0165] An extrapolation module predicts the historical track to obtain a first extrapolated track; the extrapolation module predicts the historical track in the following manner;

[0166] x_T_pre=0.5*ax_T_old*dt 2 +vx_T_old*dt+x_T_old;

[0167] y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old;

[0168] vx_T_pre=ax_T_old*dt+vx_T_old;

[0169] vy_T_pre=ay_T_old*dt+vy_T_old;

[0170] ax_T_pre = ax_T_old;

[0171] ay_T_pre = ay_T_old;

[0172] x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track prediction respectively;

[0173] A first track management module, which sequentially associates multiple camera targets with the first extrapolated track for the first time, updates the first extrapolated track if the first association succeeds, and creates a new first extrapolated track for the target if the first association fails, and updates each sensor status mark for the first time;

[0174] For example, in a certain operation cycle, n_ObjList_Cam target information detected by the camera is received, and this information is associated with n_Track_predict1 extrapolated tracks, and n_Match1 pairs of associations are successful. Each target detected by the camera can only be successfully associated with one extrapolated track at most, and each extrapolated track can only be successfully associated with a target detected by one camera at most, so n_Match1 is less than or equal to n_ObjList_Cam, and n_Match1 is also less than or equal to n_Track_predict1. At the same time, there are n_NoMatch1 extrapolated tracks that have not been successfully associated.

[0175] n_Track_predict1=n_Match1+n_NoMatch1;

[0176] Receive n_ObjList_Cam target information detected by the camera, associate this information with n_Track_predict1 extrapolated tracks, and successfully associate n_Match1 pairs. After completing the update of n_Match1 tracks, you need to create n_Setup1 tracks, where: n_Setup1 = n_ObjList_Cam-n_Match1;

[0177] If a camera target is not successfully associated with an extrapolated track, a new track will be created for this target:

[0178] state_T_new1 = state_C;

[0179] Wherein, state_T_new1 is the state quantity of the newly created first extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0180] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0181] For example, after the track of the camera information is newly created and updated, the sensor flag Sensor_Flag of each track needs to be updated. The sensor flag update method of the first step of track management is as follows: if the track is a newly created track, set the Sensor_Flag to the first flag; if the track is not updated, keep the Sensor_Flag unchanged; if the track is updated, if the Sensor_Flag before the update is 1, then the Sensor_Flag is still kept as 1, and if the Sensor_Flag before the update is 2, then set the Sensor_Flag to 3;

[0182] The first extrapolated track is updated using Kalman filtering;

[0183] state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C);

[0184] K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise;

[0185] state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0186] Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0187] Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0188] The second track management module uses the newly created first extrapolated track, the updated first extrapolated track and the unupdated first extrapolated track as the second extrapolated track, and sequentially associates multiple radar targets with the second extrapolated track. If the second association is successful, the second extrapolated track is updated. If the second association fails, a new second extrapolated track is created for the target, and the status flags of each sensor are updated for the second time. The number of second extrapolated tracks n_Track_predict2 is: n_Track_predict2=n_NoMatch1+n_Setup1;

[0189] The n_ObjList_Radar target information detected by the radar is associated with the n_Track_predict2 extrapolated tracks, and n_Match2 pairs are successfully associated. Each target detected by the radar can only be successfully associated with one extrapolated track at most, and each extrapolated track can only be successfully associated with one target detected by the radar at most, so n_Match2 is less than or equal to n_ObjList_Radar, and n_Match2 is also less than or equal to n_Track_predict2. At the same time, there are n_NoMatch2 extrapolated tracks that have not been successfully associated: n_Track_predict2 = n_NoMatch2 + n_NoMatch2;

[0190] Receive n_ObjList_Radar target information detected by the camera, associate this information with n_Track_predict2 extrapolated tracks, and successfully associate n_Match2 pairs. After completing the update of n_Match2 tracks, you need to create n_Setup2 tracks, where: n_Setup2 = n_ObjList_Radar-n_Match2;

[0191] New creation means: if a radar target is not successfully associated with the extrapolated track, a new track will be created for this target: state_T_new2 = state_R;

[0192] Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0193] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0194] The second extrapolated track is updated using Kalman filtering;

[0195] state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R);

[0196] K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise;

[0197] state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0198] Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0199] Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration;

[0200] After the radar information track is created, updated, and the sensor mark is updated, the number of tracks n_Track_Aft is: n_Track_Aft = n_Match2 + n_NoMatch2 + n_Setup2;

[0201] A track selection module, which selects the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar according to the status mark of each sensor;

[0202] The third track management module performs a third association on the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar. If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is less than a specified threshold, it is determined that the two tracks are successfully associated; if the third association is successful, a new merged track is created, the longitudinal information of the merged track selects the longitudinal information of the extrapolated track whose extrapolated track information source is only the radar, and the lateral information of the merged track selects the lateral information of the extrapolated track whose extrapolated track information source is only the camera, the extrapolated track of the newly created merged track information source is deleted, and the status marks of each sensor are updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged; the third updating of the status marks of each sensor includes, if it is a merged track, setting the sensor status mark to the third mark;

[0203] If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is calculated using the following formula, that is. If the distance between a Cam_Only track and a Radar_Only track is less than the threshold Ass_Thre2, the association is successful. The distance between the Cam_Only track and the Radar_Only track is:

[0204] {(x_C_only-x_R_only) 2 +(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2} 0.5 ;

[0205] x_C_only, y_C_only and vx_C_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera;

[0206] x_R_only, y_R_only and vx_R_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only radar;

[0207] The successfully associated Cam_Only track and Radar_Only track are deleted. Therefore, the number of tracks n_Track in the final output track array is: n_Track = n_Track_Aft + n_Setup3-2*n_Match3; in a certain operation cycle, the number of extrapolated tracks in the first step is equal to the number of tracks in the final output track array of the previous cycle;

[0208] When the present invention is executed for the first time, since there is no track in the track array output in the previous cycle, the number of extrapolated tracks in the first step is 0, that is, the n_ObjList_Cam targets detected by the camera all fail to be associated, so n_ObjList_Cam tracks are newly created based on these targets, and these tracks will be used as the extrapolated tracks in the second step;

[0209] If the sensor status mark is the first mark, it is determined that the extrapolated track information comes from only the camera;

[0210] If the sensor status mark is the second mark, it is determined that the extrapolated track information comes from only radar;

[0211] Exemplarily, the longitudinal information of the merged track selects the longitudinal information of the extrapolated track whose source of information is only the extrapolated track of the radar;

[0212] x_T_new3 = x_R_only;

[0213] vx_T_new3 = vx_R_only;

[0214] ax_T_new3 = ax_R_only;

[0215] The lateral information of the merged track is selected from the lateral information of the extrapolated track information of the camera only;

[0216] y_T_new3 = y_C_only;

[0217] vy_T_new3 = vy_C_only;

[0218] ay_T_new3 = ay_C_only;

[0219] x_T_new3, vx_T_new3, ax_T_new3 are the longitudinal position, longitudinal velocity and longitudinal acceleration respectively;

[0220] y_T_new3, vy_T_new3, ay_T_new3 are the lateral position, lateral velocity and lateral acceleration respectively;

[0221] x_R_only, vx_R_only, ax_R_only are respectively the longitudinal distance, longitudinal velocity and longitudinal acceleration of the extrapolated track whose extrapolated track information source is only radar;

[0222] y_C_only, vy_C_only, ay_C_only are the longitudinal distance, longitudinal velocity and longitudinal acceleration of the extrapolated track whose extrapolated track information source is only the camera;

[0223] The first update of each sensor status flag includes:

[0224] If the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark.

[0225] The second updating of each sensor status mark includes: if the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark is kept unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept set to the second mark.

[0226] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. It will also be understood that, unless expressly defined herein, terms such as those defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and not interpreted in an ideal or overly formal sense.

[0227] The present invention has been described in detail above through specific implementation modes and embodiments, but these do not constitute limitations of the present invention. Without departing from the principles of the present invention, those skilled in the art may also make many variations and improvements, which should also be regarded as the protection scope of the present invention.

Claims

1. A distributed track management method based on multi-sensor fusion, characterized in that: The following steps are involved: S1, based on the camera data, the historical track is predicted to obtain the first extrapolated track; S2, first associating multiple camera targets with the first extrapolated track in sequence, if the first association is successful, updating the first extrapolated track, if the first association fails, creating a new first extrapolated track for the target, and updating each sensor status mark for the first time, including: If the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark; S3, taking the newly created first extrapolated track, the updated first extrapolated track and the first extrapolated track that has not been updated as the second extrapolated track, sequentially associating the multiple radar targets with the second extrapolated track for the second time, if the second association is successful, updating the second extrapolated track, if the second association fails, creating a new second extrapolated track for the target, and updating the status flags of each sensor for the second time, including: If the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark is kept unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is set to the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept set to the second mark; S4, according to the status mark of each sensor, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are associated for the third time. If the third association is successful, a new merged track is created, the extrapolated track whose extrapolated track information source is the new merged track is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

2. The distributed track management method based on multi-sensor fusion as claimed in claim 1, characterized in that: The historical track is predicted in the following way; x_T_pre=0.5*ax_T_old*dt 2 +vx_T_old*dt+x_T_old; y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old; vx_T_pre=ax_T_old*dt+vx_T_old; vy_T_pre=ay_T_old*dt+vy_T_old; ax_T_pre = ax_T_old; ay_T_pre=ay_T_old; x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track is predicted respectively.

3. The distributed track management method based on multi-sensor fusion as claimed in claim 1, characterized in that: In step S2, the first extrapolated track is updated using Kalman filtering; state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C); K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise; state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

4. The distributed track management method based on multi-sensor fusion as claimed in claim 3, characterized in that: In step S2, a first extrapolated track is newly created in the following manner: state_T_new1=state_C; Wherein, state_T_new1 is the state quantity of the newly created first extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

5. The distributed track management method based on multi-sensor fusion as claimed in claim 1, characterized in that: In step S3, the second extrapolated track is updated using Kalman filtering; state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R); K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise; state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

6. The distributed track management method based on multi-sensor fusion as claimed in claim 5, characterized in that: In step S3, a second extrapolated track is newly created in the following manner; state_T_new2=state_R; Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

7. The distributed track management method based on multi-sensor fusion as claimed in claim 6, characterized in that: If the sensor status mark is the first mark, it is determined that the extrapolated track information comes from only the camera; If the sensor status mark is the second mark, it is determined that the source of the extrapolated track information is only the radar.

8. The distributed track management method based on multi-sensor fusion as claimed in claim 7, characterized in that: If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is less than the specified threshold, it is determined that the two tracks are successfully associated.

9. The distributed track management method based on multi-sensor fusion as claimed in claim 8, characterized in that: If the extrapolated track information source is only the camera's extrapolated track and the extrapolated track information source is only the radar's extrapolated track, the distance between them is calculated using the following formula: {(x_C_only-x_R_only) 2 +(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2 } 0.5 ; x_C_only, y_C_only and vx_C_only are respectively the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera; x_R_only, y_R_only and vx_R_only are divided into the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track when the only source of the extrapolated track information is the radar.

10. The distributed track management method based on multi-sensor fusion as claimed in claim 9, characterized in that: The longitudinal information of the merged track selects the extrapolated track information source of which only the longitudinal information of the extrapolated track of the radar is used, and the lateral information of the merged track selects the extrapolated track information source of which only the lateral information of the extrapolated track of the camera is used.

11. The distributed track management method based on multi-sensor fusion as claimed in claim 10, characterized in that: The third updating of each sensor status mark includes setting the sensor status mark to the third mark if it is a merged track.

12. A computer-readable storage medium for executing the steps in the distributed track management method based on multi-sensor fusion according to any one of claims 1 to 11.

13. A distributed track management system based on multi-sensor fusion, characterized in that: include: A receiving module, which is used to receive track information from cameras and radars; An extrapolation module predicts the historical track to obtain a first extrapolated track; The first track management module sequentially associates multiple camera targets with the first extrapolated track, updates the first extrapolated track if the first association succeeds, and creates a new first extrapolated track for the target if the first association fails, and updates the status flags of each sensor for the first time, including: If the track is a newly created extrapolated track, the sensor status mark is set to the first mark; if the track is not updated, the sensor status mark remains unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is still kept as the first mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is set to the third mark; The second track management module uses the newly created first extrapolated track, the updated first extrapolated track and the unupdated first extrapolated track as the second extrapolated track, associates multiple radar targets with the second extrapolated track in sequence, updates the second extrapolated track if the second association succeeds, and creates a new second extrapolated track for the target if the second association fails, and updates the status flags of each sensor for the second time, including: If the track is a newly created second extrapolated track, the sensor status mark is set to the second mark; if the track is not updated, the sensor status mark is kept unchanged; if the track is updated and the sensor status mark before the update is the first mark, the sensor status mark is set to the third mark; if the track is updated and the sensor status mark before the update is the second mark, the sensor status mark is still kept set to the second mark; A track selection module, which selects the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar according to the status mark of each sensor; The third track management module performs a third association on the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar. If the third association is successful, a new merged track is created, the extrapolated track of the new merged track information source is deleted, and the status mark of each sensor is updated for the third time; if the third association fails, the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar are maintained unchanged.

14. The distributed track management system based on multi-sensor fusion as claimed in claim 13, characterized in that: The extrapolation module predicts the historical track in the following way; x_T_pre=0.5*ax_T_old*dt 2 +vx_T_old*dt+x_T_old; y_T_pre=0.5*ay_T_old*dt 2 +vy_T_old*dt+y_T_old; vx_T_pre=ax_T_old*dt+vx_T_old; vy_T_pre=ay_T_old*dt+vy_T_old; ax_T_pre = ax_T_old; ay_T_pre=ay_T_old; x_T_old, y_T_old, vx_T_old, vy_T_old, ax_T_old, and ay_T_old are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration of the historical track respectively; x_T_pre, y_T_pre, vx_T_pre, vy_T_pre, ax_T_pre, and ay_T_pre are the longitudinal position, lateral position, longitudinal velocity, lateral velocity, longitudinal acceleration, and lateral acceleration after the historical track is predicted respectively.

15. The distributed track management system based on multi-sensor fusion as claimed in claim 13, characterized in that: The first extrapolated track is updated using Kalman filtering; state_T_upd1=state_T_pre1+K1(state_T_pre1-state_C); K1 is the Kalman gain calculated based on the system noise and the camera’s measurement noise; state_T_upd1 is the state quantity after the first extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_T_pre1 is the state quantity of the first extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

16. The distributed track management system based on multi-sensor fusion as claimed in claim 15, characterized in that: Create a new first extrapolated track in the following way; state_T_new1=state_C; Wherein, state_T_new1 is the state quantity of the newly created first extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_C is the state quantity detected by the camera, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

17. The distributed track management system based on multi-sensor fusion as claimed in claim 13, characterized in that: The second extrapolated track is updated using Kalman filtering; state_T_upd2=state_T_pre2+K2(state_T_pre2-state_R); K2 is the Kalman gain calculated based on the system noise and the radar’s measurement noise; state_T_upd2 is the state quantity after the second extrapolated track is updated, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_T_pre2 is the state quantity of the second extrapolated track, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

18. The distributed track management system based on multi-sensor fusion as claimed in claim 17, characterized in that: Create a new second extrapolated track in the following way; state_T_new2=state_R; Wherein, state_T_new2 is the state quantity of the newly created second extrapolated track, which is longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration; Correspondingly, state_R is the state quantity detected by the radar, which is the longitudinal position, longitudinal velocity, longitudinal acceleration, lateral position, lateral velocity or lateral acceleration.

19. The distributed track management system based on multi-sensor fusion as claimed in claim 13, characterized in that: If the sensor status mark is the first mark, it is determined that the extrapolated track information comes from only the camera; If the sensor status mark is the second mark, it is determined that the source of the extrapolated track information is only the radar.

20. The distributed track management system based on multi-sensor fusion as claimed in claim 19, characterized in that: If the distance between the extrapolated track whose extrapolated track information source is only the camera and the extrapolated track whose extrapolated track information source is only the radar is less than the specified threshold, it is determined that the two tracks are successfully associated.

21. The distributed track management system based on multi-sensor fusion as claimed in claim 20, characterized in that: If the extrapolated track information source is only the camera's extrapolated track and the extrapolated track information source is only the radar's extrapolated track, the distance between them is calculated using the following formula: {(x_C_only-x_R_only) 2 +(y_C_only-y_R_only) 2 +(vx_C_only-vx_R_only) 2 } 0.5 ; x_C_only, y_C_only and vx_C_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track whose extrapolated track information source is only the camera; x_R_only, y_R_only and vx_R_only are the longitudinal distance, lateral distance and longitudinal speed of the extrapolated track when the only source of the extrapolated track information is the radar.

22. The distributed track management system based on multi-sensor fusion as claimed in claim 20, characterized in that: The longitudinal information of the merged track selects the extrapolated track information source of which only the longitudinal information of the extrapolated track of the radar is used, and the lateral information of the merged track selects the extrapolated track information source of which only the lateral information of the extrapolated track of the camera is used.

23. The distributed track management system based on multi-sensor fusion as claimed in claim 22, characterized in that: The third updating of each sensor status mark includes setting the sensor status mark to the third mark if it is a merged track.

Images