Object tracking method, device and vehicle

Abstract

The present invention discloses an object tracking method, device and vehicle. The method comprises: obtaining first state information of a target object to be tracked at a first moment; filtering the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filter parameters corresponding to the prediction error and detection error of the first state information; determining second state information of the target object at a second moment based on the state features of multiple dimensions, wherein the second moment is a moment after the first moment; tracking the target object based on the second state information to obtain a tracking result. The present invention solves the technical problem that objects cannot be effectively tracked in nonlinear scenes in target tracking.

Description

Technical Field

[0001] The present invention relates to the field of vehicles, and in particular to an object tracking method, device and vehicle. Background Art

[0002] In actual traffic environments, vehicles are in a complex environment. In order to meet the needs of automatic obstacle avoidance and path planning, it is necessary to screen out valid targets from a large amount of target data, establish corresponding track management, accurately obtain vehicle status information, and track multiple targets in real time.

[0003] Kalman filtering is widely used in radar target tracking systems because of its fast speed and easy implementation. However, its processing capability for nonlinear and non-Gaussian scenarios is relatively poor. In related technologies, for nonlinear scenarios in target tracking, the tracking deviation is usually large and good tracking effects cannot be obtained. There is a technical problem that the object cannot be effectively tracked.

[0004] Currently, no effective solution has been proposed to the above-mentioned problem that objects cannot be effectively tracked in nonlinear scenes in target tracking. Summary of the invention

[0005] The embodiments of the present invention provide a method, device and vehicle for tracking an object, so as to at least solve the technical problem that the object cannot be effectively tracked in a nonlinear scene in target tracking.

[0006] According to one aspect of an embodiment of the present invention, a method for tracking an object is provided. The method may include: obtaining first state information of a target object to be tracked at a first moment; filtering the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filter parameters corresponding to the prediction error and the detection error of the first state information; determining second state information of the target object at a second moment based on the state features of multiple dimensions, wherein the second moment is a moment after the first moment; and tracking the target object based on the second state information to obtain a tracking result.

[0007] Optionally, attribute data of the target object at a first moment is obtained; and a prediction error and a detection error are determined based on the attribute data.

[0008] Optionally, determining the prediction error based on the attribute data includes: filtering abnormal data in the attribute data; determining the prediction error based on a time series corresponding to the target object and the filtered attribute data, wherein the time series includes times when the target object appears multiple times.

[0009] Optionally, determining the detection error based on the attribute data includes: filtering abnormal data in the attribute data; and determining the detection error based on a difference between the filtered attribute data and corresponding target attribute data.

[0010] Optionally, determining the second state information of the target object at the second moment based on the state characteristics of multiple dimensions includes: establishing a target matrix corresponding to the state characteristics of multiple dimensions, wherein the target matrix is ​​used to represent the difference between the state characteristics of multiple dimensions and the characteristics of a target state sequence, and the target state sequence includes multiple target state information of the target object; matching the target matrix to obtain the second state information.

[0011] Optionally, the target matrix is ​​matched to obtain the second state information, including: determining that there is a target state sequence that matches the target matrix, performing Kalman filtering on the first state information to obtain the second state information; determining that there is no target state sequence that matches the target matrix, updating the first state information to the second state information based on a nonlinear model, or determining the second state information based on a newly created target state sequence.

[0012] Optionally, performing Kalman filtering on the first state information to obtain the second state information includes: performing Kalman filtering on the first state information based on a nonlinear model to obtain the second state information, wherein the detection error corresponding to the nonlinear model is corrected based on the distance between the target object and the vehicle.

[0013] Optionally, tracking the target object based on the second state information to obtain a tracking result includes: determining motion information of the target object and / or position information of the target object based on the second state information, wherein the tracking result includes motion information and / or position information.

[0014] According to another aspect of an embodiment of the present invention, a device for tracking an object is also provided. The device may include: an acquisition unit, used to acquire first state information of a target object to be tracked at a first moment; a filtering unit, used to filter the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to the prediction error and the detection error of the first state information; a determination unit, used to determine second state information of the target object at a second moment based on the state features of multiple dimensions, wherein the second moment is a moment after the first moment; and a tracking unit, used to track the target object based on the second state information to obtain a tracking result.

[0015] According to another aspect of an embodiment of the present invention, a vehicle is further provided, and the vehicle is used to execute the object tracking method of the embodiment of the present invention.

[0016] In an embodiment of the present invention, first state information of a target object to be tracked at a first moment is obtained; the first state information is filtered based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filter parameters corresponding to the prediction error and the detection error of the first state information; the second state information of the target object at a second moment is determined based on the state features of multiple dimensions, wherein the second moment is a moment after the first moment; the target object is tracked based on the second state information to obtain a tracking result. In other words, the present invention uses a nonlinear model to filter the state information of the target object, fully considering the state feature information of each dimension of the target object, so as to obtain more accurate real-time state information of the target object based on the fusion of detection information and prediction information, thereby solving the technical problem that the object cannot be effectively tracked in nonlinear scenes in target tracking. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0018] Figure 1 is a flow chart of a method for tracking an object according to an embodiment of the present invention;

[0019] Figure 2 is a schematic diagram of a method for calculating filter adaptive parameters according to an embodiment of the present invention;

[0020] Figure 3 is a schematic diagram of a prediction error calculation method according to an embodiment of the present invention;

[0021] Figure 4 is a schematic diagram of a detection error calculation method according to an embodiment of the present invention;

[0022] Figure 5 is a schematic diagram of a device for tracking an object according to an embodiment of the present invention. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] Example 1

[0026] According to an embodiment of the present invention, a method embodiment for tracking an object is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0027] Figure 1 is a flow chart of a method for tracking an object according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0028] Step S1, obtaining first state information of a target object to be tracked at a first moment.

[0029] In the technical solution provided in the above step S1 of the present invention, at the first moment, the first state information of the target object to be tracked is obtained. The first moment can be represented as moment t-1, the target object to be tracked can include obstacles, etc., the first state information can be state quantity information, and the state quantity information can include information such as the size, center point, orientation angle, velocity component, acceleration component and angular velocity of the target object, which is only used as an example and is not specifically limited.

[0030] Optionally, the size of the target object may include the length, width and height of the target object {l, w, h}, the center point of the target object may include the coordinates of the center point of the target object {x, y, z}, the velocity component of the target object may include the velocity components of the target object in different directions of the center point {vx, vy}, and the acceleration component of the target object may include the acceleration components of the target object in different directions of the center point {ax, ay}.

[0031] Optionally, obtaining the first state information of the target object to be tracked at the first moment may further include: obtaining attribute data of the target object at the first moment; and determining the prediction error and the detection error based on the attribute data. The attribute data may be true value data and / or detection data, the true value data may be the true value of the annotation data of each state quantity of the image frame, and may include the sequence number, size information, and posture information of the obstacle; the detection data may be the detection data of each state quantity of the image frame, and may include the sequence number, size information, and posture information of the obstacle.

[0032] Optionally, abnormal data in the acquired attribute data of the target object at the first moment can be filtered, and the prediction error can be determined based on the time series corresponding to the target object and the filtered attribute data, wherein the time series includes the time when the target object appears multiple times, which can be a trajectory of an obstacle corresponding to a time series.

[0033] Optionally, abnormal data in the acquired attribute data of the target object at the first moment can be filtered, and the detection error can be determined based on the difference between the filtered attribute data and the corresponding target attribute data, wherein the target attribute data can be a true value annotation file, and the difference between the filtered attribute data and the corresponding target attribute data can be the difference between the detection information and the true value information.

[0034] Step S2, filtering the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to the prediction error and the detection error of the first state information.

[0035] In the technical solution provided in the above step S2 of the present invention, the first state information is filtered based on a nonlinear model to obtain state characteristics of multiple dimensions. The nonlinear model may be a nonlinear motion model, and the state characteristics of multiple dimensions may be state characteristics of each dimension.

[0036] Optionally, performing filtering processing based on a nonlinear model may include using a nonlinear motion model to perform adaptive filtering state prediction and / or adaptive filtering state update.

[0037] Step S3: determining second state information of the target object at a second moment based on state features of multiple dimensions.

[0038] In the technical solution of step S3 of the present invention, determining the second state information of the target object at the second moment based on the state characteristics of multiple dimensions includes: establishing a target matrix corresponding to the state characteristics of multiple dimensions, matching the target matrix, and obtaining the second state information. The second moment may be moment t, and the second state information may be state quantity information, which may include information such as size, center point, orientation angle, velocity component, acceleration component, and angular velocity, which is only illustrated here by way of example and is not specifically limited.

[0039] Optionally, the target matrix is ​​used to represent the difference between state features of multiple dimensions and features of a target state sequence, and may be an association matrix. The target state sequence includes multiple target state information of the target object, which may be multiple target state quantity information stored in a tracking sequence or multiple target state quantity information in a detection sequence. The features of the target state sequence may be features of the prediction values ​​of the tracking sequence.

[0040] Optionally, matching the target matrix may be performing an error-based distance feature matching calculation on the target matrix using a matching algorithm, thereby obtaining the second state information.

[0041] Step S4: Track the target object based on the second state information to obtain a tracking result.

[0042] In the technical solution of step S4 of the present invention, the relevant information of the target object can be determined through the second state information to obtain a tracking result, wherein the tracking result includes motion information and / or position information.

[0043] Optionally, the state information of the obstacle at the current moment with a smaller variance is obtained through adaptive filtering, so as to perform the tracking cycle of the next frame. By repeating the above updating process, the speed and position information of the obstacle with less noise and accuracy can be obtained in real time, thereby providing information basis for the automatic obstacle avoidance and path planning of the subsequent vehicle.

[0044] The above method of this embodiment is further introduced below.

[0045] As an optional implementation manner, step S1, obtaining first state information of a target object to be tracked at a first moment, includes: obtaining attribute data of the target object at the first moment; and determining a prediction error and a detection error based on the attribute data.

[0046] In this embodiment, attribute data of the target object at the first moment is obtained, and a prediction error and a detection error are determined based on the attribute data, wherein the attribute data may be true value data and / or detection data, the true value data may be the true value of the labeled data of each state quantity of the image frame, and may include the serial number, size information, and posture information of the obstacle; the detection data may be the detection data of each state quantity of the image frame, and may include the serial number, size information, and posture information of the obstacle.

[0047] Optionally, the single target detection state information can be obtained according to the target detection result, {l m-1 , w m-1 ,h m-1} represents the size of the obstacle, {x m-1 ,y m-1 , z m-1} represents the center point of the obstacle, yaw m-1 represents the direction angle of the obstacle, where the subscript m-1 Represents the detection result at time t-1.

[0048] Optionally, a single target state information can be stored in the tracking sequence. t-1 , w t-1 ,h t-1} represents the size of the obstacle, {x t-1 ,y t-1 , z t-1} represents the center point of the obstacle, yaw t-1 Indicates the direction of the obstacle, {vx t-1 ,vy t-1} represents the velocity component of the obstacle, {ax t-1 ,ay t-1} represents the acceleration component of the obstacle, ω t-1 represents the angular velocity of the obstacle, where the subscript t-1 Represents the tracking result at time t-1.

[0049] Optionally, based on the nonlinear motion model, according to the first state information acquired at the first moment, the predicted value of each state quantity is calculated:

[0050] l t_ = l t-1 +q l

[0051] w t_ =w t-1 +q w

[0052] h t_ =h t-1 +q h

[0053]

[0054]

[0055] z t_ =z t-1 +q z

[0056] yaw t_ =yaw t-1 +ω t-1 ×Δt+q yaw

[0057] vx t_ =vx t-1 +ax t-1 ×Δt+q vx

[0058] vy t_ =vy t-1 +ay t-1 ×Δt+q vy

[0059] ax t_ =ax t-1 +q ax

[0060] ayt _ =ay t-1 +q ay

[0061] ω t_ =ω t-1 +q ω (1)

[0062] In the above formula (1), {l t_ , w t_ ,h t_} represents the predicted value of the obstacle size, {x t_ ,y t_ , z t_} represents the predicted value of the center point of the obstacle, yaw t_ Represents the predicted value of the obstacle's orientation angle, {vx t_ ,vy t_} represents the predicted value of the obstacle's speed, {ax t_ ,ay t_} represents the predicted value of the acceleration of the obstacle, ω t_ represents the angular velocity of the obstacle, subscript t_ represents the predicted value of each state quantity, and q represents the noise of each state.

[0063] Optionally, for formula (1), in order to prevent the denominator from being 0, a judgment is made. If cos(yaw t-1 ) or sin(yaw t-1 ) is less than a certain threshold, a small value is added to these two values ​​so that the denominator is not 0.

[0064] Optionally, calculating the prediction error includes: loading true value data, classifying different obstacles according to obstacle types, calculating respectively for different obstacles, filtering outliers in the true value through statistical data, and setting a region range for the true value, so that only data within the region range is valid, and performing calculations:

[0065] q l = l t_ -l t-1

[0066] q w =w t- -w t-1

[0067] q h =h t_ -h t-1

[0068]

[0069]

[0070] q z =z t_ -z t-1

[0071] q yaw =yaw t_ -yaw t-1 -ω t-1 ×Δt≈yaw t_ -yaw t-1 -(yaw t-1 -yaw t-2 )

[0072] q vx =vx t_ -vx t-1 -ax t-1 ×Δt≈vx t_ -vx t-1 -(vx t-1 -vx t-2 )

[0073] q vy =vy t_ -vy t-1 -ayt-1 ×Δt≈vy t_ -vy t-1 -(vy t-1 -vy t-2 )

[0074] q ax =ax t_ -ax t-1

[0075] q ay =ay t_ -ay t-1

[0076] q ω =ω t_ -ω t-1 (2)

[0077] In the above formula (2), q represents the difference between the previous and next frame differences of each state quantity.

[0078] Optionally, all the times of the same obstacle are connected in series by marking the serial number. The final data format is a trajectory of one obstacle corresponding to a time series. The serial number of the true value is checked to see whether the time of the obstacle is continuous to prevent the loss of the data annotation true value due to occlusion. The variance of the difference between the previous and next frames is calculated to obtain the prediction error of each state quantity:

[0079] Q l =Var(q l )

[0080] Q w =Var(q w )

[0081] Q h =Var(q h )

[0082] Q x =Var(q x )

[0083] Q y =Var(q y )

[0084] Q z =Var(q z )

[0085] Q yaw =Var(q yaw )

[0086] Q vx =Var(q vx )

[0087] Q vy =Var(q vy )

[0088] Q ax =Var(q ax )

[0089] Q ay =Var(q ay )

[0090] Q ω =Var(q ω ) (3)

[0091] In the above formula (3), Q represents the prediction error of each state quantity.

[0092] Optionally, calculating the detection error includes: loading the detection data, classifying different obstacles according to the obstacle type, calculating separately for different obstacles, filtering by counting abnormal values ​​in the detection data, and loading Gt true value data at the same time, comparing data with the same frame number, retaining data with the same timestamp as the detection data and the true value data, and matching the detection data and the true value data with the same timestamp of the same frame.

[0093] Optionally, matching the detection data and true value data of the same frame and the same timestamp includes: setting an area range for detection, only the data within the area range is valid, and performing calculations; calculating the Mahalanobis distance of each obstacle for the detection data and the true value data, forming a metric matrix, and finally performing Hungarian matching; obtaining a matching result, storing the matched true value data and detection data respectively; performing thresholding on the stored true value data and detection data, deleting detection true value data pairs whose Mahalanobis distance is greater than a certain threshold and matched, and calculating the difference between the detection information and the true value information of the same frame:

[0094]

[0095]

[0096]

[0097]

[0098] In the above formula (4), It represents the true value of the l-labeled data of the k-th frame, and the same is true for the other variables; It represents the l detection data of the kth frame, and the other variables are similar.

[0099] Optionally, calculate the variance of the difference between the detection information and the true value information:

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106]

[0107]

[0108]

[0109]

[0110] In the above formula (5), R l It represents the detection error of l, and the same is true for the other variables.

[0111] As an optional implementation method, step S2 filters the first state information based on a nonlinear model to obtain state characteristics of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to the prediction error and detection error of the first state information.

[0112] Optionally, performing filtering processing based on a nonlinear model may include using a nonlinear motion model to perform adaptive filtering state prediction and / or adaptive filtering state update.

[0113] Optionally, using a nonlinear motion model to perform adaptive filtering state prediction includes: initializing the Jacobian matrix J A , based on the Jacobian matrix, the derivatives of each state quantity are taken and the state covariance is calculated.

[0114] Optionally, initialize the Jacobian matrix J A is a (12×12) unit matrix. When the angular velocity ω t-1 When it is not 0, the Jacobian matrix J A Calculation method:

[0115]

[0116]

[0117]

[0118]

[0119]

[0120]

[0121]

[0122] J A (7, 8) = J A (9, 11) = J A (10, 12) = Δt (6)

[0123] Q=(Q l , Q w , Q h , Q x , Q y , Q z , Q yaw , Q vx , Q vy , Q ax , Q ay , Q ω ) (7)

[0124] In the above formula (6), JA represents the state transfer matrix; in the above formula (7), Q represents the prediction error.

[0125] Optionally, the state covariance is calculated based on the Jacobian matrix, and the Jacobian matrix is ​​substituted into equation (8):

[0126]

[0127] Optionally, when the angular velocity is 0, a minimum value is taken.

[0128] As an optional implementation manner, step S3 determines second state information of the target object at a second moment based on state features of multiple dimensions, wherein the second moment is a moment after the first moment.

[0129] Optionally, the second moment may be moment t, and the second state information may be state quantity information, which may include information such as size, center point, orientation angle, velocity component, acceleration component, angular velocity, etc.

[0130] As an optional implementation method, determining the second state information of the target object at the second moment based on state features of multiple dimensions includes: establishing a target matrix corresponding to the state features of multiple dimensions, matching the target matrix, and obtaining the second state information.

[0131] Optionally, establishing a target matrix corresponding to multiple dimensional state features may include: calculating an association matrix of each dimensional state feature:

[0132] Assuming that the detection model detects (i) obstacles in the tth frame and there are (j) tracking sequences, we can finally get an (i×j) M matrix:

[0133]

[0134] In the above formula (9), m ij It represents the feature difference between the (i)th obstacle and the (j)th tracking sequence. The larger the value, the better.

[0135] Set the observation transfer matrix to H:

[0136]

[0137] The observation model of the state quantity is:

[0138] R=(R l , R w , R h , R x , R y , R z , R yaw , R vx , R vy , R ω )

[0139] O t (j) = H × μ t_ (j)

[0140] S t (j) = H × P t (j) × H-+R (11)

[0141] In the above formula (11), R represents the detection system error, μ t_ (j) represents the predicted value of the state quantity, S t (j) indicates the calculation of intermediate variables.

[0142] Optionally, the characteristic distance Maha(i, j) between the detected obstacle and the predicted value of the tracking sequence is calculated:

[0143]

[0144] m ij =Maha(i, j) (12)

[0145] In the above formula (12), Maha(i, j) characteristic distance represents the difference in the characteristics of two obstacles. The smaller the value, the better. t Indicates the detection value of the state quantity.

[0146] As an optional implementation method, a matching algorithm is used to perform error-based distance feature matching calculations on the target matrix to obtain second state information, including: determining that there is a target state sequence that matches the target matrix, performing Kalman filtering on the first state information, and obtaining second state information; determining that there is no target state sequence that matches the target matrix, updating the first state information to the second state information based on a nonlinear model, or determining the second state information based on a newly created target state sequence.

[0147] Optionally, it is determined that there is a target state sequence that matches the target matrix, and a Kalman filter is performed on the first state information based on the nonlinear model to obtain the second state information. The Kalman filter on the first state may be an adaptive extended Kalman filter update on the first state, and the detection error corresponding to the nonlinear model is corrected based on the distance between the target object and the vehicle, and the detection error may be adaptively corrected to meet the adaptive adjustment of parameters of obstacles in different ranges.

[0148] Optionally, an adaptive extended Kalman filter update process is used for the matching result, including: adaptively correcting the detection error; using a nonlinear motion model to adaptively update the extended Kalman filter state to obtain a state prediction quantity, and performing an update process to calculate the state quantity detection value.

[0149] Optionally, an adaptive extended Kalman filter update process is used for the matching result, including: adaptively correcting the detection error; using a nonlinear motion model to adaptively update the extended Kalman filter state to obtain a state prediction quantity, and performing an update process to calculate the state quantity detection value.

[0150] Optionally, the detection error is adaptively corrected by: calculating the velocity component {vx m ,vy m}:

[0151]

[0152]

[0153] Optionally, an adaptive solution is used to correct the measurement variance R. The measurement variance R is related to the range of the obstacle distance from the vehicle. Assuming that the range of the obstacle distance from the vehicle is represented by Range, the larger the range of the obstacle distance from the vehicle, the larger the measurement variance R, indicating that the inaccuracy of the measurement error is greater. The parameters range_limit and thres represent the limit parameters of the measurement error, which can be obtained through testing;

[0154]

[0155] In the above formula (15), Rimprove Represents the detection error adaptive correction parameter.

[0156] Optionally, a nonlinear motion model is used to perform adaptive extended Kalman filter state update to obtain μ t The measured state quantity calculated in the update process is ME t , where μ t = {l t , w t ,h t , x t ,y t , z t , yaw t , vx t ,vy t , ax t ,ay t ,ω t}, ME t = {l m , w m ,h m , x m ,y m , z m , yaw m , vx m ,vy m}:

[0157] K t =P t_ H T (HP t_ H T +R+R improve ) -1 (16)

[0158] μ t =μ t_ +K t (ME t -H×μ t_ ) (17)

[0159] P t =(IK t H)P t_ (18)

[0160] In the above formula (16), K represents the Kalman gain; in the above formula (18), I represents the unit matrix.

[0161] Optionally, it is determined that there is no target state sequence matching the target matrix, and the first state information is updated to the second state information based on the nonlinear model, or the second state information is determined based on a newly created target state sequence. The newly created target state sequence may be a newly created tracking sequence or a newly created trajectory initialization.

[0162] Optionally, it is determined that there is no target state sequence that matches the target matrix, including an unmatched tracking sequence and an unmatched detection sequence.

[0163] As an optional implementation manner, step S4, tracking the target object based on the second state information to obtain a tracking result, includes: determining the motion information of the target object and / or the position information of the target object through the second state information.

[0164] Optionally, the state information of the obstacle at the current moment with a smaller variance is obtained through adaptive filtering {l t , w t ,h t , x t ,y t , z t , yaw t , vx t ,vy t , ax t , ax t ,ω t}, based on the current state information obtained, the adaptive filtering state prediction is performed again, so as to carry out the tracking cycle of the next frame. By repeating the above updating process, the speed and position information of the obstacle with low noise and accuracy can be obtained in real time.

[0165] This embodiment obtains the first state information of the target object to be tracked at the first moment; filters the first state information based on a nonlinear model to obtain state features in multiple dimensions; determines the second state information of the target object at the second moment based on the state features in multiple dimensions; and tracks the target object based on the second state information to obtain a tracking result. In other words, the present invention adopts a nonlinear model to filter the state information of the target object, fully considering the state feature information of each dimension of the obstacle, so as to obtain more accurate real-time state information of the target object based on the fusion of detection information and prediction information, thereby achieving the technical effect of improving the effectiveness of nonlinear scene object tracking, and solving the technical problem that the nonlinear scene in target tracking cannot effectively track the object.

[0166] Example 2

[0167] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0168] In actual traffic environments, vehicles are in a complex environment. In order to meet the needs of automatic obstacle avoidance and path planning, it is necessary to screen out valid targets from a large amount of target data, establish corresponding track management, accurately obtain vehicle status information, and track multiple targets in real time.

[0169] In related technologies, classic filtering methods such as Kalman filtering and particle filtering are generally used to estimate the motion and position information of the target. However, their processing capabilities for nonlinear and non-Gaussian scenes are relatively poor. When nonlinear problems are encountered in practice, the tracking deviation is large and good tracking effects cannot be obtained.

[0170] In order to solve the above problems, the present invention adopts a filtering algorithm based on a constant rotation rate and speed motion model, uses an adaptive extended Kalman filter to estimate the obstacle motion, and adopts (intersection over union loss function, abbreviated as IOU) error, direction error and Mahalanobis distance weighted value for data association to meet the adaptive adjustment of obstacle parameters in different ranges, so that the motion information and position information of the obstacle are more accurate; at the same time, based on the matching of error combined with distance feature calculation, the feature information of each dimension of the obstacle is fully considered, so that the feature gap between different obstacles becomes larger and the matching effect is better, and finally the track management is performed through heuristic rules.

[0171] In this embodiment, Figure 2 is a schematic diagram of a method for calculating filter adaptive parameters according to an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0172] Step S2010, obtaining status information at time t-1.

[0173] Optionally, the state information at time t-1 may include target size, center point, orientation angle, velocity component, acceleration component and angular velocity information.

[0174] Step S2020: Adopting a nonlinear motion model to perform adaptive filtering state prediction on the state information acquired at time t-1.

[0175] Optionally, using a nonlinear motion model to perform adaptive filtering state prediction on the state information acquired at time t-1 may include calculation of prediction values ​​of various state quantities and calculation of prediction errors and detection errors of various state quantities.

[0176] Step S2030, obtaining information of the detection model at time t.

[0177] Optionally, the information of the detection model at time t may include size, position, orientation angle and other information.

[0178] Step S2040, performing calculations between state features of each dimension of the correlation matrix.

[0179] Optionally, the calculation between the state features of each dimension of the correlation matrix may include the calculation of the feature distance between the detected obstacle and the predicted value of the tracking sequence. The feature distance represents the difference in the features of the two obstacles, and the smaller the value, the better.

[0180] Step S2050, matching calculation.

[0181] Optionally, matching the calculation result with the information of the detection model obtained at time t can be performed by using a matching algorithm to calculate the association matrix, and the optimization goal is to find the largest number of matching pairs while maximizing the total weight value.

[0182] Optionally, a heuristic rule is used to process the final result obtained by matching.

[0183] Step S2051, the detection error is adaptively corrected according to the distance.

[0184] Optionally, the detection error is adaptively corrected according to the feature distance based on the matched data.

[0185] Step S2052: Adopting a nonlinear motion model to perform adaptive filtering state update.

[0186] Optionally, the component speed of the measured state quantity and the detection error adaptive correction parameter are calculated after matching, so as to adopt the nonlinear motion model to perform adaptive extended Kalman filter state update to obtain the predicted value of the state quantity and the detected value of the state quantity.

[0187] Step S2053: There is no matching tracking sequence.

[0188] Step S2054, accumulate the times and make a judgment based on the set threshold.

[0189] Optionally, the number of unmatched tracking sequences is accumulated and compared with a preset threshold to determine whether it is greater than the threshold.

[0190] Step S2055: using a nonlinear motion model to perform state update.

[0191] Step S2056, deleting the tracking sequence.

[0192] Optionally, a nonlinear motion model is used to update unmatched tracking sequences until the time or number of times of non-update exceeds a set threshold, and then the trajectory is deleted to prevent too many false detections from causing problems for back-end decision planning.

[0193] Step S2057, for the detection sequence that is not matched.

[0194] Step S2058: create a new tracking sequence.

[0195] Optionally, for unmatched detection sequences, a new trajectory is initialized.

[0196] Step S2059, obtaining the status information at time t, including target size, center point, orientation angle, speed, acceleration and angular velocity information.

[0197] In this embodiment, Figure 3 is a schematic diagram of a prediction error calculation method according to an embodiment of the present invention. Figure 3 As shown, the method may include the following steps:

[0198] Step S301, loading the true value annotation file.

[0199] Optionally, the true value annotation file may be true value data, which may include the sequence number, size information, and position information of the obstacle.

[0200] Step S302: classify obstacles according to categories.

[0201] Optionally, different obstacles are classified according to obstacle types, and calculations are performed for different obstacles respectively.

[0202] Step S303: filter outliers based on size, speed, and acceleration information.

[0203] Optionally, abnormal values ​​in the true value of the statistical data are filtered, and a regional range is set for the true value. Only data within the regional range is valid and calculated.

[0204] Step S304, construct the trajectory according to the marked sequence number, and put obstacles with the same ID together.

[0205] Optionally, all the times of the same obstacle are connected in series by marking the serial numbers, and the final data format is a trajectory in which one obstacle corresponds to one time series.

[0206] Step S305: performing interpolation calculation on the trajectory.

[0207] Optionally, the sequence number of the true value is checked to see whether the time of the obstacle is continuous to prevent the loss of the data annotation true value due to occlusion.

[0208] Step S306, calculating the difference between the previous and next frames.

[0209] Step S307, solving the variance Q of each type.

[0210] Optionally, Q can represent the prediction error of each state quantity.

[0211] In this embodiment, Figure 4 is a schematic diagram of a detection error calculation method according to an embodiment of the present invention. Figure 4 As shown, the method may include the following steps:

[0212] Step S4010, loading the Dt detection information file.

[0213] Optionally, the detection information file may be detection data of various state quantities of an image frame, and may include a sequence number, size information, and position information of the obstacle.

[0214] Step S4020: classify obstacles according to categories.

[0215] Optionally, different obstacles are classified according to obstacle types, and calculations are performed for different obstacles respectively.

[0216] Step S4030, filtering outliers based on size, speed, and acceleration information.

[0217] Optionally, abnormal values ​​in the statistical detection data are filtered.

[0218] Step S4040, loading the Gt true value annotation file.

[0219] Optionally, the true value annotation file may be true value data, which may include the sequence number, size information, and position information of the obstacle.

[0220] Step S4050, retaining the value of the same timestamp.

[0221] Optionally, data with the same frame number are compared, and data with the same timestamp as the true value data is retained.

[0222] Step S4060, matching.

[0223] Optionally, the detection data of the same frame and the same timestamp are matched with the true value data.

[0224] Step S4061, perform data range restriction.

[0225] Optionally, a region range is set for the detection information file, and only data within the region range is valid and calculated.

[0226] Step S4062, performing Mahalanobis distance calculation.

[0227] Optionally, the Mahalanobis distance of each obstacle is calculated for the detection data and the true value data to form a metric matrix.

[0228] Step S4063: Hungarian matching.

[0229] Step S4070, obtaining the matching result.

[0230] Optionally, the matching result is obtained, and the matched true value data and the detection data are stored separately.

[0231] Step S4080: threshold limiting is performed, and those greater than a certain threshold are deleted.

[0232] Optionally, thresholds are applied to the stored true value data and detection data, and pairs of detection true value data whose Mahalanobis distance is greater than a certain threshold and are matched are deleted.

[0233] Step S4091, calculating the difference between the same frame detection information and the true value information.

[0234] Step S4092, solving the variance of each type.

[0235] Optionally, the variance of the difference between the detection information and the true value information is calculated.

[0236] The present invention discloses the use of nonlinear models, which has good results in tracking obstacles during turns and speed changes. For different obstacles, automatic parameter adjustment is adopted according to the time when the obstacle appears and the distance from the vehicle. In view of the low accuracy of the existing deterministic target association algorithm for small objects, data association calculation based on error and Mahalanobis distance is adopted to improve the accuracy of association for small object matching. Finally, heuristic rules such as the number of occurrences are used to manage the trajectory of obstacles, reducing the object missed detection rate of obstacles. At the same time, the parameter adjustment method based on previous filtering experience is time-consuming and labor-intensive, and often relies on the work experience of engineers. This patent invents a filtering parameter calculation method that can effectively and accurately calculate the detection error and prediction error in filtering, greatly reducing the time cost and improving the accuracy.

[0237] Example 3

[0238] According to an embodiment of the present invention, a device for tracking an object is also provided. It should be noted that the device for tracking an object can be used to execute the method for tracking an object in the first embodiment.

[0239] Figure 5 FIG. 1 is a schematic diagram of an object tracking device according to an embodiment of the present invention. Figure 5 As shown, the object tracking device 50 may include: an acquisition unit 501 , a filtering unit 502 , a determination unit 503 and a tracking unit 504 .

[0240] The acquisition unit 501 is used to acquire first state information of a target object to be tracked at a first moment.

[0241] A filtering unit 502 is used to filter the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to the prediction error and the detection error of the first state information;

[0242] A determining unit 503 is configured to determine second state information of the target object at a second moment based on the state features of the multiple dimensions, wherein the second moment is a moment after the first moment;

[0243] The tracking unit 504 is configured to track the target object based on the second state information to obtain a tracking result.

[0244] Optionally, the acquisition unit 501 includes: a first processing module, configured to acquire attribute data of the target object at a first moment; and filter abnormal data in the attribute data.

[0245] Optionally, the first processing module includes: a first determining submodule, configured to filter abnormal data in the acquired attribute data of the target object at the first moment.

[0246] Optionally, the acquisition unit 501 includes: a second processing module, configured to determine a prediction error and a detection error based on the attribute data.

[0247] Optionally, the second processing module includes: a first processing submodule, configured to determine a prediction error based on a time series corresponding to the target object and filtered attribute data.

[0248] Optionally, the second processing module includes: a second processing submodule, configured to determine a detection error based on a difference between the filtered attribute data and the corresponding target attribute data.

[0249] In an embodiment of the present invention, first state information of a target object to be tracked at a first moment is acquired through an acquisition unit; the first state information is filtered based on a nonlinear model through a filtering unit to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to a prediction error and a detection error of the first state information; second state information of the target object at a second moment is determined based on the state features of multiple dimensions through a determination unit, wherein the second moment is a moment after the first moment; the target object is tracked based on the second state information through a tracking unit to obtain a tracking result, thereby achieving a technical effect of improving the effectiveness of nonlinear scene object tracking and solving the technical problem that objects cannot be effectively tracked in nonlinear scenes in target tracking.

[0250] Example 4

[0251] According to an embodiment of the present invention, a computer-readable storage medium is further provided. The storage medium includes a stored program, wherein the program executes the object tracking method described in Embodiment 1.

[0252] Example 5

[0253] According to an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein the object tracking method described in Embodiment 1 is executed when the program is run.

[0254] Example 6

[0255] According to an embodiment of the present invention, a vehicle is further provided. The vehicle is used to execute the object tracking method according to the embodiment of the present invention.

[0256] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0257] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0258] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0259] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0260] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0261] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0262] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for tracking an object, characterized in that: include: Acquire first state information of a target object to be tracked at a first moment; Performing filtering processing on the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to a prediction error and a detection error of the first state information, the prediction error and the detection error are determined based on attribute data of the target object at the first moment, and the attribute data is true value data of the target object and / or detection data of the target object; Determine second state information of the target object at a second moment based on the state features of the multiple dimensions, wherein the second moment is a moment after the first moment; The target object is tracked based on the second state information to obtain a tracking result.

2. The method according to claim 1, characterized in that The method further comprises: Acquire attribute data of the target object at the first moment; The prediction error and the detection error are determined based on the attribute data.

3. The method according to claim 2, characterized in that Determining the prediction error based on the attribute data includes: Filtering abnormal data in the attribute data; The prediction error is determined based on a time series corresponding to the target object and the filtered attribute data, wherein the time series includes times when the target object appears multiple times.

4. The method according to claim 2, characterized in that: Determining the detection error based on the attribute data includes: Filtering abnormal data in the attribute data; The detection error is determined based on a difference between the filtered attribute data and the corresponding target attribute data.

5. The method according to claim 1, characterized in that Determining second state information of the target object at a second moment based on the state features of the multiple dimensions includes: Establishing a target matrix corresponding to the state features of the multiple dimensions, wherein the target matrix is ​​used to represent the difference between the state features of the multiple dimensions and features of a target state sequence, wherein the target state sequence includes multiple target state information of the target object; The target matrix is ​​matched to obtain the second state information.

6. The method according to claim 5, characterized in that Matching the target matrix to obtain the second state information includes: Determine whether there is the target state sequence matching the target matrix, perform Kalman filtering on the first state information, and obtain the second state information; It is determined that there is no target state sequence matching the target matrix, and the first state information is updated to the second state information based on the nonlinear model, or the second state information is determined based on the newly created target state sequence.

7. The method according to claim 6, characterized in that Performing Kalman filtering on the first state information to obtain the second state information includes: The first state information is subjected to Kalman filtering based on the nonlinear model to obtain the second state information, wherein the detection error corresponding to the nonlinear model is corrected based on the distance between the target object and the vehicle.

8. The method according to any one of claims 1 to 7, characterized in that Tracking the target object based on the second state information to obtain a tracking result includes: The motion information of the target object and / or the position information of the target object are determined based on the second state information, wherein the tracking result includes the motion information and / or the position information.

9. A tracking device for an object, characterized in that: include: An acquisition unit, used to acquire first state information of a target object to be tracked at a first moment; a filtering unit, configured to filter the first state information based on a nonlinear model to obtain state features of multiple dimensions, wherein the nonlinear model is established based on filtering parameters corresponding to a prediction error and a detection error of the first state information, the prediction error and the detection error are determined based on attribute data of the target object at the first moment, and the attribute data is true value data of the target object and / or detection data of the target object; A determining unit, configured to determine second state information of the target object at a second moment based on the state characteristics of the multiple dimensions, wherein the second moment is a moment after the first moment; A tracking unit is used to track the target object based on the second state information to obtain a tracking result.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, and the program is used to execute the object tracking method according to any one of claims 1 to 8.

Images