Point cloud radar extended target data association method

Abstract

The invention provides a point cloud radar extended target data association method, and the method comprises the steps: obtaining target state information and historical information, constructing a historical information updating network, inputting the target state information and the historical information into the historical information updating network, and obtaining updated historical information and track historical information covariance; obtaining measurement information and a target state prediction value, constructing an error calculation network, and inputting the updated historical information, the track historical information covariance, the measurement information and the target state prediction value into the error calculation network to obtain a measurement error term and an error term state covariance; obtaining a corrected state predicted value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information; and calculating Kalman gain according to the corrected state predicted value and the corrected measurement covariance, and completing track updating to obtain an updated track. According to the method, the association weight is corrected, the tracking accuracy of the mass center motion state of the target is improved, and the accuracy of track updating is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of millimeter wave radars, and in particular to a point cloud radar extended target data association method. Background Art

[0002] With the development of autonomous driving and intelligent transportation systems, millimeter-wave radar has become one of the key sensors for environmental perception due to its advantages of all-day, all-weather, high resolution and low cost. The angular resolution of traditional millimeter-wave radar is too low, and the observed target scattering points are sparse, so it cannot be used for L4 and above autonomous driving. To solve the technical bottleneck, point cloud radar has improved the angular resolution and added high-dimensional observation, which can provide higher-quality point clouds and achieve high-precision environmental perception.

[0003] As the key to point cloud radar environmental perception, the multi-target tracking algorithm can complete the target track initiation and maintenance based on scattering point measurement. Among them, the track initiation stage is to determine the number of new targets in the scene and establish a track for them; the track maintenance stage is to use the data association algorithm to find the correct measurement of the track and use the above observations to complete the update of the track motion state. In the track maintenance stage, the mainstream data association algorithm is implemented based on the Bayesian criterion, including nearest neighbor, joint probabilistic data association and probabilistic multi-hypothesis data interconnection. The above algorithms can achieve high-precision data association. However, for the increasingly complex traffic environment, the above algorithms still face the difficulty of associating multiple scattering points and multiple tracks. The higher resolution of the point cloud radar makes the target present multiple scattering points with spatial extension. The measurement error is affected by factors such as the difference in target morphology, deviating from the mixed Gaussian model, resulting in association errors. Both the nearest neighbor and probabilistic data association assume that a track matches only one measurement point, and do not make full use of the target's scattering point information, resulting in limited association performance; the joint probabilistic data association and probabilistic multi-hypothesis data interconnection algorithms can assign multiple scattering points to a track, but cannot deal with the association error problem caused by the complex distribution of measurement errors. Multiple tracks and multiple scattering points will interfere with data association, resulting in reduced tracking accuracy and even track loss. The accelerated urbanization process has led to frequent traffic congestion. Due to the unstable measurement process, the targets in the above traffic area are densely distributed, and the target scattering points overlap with the clutter, which increases the difficulty of associating multiple scattering points with multiple tracks. Summary of the invention

[0004] Based on this, it is necessary to provide a point cloud radar extended target data association method to address the above technical problems.

[0005] A point cloud radar extended target data association method comprises the following steps:

[0006] Acquire target state information and historical information, construct a historical information update network, input the target state information and the historical information into the historical information update network, and obtain the covariance of the updated historical information and the track historical information;

[0007] Acquire measurement information and target state prediction value, construct an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain the measurement error term and the error term state covariance;

[0008] Obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information;

[0009] The Kalman gain is calculated according to the corrected state prediction value and the corrected measurement covariance, and the track update is completed to obtain an updated track.

[0010] In one embodiment, it also includes:

[0011] Obtaining an authentic track, and constructing a loss function according to the authentic track and the updated track;

[0012] The historical information update network and the error calculation network are trained by the loss function until the historical information update network and the error calculation network converge, thereby obtaining updated historical information update network and error calculation network.

[0013] In one embodiment, constructing a historical information update network includes:

[0014] Calculate the historical information influence coefficient by resetting the gate:

[0015]

[0016] Among them, r k represents the historical information influence coefficient, represents the i-th target state at the k-1th moment, σ() represents the sigmoid activation function, represents historical information, * represents the convolution layer, W1 represents the convolution kernel size is W1, W2 represents the convolution kernel size is W2, and b1 represents the bias term of the reset gate;

[0017] The influence coefficient of the target state is calculated by updating the gate:

[0018]

[0019] Among them, u k represents the influence coefficient of the target state, σ() represents the sigmoid activation function, represents the i-th target state at the k-1th moment, W3 represents the convolution kernel size is W3, W4 represents the convolution kernel size is W4, represents historical information, b2 represents the bias term of the update gate;

[0020] Calculate candidate history information:

[0021]

[0022] in, represents candidate historical information, tanh() is the hyperbolic tangent activation function, represents the i-th target state at the k-1th moment, W5 represents the convolution kernel size is W5, W6 represents the convolution kernel size is W6, r k represents the historical information influence coefficient, represents the Hadamard product, represents historical information, b3 represents the bias term of the candidate historical information network;

[0023] Calculate update history information:

[0024]

[0025] in, Indicates update history information. represents the mth update unit at the kth time, represents the mth update unit at the k-1th time, represents the mth update unit at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th moment;

[0026] Calculate the covariance of track history information:

[0027]

[0028] in, represents the covariance of track history information, g k () represents the updating unit of the historical information covariance at the kth moment, g k-1 () represents the updating unit of the historical information covariance at the k-1th moment, g k-2 () represents the updating unit of the historical information covariance at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th time.

[0029] In one embodiment, constructing an error calculation network includes:

[0030] Calculate the measurement error term:

[0031]

[0032] in, represents the measurement error term, σ() represents the sigmoid activation function, L1() and L2() represent the linear transformation functions of different transformation dimensions, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment;

[0033] Compute the error term state covariance:

[0034]

[0035]

[0036]

[0037] in, represents the error term covariance, L1() and L2() represent the linear transformation functions of different transformation dimensions, σ() represents the sigmoid activation function, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment, () T represents the transpose operation, F k-1 represents the state transition matrix of the CA model, represents the i-th target state at the k-1th time, v k represents the process noise, represents the one-step predicted state covariance of the i-th target at the k-1th moment, represents the state covariance of the i-th target at the k-1th time, Q k represents the noise covariance, v k Subject to the noise covariance Q k Gaussian distribution.

[0038] In one embodiment, obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information includes:

[0039] Calculate the corrected state prediction:

[0040]

[0041] in, represents the corrected state prediction value, H k represents the measurement matrix of the target, represents the target state prediction value, represents the measurement error term, w k represents the measurement noise;

[0042] Compute the corrected measurement covariance:

[0043]

[0044] in, represents the corrected measurement covariance, H k Represents the measurement matrix of the target, P k+1|k represents the one-step predicted state covariance at the kth moment, () T represents the transpose operation, R k represents the measurement noise covariance, represents the error term covariance.

[0045] A point cloud radar extended target data association system, used to implement the point cloud radar extended target data association method as described above, comprising:

[0046] An information updating module, used for acquiring target state information and historical information, constructing a historical information updating network, inputting the target state information and the historical information into the historical information updating network, and obtaining the covariance of the updated historical information and the track historical information;

[0047] An error calculation module is used to obtain measurement information and target state prediction values, construct an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain a measurement error term and an error term state covariance;

[0048] An information correction module, used for obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information;

[0049] The track updating module is used to calculate the Kalman gain according to the corrected state prediction value and the corrected measurement covariance, and complete the track updating to obtain the updated track.

[0050] A device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a point cloud radar extended target data association method described in each of the above embodiments are implemented.

[0051] A storage medium stores a computer program, which, when executed by a processor, implements the steps of a point cloud radar extended target data association method described in each of the above embodiments.

[0052] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: on the basis of the classic data association process, the present invention can use the historical information update network to learn the centroid evolution law from the tracks of different types of targets, construct an error calculation network, calculate the measurement error by constructing the error calculation network, correct the association weight, improve the tracking accuracy of the target's center of mass motion state, and ensure the accuracy of the track update. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the overall algorithm structure of a point cloud radar extended target data association method in one embodiment;

[0054] Figure 2 A schematic diagram of a flow chart of a point cloud radar extended target data association method in one embodiment;

[0055] Figure 3 A schematic diagram of a historical information update network in one embodiment;

[0056] Figure 4 A schematic diagram of an error calculation network in one embodiment;

[0057] Figure 5 It is a structural schematic diagram of a point cloud radar extended target data association system in one embodiment;

[0058] Figure 6 A schematic diagram of the internal structure of a device in one embodiment;

[0059] Figure 7 This is a comparison diagram of the track results of the method at a distance of 3m in an embodiment;

[0060] Figure 8 This is a comparison diagram of the track results of the method at a spacing of 5m in an embodiment;

[0061] Fig. 9 This is a comparison diagram of the track results of the method at a spacing of 7m in an embodiment;

[0062] Fig.10 In one embodiment, the clutter density of this method is 0.1m -2 Comparison chart of track results;

[0063] Fig.11 In one embodiment, the clutter density of this method is 0.05m -2 Comparison chart of track results;

[0064] Fig.12 Result diagram of the root mean square error of the track of the method at a distance of 3m in an embodiment;

[0065] Fig.13 This is a diagram showing the root mean square error of the track under a spacing of 5 m in an embodiment of the present method;

[0066] Fig.14 This is a diagram showing the root mean square error of the track under a spacing of 7 m in an embodiment of the present method;

[0067] Fig.15 In one embodiment, the clutter density of this method is 0.1m -2 The root mean square error result diagram of the track under

[0068] Fig.16 In one embodiment, the clutter density of this method is 0.05m -2 The root mean square error result diagram of the track under

[0069] Fig.17 A schematic diagram of a distribution scenario of vehicles of different sizes in one embodiment;

[0070] Fig.18 It is a track result diagram of vehicles of different sizes in one embodiment;

[0071] Fig.19 Result diagram of the root mean square error of the tracks of vehicles of different sizes in one embodiment. DETAILED DESCRIPTION

[0072] In view of the association error problem caused by factors such as dense target distribution, complex morphology, and complex measurement error modeling in the background technology, this application provides a point cloud radar extended target data association method, which learns the measurement model error terms of the target from the historical track, and uses the error terms to reconstruct the data association gate to improve the association accuracy. A historical information update network and a measurement error learning network are designed, wherein the former is based on the gated recurrent neural network design and implementation, which can continuously extract the historical state of the track and iteratively update the historical information for each track; the latter is used to measure the model error terms and covariance to compensate for the measurement model mismatch.

[0073] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.

[0074] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present application belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0075] For ease of understanding, the terms involved in the embodiments of the present invention are explained below:

[0076] Data association: Data association is actually a process of connecting signals from the same object at different times along the time axis. Data association is usually performed before state estimation. Only by obtaining accurate data association processing results can the correctness of subsequent processing be guaranteed.

[0077] In one embodiment, Figure 1 As shown, an overall algorithm structure diagram of a point cloud radar extended target data association method is provided.

[0078] In one embodiment, Figure 2 As shown, a point cloud radar extended target data association method is provided, comprising the following steps:

[0079] Step S201, obtaining target state information and historical information, constructing a historical information update network, inputting the target state information and the historical information into the historical information update network, and obtaining the covariance of the updated historical information and the track historical information.

[0080] Specifically, each track has corresponding historical information and an update network. The target state information includes the i-th target state. The historical information is constructed using the motion state and covariance of the track. The historical information update network is implemented based on a gated recurrent neural network. As the measurement time goes by, the historical information uses the update unit to continuously extract information from the track motion state to complete the iterative update.

[0081] On this basis, the historical information update network is constructed including:

[0082] Calculate the historical information influence coefficient by resetting the gate:

[0083]

[0084] Among them, r k represents the historical information influence coefficient, represents the i-th target state at the k-1th moment, σ() represents the sigmoid activation function, represents historical information, * represents the convolution layer, W1 represents the convolution kernel size is W1, W2 represents the convolution kernel size is W2, and b1 represents the bias term of the reset gate;

[0085] The influence coefficient of the target state is calculated by updating the gate:

[0086]

[0087] Among them, u k represents the influence coefficient of the target state, σ() represents the sigmoid activation function, represents the i-th target state at the k-1th moment, W3 represents the convolution kernel size is W3, W4 represents the convolution kernel size is W4, represents historical information, b2 represents the bias term of the update gate;

[0088] Calculate candidate history information:

[0089]

[0090] in, represents candidate historical information, tanh() is the hyperbolic tangent activation function, represents the i-th target state at the k-1th moment, W5 represents the convolution kernel size is W5, W6 represents the convolution kernel size is W6, r k represents the historical information influence coefficient, represents the Hadamard product, represents historical information, b3 represents the bias term of the candidate historical information network;

[0091] Calculate update history information:

[0092]

[0093] in, Indicates update history information. represents the mth update unit at the kth time, represents the mth update unit at the k-1th time, represents the mth update unit at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th moment;

[0094] Calculate the covariance of track history information:

[0095]

[0096] in, represents the covariance of track history information, g k () represents the updating unit of the historical information covariance at the kth moment, g k-1 () represents the updating unit of the historical information covariance at the k-1th moment, g k-2 () represents the updating unit of the historical information covariance at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th time.

[0097] Specifically, Figure 3 As shown in the figure, as the measurement time goes by, the historical information m uses the update unit to continuously extract information from the track state x to complete the iterative update. Set the k-1th moment as the current moment, then the kth moment is the next moment, the k-2th moment is the previous moment, and so on. For the kth update unit, input the i-th target state at the current moment and historical information The unit processes through the reset gate and the update gate to obtain the track history information at time k and pass it to the next unit.

[0098] First, use the reset gate to calculate The influence coefficient r k ;

[0099]

[0100] Where * represents the convolution layer, the convolution kernel parameters and bias coefficients are W and b; σ(·) is the sigmoid activation function. and The convolution operation with convolution kernel size W1 and W2 is respectively performed, and the bias term b1 of the reset gate is added. Finally, the historical information influence coefficient r is calculated by the sigmoid activation function. k .

[0101] Then, the update gate is used to calculate the influence coefficient u of the target state k ;

[0102]

[0103] Afterwards, the candidate history information is calculated

[0104]

[0105] in, represents the Hadamard product; tanh(·) represents the hyperbolic tangent activation function. First, with r k Calculate the reset historical information and compare it with the current input information Convolve together, and pass the convolution result as input to an activation function to calculate the candidate historical information for the next moment

[0106] Finally, calculate the updated update history information

[0107]

[0108] When u k When it is close to 1, it means that Get more information from k A value close to 0 indicates that the previous state information needs to be selectively forgotten.

[0109] In summary, in the starting frame, the track history information of the i-th target is is the mth update unit, The gated recurrent neural network is used to extract regular centroid state evolution information from historical tracks, and the target state information at the current moment is introduced to obtain the updated historical information at time k:

[0110]

[0111] Similarly, assuming g k (·) is the updating unit of historical information covariance, then the covariance of track historical information at time k is The calculation is as follows:

[0112]

[0113] In this embodiment, the historical information is the three-dimensional position and speed information of the target, and an update network is built using RNN to complete the iterative update of the historical information at time k.

[0114] Step S202, obtain measurement information and target state prediction value, build an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain the measurement error term and the error term state covariance.

[0115] Specifically, the measurement information includes the measurement at time k. The error calculation network calculates the covariance of the measurement error and the error term according to the prior distribution characteristics of the historical information of different targets, corrects the associated weights, and improves the tracking accuracy of the target's center of mass motion state. Due to the complex statistical distribution of the measurement error, the influence of factors such as the superposition of the measurement angle, the expansion form, and the occlusion, the measurement deviates from the mixed Gaussian model, so the measurement error term will be calculated based on the current moment measurement, the target state prediction value, and the historical information prior.

[0116] On this basis, the error calculation network is constructed including:

[0117] Calculate the measurement error term:

[0118]

[0119] in, represents the measurement error term, σ() represents the sigmoid activation function, L1() and L2() represent the linear transformation functions of different transformation dimensions, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment;

[0120] Compute the error term state covariance:

[0121]

[0122]

[0123]

[0124] in, represents the error term covariance, L1() and L2() represent the linear transformation functions of different transformation dimensions, σ() represents the sigmoid activation function, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment, () T represents the transpose operation, F k-1 represents the state transition matrix of the CA model, represents the i-th target state at the k-1th time, v k represents the process noise, represents the one-step predicted state covariance of the i-th target at the k-1th moment, represents the state covariance of the i-th target at the k-1th time, Q krepresents the noise covariance, v k Subject to the noise covariance Q k Gaussian distribution.

[0125] Specifically, Figure 4 As shown, the initial input at time k is the measurement z k , target state prediction value and historical information and its covariance Among them, due to the inconsistent numerical scales of dimensions such as speed, position, and height, z k and After cascading, normalization needs to be performed uniformly; and After cascading, the sigmoid activation function is used to improve the nonlinearity of the network. The above initial input is cascaded to obtain the final input form, and the measurement error term and the corresponding error term state covariance are calculated through different network layers.

[0126] Calculating the measurement error term The error calculation network is implemented by two linear layers and an activation function layer. The first linear layer reduces the dimension of the input, reducing the cascaded input from 1×236 to 1×32; the second linear layer reduces 1×32 to the 1×2 dimension of the measurement. The input data dimension 1×236 is obtained by adding the measurement number 1×100×2, the target state dimension 1×4, and the historical information dimension 1×32.

[0127] Calculate the error term covariance The calculation process is the same as above, but the error covariance term adds a transposed multiplication network layer, whose goal is to calculate a covariance matrix of size 2×2.

[0128] L1() and L2(·) represent the linear transformation functions of different transformation dimensions respectively, then and The expression is as follows:

[0129]

[0130]

[0131] The i-th target state at time k-1 is known The target state prediction value of the target is obtained by calculating the Kalman filter under the CA model And covariance:

[0132]

[0133]

[0134] Among them, F k-1 is the state transfer matrix of the CA model; v k and Q k are process noise and noise covariance, respectively, v k Subject to the noise covariance Q k Gaussian distribution.

[0135] In this embodiment, historical information and measurement information can be used to calculate the measurement error term and the error term state covariance, and correct the track centroid state.

[0136] Step S203, obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information.

[0137] Specifically, the error term overcomes the influence of target morphology, sensor, etc. on the measurement distribution by correcting the measurement prediction value in the measurement model; the error term covariance improves the correlation gate by correcting the new information covariance, thereby improving the correlation accuracy.

[0138] On this basis, according to the measurement error term, the error term state covariance and the target state information, obtaining the corrected state prediction value and the corrected measurement covariance includes:

[0139] Calculate the corrected state prediction:

[0140]

[0141] in, represents the corrected state prediction value, H k represents the measurement matrix of the target, represents the target state prediction value, represents the measurement error term, w k represents the measurement noise;

[0142] Compute the corrected measurement covariance:

[0143]

[0144] in, represents the corrected measurement covariance, H k Represents the measurement matrix of the target, P k+1|k represents the one-step predicted state covariance at the kth moment, () T represents the transpose operation, R k represents the measurement noise covariance, represents the error term covariance.

[0145] Specifically, the corrected state prediction value is calculated to obtain the center of the associated gate. Then, the measurement covariance is corrected using the error term state covariance, and the gate is constructed in combination with the gate center to filter out valid measurements entering the gate. Finally, the interconnection probability of valid measurements is calculated based on the JPDA association algorithm to complete data association.

[0146] Measurement prediction value under the classic algorithm The calculation formula is as follows:

[0147]

[0148] Among them, w k is the corresponding measurement noise, with covariance R k The measurement error deviates from the Gaussian distribution, and the complex deviation term is calculated by network calculation to correct the measurement prediction value. Therefore, the measurement process of the i-th target is defined as:

[0149]

[0150] in, To correct the state prediction value, Corresponding to the measurement error term caused by factors such as measurement angle, expansion shape and occlusion.

[0151] At this time, the expression of the associated wave gate is:

[0152]

[0153] in, is the corrected measurement covariance, and γ is the gate threshold. k If each radar measurement data in satisfies the above formula, it is considered to be a valid measurement and enters the interconnection probability calculation.

[0154] The calculation formula is as follows:

[0155]

[0156] Among them, R k is the measurement noise covariance, is the error term covariance.

[0157] Calculate the interconnection probability of the i-th target according to the joint probabilistic data association algorithm Enter track update.

[0158] Step S204, calculating the Kalman gain according to the corrected state prediction value and the corrected measurement covariance, and completing the track update to obtain an updated track.

[0159] On this basis, it also includes:

[0160] Obtaining an authentic track, and constructing a loss function according to the authentic track and the updated track;

[0161] The historical information update network and the error calculation network are trained by the loss function until the historical information update network and the error calculation network converge, thereby obtaining updated historical information update network and error calculation network.

[0162] Specifically, the network loss function is composed of the root mean square error between the track prediction value calculated by each algorithm and the track true value. The network continuously adjusts the parameters of the network layer during the training phase by minimizing the error loss value, thereby improving the association performance of this method.

[0163] The present invention can, on the basis of the classical data association process, utilize the historical information update network to learn the centroid evolution law from the tracks of different types of targets, construct an error calculation network, calculate the measurement error by constructing the error calculation network, correct the association weight, improve the tracking accuracy of the target's centroid motion state, and ensure the accuracy of track update.

[0164] It should be noted that the method of the embodiment of the present invention can be performed by a single device, such as a computer or a server. The method of this embodiment can also be applied in a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present invention, and the multiple devices will interact with each other to complete the described method.

[0165] It should be noted that some embodiments of the present invention are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0166] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present invention also provides a point cloud radar extended target data association system.

[0167] refer to Figure 5 , the point cloud radar extended target data association system comprises:

[0168] The information updating module 501 is used to obtain target state information and historical information, construct a historical information updating network, input the target state information and the historical information into the historical information updating network, and obtain the covariance of the updated historical information and the track historical information;

[0169] The error calculation module 502 is used to obtain measurement information and target state prediction values, construct an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain the measurement error term and the error term state covariance;

[0170] An information correction module 503, configured to obtain a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information;

[0171] The track updating module 504 is used to calculate the Kalman gain according to the corrected state prediction value and the corrected measurement covariance, and complete the track update to obtain an updated track.

[0172] For the convenience of description, the above device is described as being divided into various modules according to their functions. Of course, when implementing the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0173] The device of the above embodiment is used to implement a corresponding point cloud radar extended target data association method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0174] Based on the same inventive concept, corresponding to any of the above-mentioned embodiments and methods, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a point cloud radar extended target data association method described in any of the above embodiments is implemented.

[0175] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.

[0176] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0177] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.

[0178] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.

[0179] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).

[0180] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).

[0181] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.

[0182] The electronic device of the above embodiment is used to implement a corresponding point cloud radar extended target data association method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiment, which will not be repeated here.

[0183] Based on the same inventive concept, corresponding to any of the above-mentioned embodiment methods, the present invention also provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute a point cloud radar extended target data association method as described in any of the above embodiments.

[0184] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0185] The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute a point cloud radar extended target data association method as described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0186] The technical solutions in the present invention will be described clearly and completely below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0187] Embodiment 1

[0188] In this example, in order to verify the sensitivity of this method to target spacing, clutter density and vehicle size, this embodiment conducts simulation experiments based on simulation data built by MATLAB software.

[0189] The simulation data set contains 1620 simulation scenarios. With reference to the classic parallel scenarios in data association problems, each scenario is built with the target's scattering point distribution, target spacing, clutter density, and target motion mode as variables. Assuming that the clutter is randomly and evenly distributed in the scene, the specific parameters of the simulation data are shown in Table 1:

[0190]

[0191]

[0192] Table 1 Specific parameters of simulation data

[0193] In order to be close to the actual driving scene, the target initial state value is randomly selected in the range near the target true value. The training set, validation set and test set are divided in a ratio of 4:1:1. During the network training process, 16 sets of data are randomly selected for training each time. The network optimizer uses Adam and the initial learning rate is 10 -2 ,The learning rate attenuator uses an exponential attenuator, which decays to 50% of the original value every 1000 training rounds.,The tracking algorithm and the parameters required for training are shown in Table 2.

[0194]

[0195] Table 2 Tracking algorithm and parameters required for training

[0196] To facilitate the presentation of the correlation results, all the tracks in the accompanying figures are presented in the form of a bird’s-eye view of the scattering point observations. Figure 7-9 and attached Figure 10-11 The comparison results between the present invention and the traditional data association method at different target intervals and different clutter densities are shown respectively, where (a) is the actual track of the target; (b) is the track result under the NNDA algorithm; (c) is the track result under the JPDA algorithm; and (d) is the track result under the algorithm proposed in the present invention.

[0197] Attached Figure 12-16 The root mean square error results of the proposed association algorithm and the traditional method at different target intervals and different clutter densities are statistically analyzed, where (a) and (b) are the root mean square error results of distance and speed for track one, respectively; (c) and (d) are the root mean square error results of distance and speed for track two, respectively.

[0198] The above track graph and error statistics show that: when the target spacing is 3m, the distance RMSE is less than 2m because the present invention does not have any association errors; when the target spacing is 5m, the PMHT algorithm and the proposed algorithm do not have any association errors, but affected by factors such as the target morphology, the PMHT algorithm is not as good as the present invention in determining the track of the large vehicle and the affiliation of the scattering points; when the target spacing is 7m, all algorithms are correctly associated. The present invention can adapt to the driving scene under the traffic millimeter wave radar.

[0199] In addition, in order to verify the correlation performance of the present invention for different vehicle sizes, the experiment selected two common vehicle sizes, a car and a bus, as experimental objects, and built a Fig.17 The traffic scene shown in the figure. In the scene, the right vehicle is a truck, and the front and rear vehicles are cars. The spatial positions of the vehicles are as follows. All three targets maintain uniform linear motion. The lateral distance between the front and right vehicles is 3 to 6 meters, and the longitudinal distance between the front and rear vehicles is 4 to 7 meters.

[0200] Attached Fig.18 The track results of the algorithm proposed in this invention and the traditional method. The track results show that the three tracks of the NNDA algorithm are all associated incorrectly, resulting in track overlap; the tracks of the front and rear vehicles in the JPDA algorithm completely overlap; the PMHT algorithm associates the measurement of the right vehicle with the front vehicle in the early stage of the track, resulting in a decrease in the tracking accuracy of the two vehicles and the true value.

[0201] Attached Fig.19 The root mean square error results of each track of the algorithm proposed in this invention and the traditional method are shown in Figure 2. The results show that in terms of handling vehicle measurement mismatch, the proposed algorithm has stronger correlation performance than the PMHT algorithm, and the tracking result is closer to the true value of the target center of mass.

[0202] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present invention (including the claims) is limited to these examples. Under the concept of the present invention, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0203] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0204] In addition, to simplify the description and discussion, and in order not to obscure the embodiments of the present invention, known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided figures. In addition, the devices may be shown in the form of block diagrams to avoid obscuring the embodiments of the present invention, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform on which the embodiments of the present invention will be implemented (i.e., these details should be fully within the scope of understanding of those skilled in the art). Where specific details (e.g., circuits) are set forth to describe exemplary embodiments of the present invention, it will be apparent to those skilled in the art that embodiments of the present invention may be implemented without these specific details or with variations in these specific details. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0205] Although the invention has been described in conjunction with specific embodiments of the invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.

[0206] The embodiments of the present invention are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the protection scope of the present invention.

Claims

1. A point cloud radar extended target data association method, characterized in that: include: Acquire target state information and historical information, construct a historical information update network, input the target state information and the historical information into the historical information update network, and obtain the covariance of the updated historical information and the track historical information; Acquire measurement information and target state prediction value, construct an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain the measurement error term and the error term state covariance; Obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information; The Kalman gain is calculated according to the corrected state prediction value and the corrected measurement covariance, and the track update is completed to obtain an updated track.

2. According to claim 1, a point cloud radar extended target data association method is characterized in that: Also includes: Obtaining an authentic track, and constructing a loss function according to the authentic track and the updated track; The historical information update network and the error calculation network are trained by the loss function until the historical information update network and the error calculation network converge, thereby obtaining updated historical information update network and error calculation network.

3. According to claim 1, a point cloud radar extended target data association method is characterized in that: The construction history information update network includes: Calculate the historical information influence coefficient by resetting the gate: Among them, r k represents the historical information influence coefficient, represents the i-th target state at the k-1th moment, σ() represents the sigmoid activation function, represents historical information, * represents the convolution layer, W1 represents the convolution kernel size is W1, W2 represents the convolution kernel size is W2, and b1 represents the bias term of the reset gate; Calculate the influence coefficient of the target state through the update gate: Among them, u k represents the influence coefficient of the target state, σ() represents the sigmoid activation function, represents the i-th target state at the k-1th moment, W3 represents the convolution kernel size is W3, W4 represents the convolution kernel size is W4, represents historical information, b2 represents the bias term of the update gate; Calculate candidate history information: in, represents candidate historical information, tanh() is the hyperbolic tangent activation function, represents the i-th target state at the k-1th time, W5 represents the convolution kernel size is W5, W6 represents the convolution kernel size is W6, r k represents the historical information influence coefficient, represents the Hadamard product, represents historical information, b3 represents the bias term of the candidate historical information network; Calculate update history information: in, Indicates update history information. represents the mth update unit at the kth time, represents the mth update unit at the k-1th time, represents the mth update unit at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th moment; Calculate the covariance of track history information: in, represents the covariance of track history information, g k () represents the updating unit of the historical information covariance at the kth moment, g k-1 () represents the updating unit of the historical information covariance at the k-1th moment, g k-2 () represents the updating unit of the historical information covariance at the k-2th moment, represents the i-th target state at the k-2th moment, represents the i-th target state at the k-1th time.

4. According to claim 1, a point cloud radar extended target data association method is characterized in that: The error calculation network is constructed as follows: Calculate the measurement error term: in, represents the measurement error term, σ() represents the sigmoid activation function, L1() and L2() represent the linear transformation functions of different transformation dimensions, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment; Compute the error term state covariance: in, represents the error term covariance, L1() and L2() represent the linear transformation functions of different transformation dimensions, σ() represents the sigmoid activation function, Indicates update history information. represents the covariance of track history information, represents the target state prediction value, z k represents the measurement at the kth moment, () T represents the transpose operation, F k-1 represents the state transition matrix of the CA model, represents the i-th target state at the k-1th time, v k represents the process noise, represents the one-step predicted state covariance of the i-th target at the k-1th moment, represents the state covariance of the i-th target at the k-1th time, Q k represents the noise covariance, v k Subject to the noise covariance Q k Gaussian distribution.

5. According to claim 1, a point cloud radar extended target data association method is characterized in that: The obtaining of a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information comprises: Calculate the corrected state prediction: in, represents the corrected state prediction value, H k represents the measurement matrix of the target, represents the target state prediction value, represents the measurement error term, w k represents the measurement noise; Compute the corrected measurement covariance: in, represents the corrected measurement covariance, H k Represents the measurement matrix of the target, P k+1|k represents the one-step predicted state covariance at the kth moment, () T represents the transpose operation, R k represents the measurement noise covariance, represents the error term covariance.

6. A point cloud radar extended target data association system, characterized in that: A method for associating point cloud radar extended target data as described in claims 1 to 5, comprising: An information updating module, used for acquiring target state information and historical information, constructing a historical information updating network, inputting the target state information and the historical information into the historical information updating network, and obtaining the covariance of the updated historical information and the track historical information; An error calculation module is used to obtain measurement information and target state prediction values, construct an error calculation network, input the update history information, the track history information covariance, the measurement information and the target state prediction value into the error calculation network, and obtain a measurement error term and an error term state covariance; An information correction module, used for obtaining a corrected state prediction value and a corrected measurement covariance according to the measurement error term, the error term state covariance and the target state information; The track updating module is used to calculate the Kalman gain according to the corrected state prediction value and the corrected measurement covariance, and complete the track updating to obtain the updated track.

7. A device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images