Data Fusion Method, Apparatus, Electronic Device, and Computer Storage Medium

Abstract

An embodiment of the present application provides a data fusion method, apparatus, electronic device, and computer storage medium. The data fusion method is used to perform tracking detection on a target object within the detection range of a radar, and specifically includes: obtaining measurement data corresponding to echo signals at the k-th moment collected by at least two radars; selecting associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment; and determining the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object; performing fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k-1)-th moment to determine the estimated position of the target object at the k-th moment. This data fusion method is more accurate in positioning the target object.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of intelligent driving technology, and in particular, to a data fusion method, apparatus, electronic device, and computer storage medium. Background Art

[0002] In the field of intelligent driving, in order to improve the autonomy and safety of vehicle driving, radars (such as millimeter-wave radars) can be installed at different positions of the vehicle. By using the echo signals of the radars, it is possible to sense whether there are targets that should be concerned during vehicle driving around the vehicle, and perform tracking detection on these targets for driving decision-making or driving assistance decision-making. In this process, due to the complex and changeable environment in which the vehicle is located, there is a problem that it is difficult to accurately determine the echo signals from the targets, which in turn leads to inaccurate perception of the target positions and inaccurate target tracking detection based on the echo signals. Summary of the Invention

[0003] In view of this, the embodiments of the present application provide a data fusion solution to at least partially solve the above problems.

[0004] According to the first aspect of the embodiments of the present application, a data fusion method is provided for tracking and detecting target objects within the detection range of a radar. Specifically, it includes: obtaining measurement data corresponding to the echo signals at the k-th moment collected by at least two radars; selecting the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment; determining the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object; and performing a fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated position of the target object at the k-th moment.

[0005] According to the second aspect of the embodiments of the present application, a data fusion apparatus is provided, including: a first acquisition module for obtaining measurement data corresponding to the echo signals at the k-th moment collected by at least two radars; a selection module for selecting the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment; a first determination module for determining the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object; and a second determination module for performing a fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated position of the target object at the k-th moment.

[0006] According to a third aspect of the embodiments of the present application, there is provided an electronic device, including: a processor, a memory, a communication interface, and a communication bus. The processor, the memory, and the communication interface complete communication with each other through the communication bus; the memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to the data fusion method described in the first aspect.

[0007] According to a fourth aspect of the embodiments of the present application, there is provided a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the data fusion method described in the first aspect.

[0008] According to the data fusion solution provided by the embodiments of the present application, the target object is detected and located based on the associated measurement data corresponding to the target object in the measurement data of multiple radars. In this way, a part of the measurement data that is not relevant to the target object is screened out, thereby improving the accuracy of positioning. Moreover, by determining the association probability of the associated measurement data and performing fusion processing on the associated measurement data to determine the estimated position of the target object at the k-th moment, the reliability is further improved, the interference of errors is reduced, and the accurate positioning of the target object is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in the embodiments of the present application, and those of ordinary skill in the art can also obtain other drawings based on these drawings.

[0010] Figure 1a It is a flowchart of the steps of a data fusion method according to Embodiment 1 of the present application;

[0011] Figure 1b is Figure 1a a schematic diagram of a scenario example in the illustrated embodiment;

[0012] Figure 2 It is a flowchart of the steps of a data fusion method according to Embodiment 2 of the present application;

[0013] Figure 3a It is a flowchart of the steps of a data fusion method according to Embodiment 3 of the present application;

[0014] Figure 3b It is a schematic diagram of the tracking errors of a single radar and multiple radars;

[0015] Figure 3c It is a schematic diagram of the long-life cycle tracking of a target object by multiple radars;

[0016] Figure 3d Schematic diagram of error variation for long - life - cycle tracking of a target object by multiple radars

[0017] Figure 3e Schematic diagram of the error between the filtered value and the true value of the speed at each moment in the track changing with time

[0018] Figure 4 Structural block diagram of a data fusion device according to Embodiment 4 of the present application

[0019] Figure 5 Structural schematic diagram of an electronic device according to Embodiment 5 of the present application Detailed implementation manners

[0020] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application shall fall within the protection scope of the embodiments of the present application.

[0021] The following further illustrates the specific implementation of the embodiments of the present application with reference to the accompanying drawings of the embodiments of the present application.

[0022] Embodiment 1

[0023] Refer to Figure 1a , which shows the step - flow chart of the data fusion method according to Embodiment 1 of the present application.

[0024] Step S102: Obtain the measurement data corresponding to the echo signals at the k - th moment collected by at least two radars.

[0025] In the field of vehicle autonomous driving, on - vehicle radars are used to detect target objects (such as pedestrians, vehicles, fixed obstacles, etc.) within a certain range around the vehicle, so as to control the vehicle according to the detection results, thereby realizing autonomous driving.

[0026] For example, by installing multiple radars on the vehicle and making these radars face different directions, the detection around the vehicle is realized. In theory, by regularly emitting detection signals outward by the radars, if obstacles (such as people, vehicles, walls, trees, etc.) are encountered during the propagation of the detection signals, these obstacles will reflect the detection signals to form echo signals, and the measurement data corresponding to the echo signals can be obtained by processing the echo signals. The measurement data carries the predicted positions of the obstacles that form the echo signals.

[0027] In fact, due to the complex and ever-changing environment in which the vehicle is located, on the one hand, the echo signals received by the radar may be interfered with during the propagation process, resulting in inaccurate predicted positions carried in the measurement data determined based on the echo signals; on the other hand, the echo signals received by the radar may not be formed by the reflection of the detection signals by real obstacles, but signals generated by other interference sources. Since the characteristics of this signal are similar to those of the echo signal, it is received by the radar, and the radar cannot distinguish whether there are such false alarm signals in the received echo signals. Therefore, the measurement data corresponding to the echo signals collected by the radar is processed through the data fusion method of the present application, so as to realize the tracking and detection of the target object within the detection range of the radar, and thus accurately determine the target object existing around the vehicle.

[0028] When performing data fusion, at the k-th moment, the radar receives the echo signal and processes the echo signal to obtain the corresponding measurement data. In order to expand the detection range and improve the tracking of the life cycle of the target object, multiple radars are installed on the vehicle. Each radar sends the measurement data to a fusion center. The fusion center can be a data processing device with data processing capabilities configured on the vehicle, or a data processing device set in the cloud that is data-connected to the radar. The fusion center centrally processes the measurement data of multiple radars, so as to integrate the measurement data of multiple radars and improve the accuracy of detecting the target object.

[0029] Step S104: Select the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment.

[0030] The predicted position area of the target object is used to represent the area where the target object may appear at the k-th moment. Based on the continuity of the movement of the target object, the position where the target object may move to at the k-th moment can be predicted according to the estimated position of the target object at the (k - 1)-th moment. Due to the uncertainty of the movement direction, the positions where the target object may move to at the k-th moment form an area, that is, the predicted position area.

[0031] If the predicted position indicated by the measurement data is not within the predicted position area corresponding to the target object, it means that the echo signal of this measurement data is not reflected by the target object, and there is no need to determine the estimated position of the target object based on these measurement data. Therefore, the fusion center screens out the associated measurement data related to the target object from all the measurement data obtained at the k-th moment. These associated measurement data can be the measurement data that falls within the predicted position area.

[0032] In this way, the measurement data irrelevant to the target object can be screened out, thus avoiding the adverse effects of these irrelevant measurement data on the estimated position of the target object.

[0033] Step S106: Determine the association probability between the associated measurement data and the target object. The association probability is used to indicate the likelihood that the echo signal corresponding to the associated measurement data comes from the target object.

[0034] Since it is impossible to accurately determine from the associated measurement data which associated measurement data is generated by the target object reflecting the detection signal and which is a false alarm caused by errors, it is assumed that all associated measurement data may come from the target object. However, the likelihood of these associated measurement data coming from the target object is different. The association probability represents the likelihood that the corresponding associated measurement data is the echo signal from the target object.

[0035] In a feasible manner, the conditional probability of each associated measurement data can be calculated based on all the associated measurement data as the corresponding association probability.

[0036] Step S108: Based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)th moment, perform a fusion calculation to determine the estimated position of the target object at the kth moment.

[0037] For example, determine the weight value of the predicted position of each associated measurement data according to the association probability of each associated measurement data, and perform a fusion calculation (such as summation) based on the weight value, the predicted position corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)th moment to determine the estimated position of the target object at the kth moment.

[0038] During the fusion calculation process, the association probability of each associated measurement data is relied on. Therefore, it is possible to accurately estimate the position of the target object by fusing the associated measurement data of multiple radars. Through a fusion center, the associated measurement data of multiple radars are uniformly processed, resulting in a higher estimation accuracy. Moreover, it can ensure the timely and real-time processing of the associated measurement data of the radars, guaranteeing accurate detection and positioning of the target object in the case of dense clutter. In addition, the detection ranges covered by multiple radars are more extensive, enabling the detection and positioning of the target object over a long life cycle.

[0039] The following combines Figure 1b , and illustrates the implementation process of the data fusion method with a specific usage scenario as follows:

[0040] In this usage scenario, the vehicle equipped with radars is denoted as vehicle A, and six radars are respectively installed on the front side, the rear side, and the four top corners of vehicle A.

[0041] When vehicle A is in motion, six radars periodically emit detection signals outward. Taking the k-th moment as an example, if there are vehicles, pedestrians, or other fixed obstacles within the detection range of the radar, the detection signals will be reflected to form echo signals, which can be received by the radar. Of course, the radar may also receive signals emitted by some other signal sources in the environment. Since it is difficult for the radar to distinguish whether the received signal is an echo signal or other interference signals, all signals received by the radar are considered to be echo signals here. These echo signals themselves may be interfered with and carry noise, and there may also be interference signals in the echo signals.

[0042] For the received echo signals, the radar performs signal processing to form measurement data, which carries a predicted position, that is, the position of the object that reflects the echo signal. This object may be a real object or a false object caused by noise, error, etc.

[0043] In order to be able to detect and locate the target object by integrating the measurement data of multiple radars, multiple radars can send the measurement data to a fusion center, which can be any device with data processing capabilities, such as a chip, a server, etc.

[0044] For these measurement data, according to the predicted position area of the target object, the measurement data located within this position prediction area is selected as the associated measurement data of the target object, thus achieving a primary filtering of the measurement data and screening out the irrelevant measurement data.

[0045] Then, based on the calculation of the associated measurement data, the associated probability of the associated measurement data coming from the target object is calculated, and then the weight value of the corresponding associated measurement data is determined according to the associated probability. Based on the weight value, the predicted position of the target object indicated by the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment, the estimated position of the target object at the k-th moment is determined.

[0046] Since the weight value is calculated according to the associated probability, and the associated probability is a conditional probability calculated based on all associated measurement data, the fusion processing of the measurement data of different radars is realized, thereby improving the positioning accuracy.

[0047] Through this embodiment, the target object is detected and located based on the associated measurement data corresponding to the target object in the measurement data of multiple radars. In this way, a part of the measurement data irrelevant to the target object is screened out, thereby improving the accuracy of positioning. Moreover, by determining the associated probability of the associated measurement data and performing fusion processing on the associated measurement data to determine the estimated position of the target object at the k-th moment, the reliability is further improved, the interference of errors is reduced, and the accurate positioning of the target object is realized.

[0048] The data fusion method in this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones, etc.), and PC machines, etc.

[0049] Embodiment 2

[0050] Referring to Figure 2 , a flowchart showing the steps of the data fusion method according to Embodiment 2 of the present application is shown.

[0051] In this embodiment, taking the execution of the data fusion method in the fusion center as an example, the execution process will be described. The fusion center can be an in-vehicle data processing chip, which is connected to multiple radar data. This method can ensure that the measurement data of multiple radars can be transmitted to the fusion center in a timely and fast manner, ensuring a low data transmission delay. Or the fusion center can also be a data processing chip set outside the vehicle, which can be connected to the radar through a wired network or a wireless network and receive the measurement data transmitted by the radar.

[0052] In this embodiment, the data fusion method includes the aforementioned steps S102 to S108. Among them, step S104 includes the following sub-steps:

[0053] Sub-step S1041: Obtain the inter-frame displacement amount, and determine a reference reference position for predicting the predicted position area of the target object at the k-th moment according to the inter-frame displacement amount and the estimated position of the target object at the (k - 1)-th moment.

[0054] The inter-frame displacement amount can be determined according to the estimated position at the (k - 1)-th moment and the estimated position at the (k - 2)-th moment. Since the time difference between two adjacent frames, that is, between the k-th moment and the (k - 1)-th moment, is very short (it can be in milliseconds), and based on the continuity of speed, the movement between two adjacent frames can be regarded as uniform motion. That is to say, the change in the estimated position from the (k - 2)-th moment to the (k - 1)-th moment can be regarded as the inter-frame displacement amount from the (k - 1)-th moment to the k-th moment.

[0055] Based on the sum of the estimated position of the target object at the (k - 1)-th moment and the inter-frame displacement amount, the sum result is used as the reference reference position of the target object at the k-th moment.

[0056] Step S1042: Predict the predicted position area of the target object at the k-th moment according to the reference reference position and the covariance corresponding to the estimated position of the target object at the (k - 1)-th moment according to the Gaussian distribution.

[0057] The covariance is used to represent the accuracy of the estimated position of the corresponding target object. For each moment of the target object, the corresponding estimated position and the corresponding covariance are obtained.

[0058] When determining the predicted position area of the target object, the reference reference position is used as the center of the predicted position area, that is, the mean value in the Gaussian distribution, and the covariance is used as the variance of the Gaussian distribution, then a distribution area, that is, the predicted position area, can be determined.

[0059] Step S1043: Select the measurement data within the predicted position area as the associated measurement data of the target object.

[0060] Since each measurement data indicates the predicted position of the target object, if the predicted position indicated by the measurement data is within the predicted position area, it means that the measurement data may be formed by the target object reflecting the detection signal, that is, the associated measurement data; conversely, if the measurement data is not within the predicted position area, it means that the measurement data has nothing to do with the target object, that is, it is not the associated measurement data.

[0061] In this way, the associated measurement data associated with the target object can be selected from the measurement data obtained from multiple radars, and the irrelevant measurement data can be filtered out, thereby improving the accuracy of the subsequent position estimation of the target object.

[0062] In a specific example, assume that the number of radars on the vehicle is Ns, the total number of known target objects is T, and the number of measurement data corresponding to the i-th radar at the k-th moment is zk. Then, according to the predicted position area of the target object, the associated measurement data associated with the target object t in the measurement data of the i-th radar is determined, and the number of associated measurement data is denoted as mki. In step S106, determining the association probability for the associated measurement data can be realized as: based on the multiple associated measurement data corresponding to the target object, calculate the conditional probability that the echo signals corresponding to the multiple associated measurement data come from the target object, as the association probability between the associated measurement data and the target object.

[0063] For a certain associated measurement data, when calculating the association probability, the conditional probability of the associated measurement data can be determined based on the total number of the associated measurement data of the target object as its association probability.

[0064] Step S108 can be implemented by the following sub-steps:

[0065] Sub-step S1081: According to the radar to which the associated measurement data of the target object belongs, determine the combination of associated measurement data that meets the set rules.

[0066] The following two relationships are satisfied between the target object and the associated measurement data in the set rules:

[0067] One: Each associated measurement data comes from at most one target object;

[0068] Second: Each target object has at most one associated measurement data with it as the source. When there are multiple radars, the target object can be detected by multiple radars simultaneously. This makes it such that among the associated measurement data belonging to the same radar in the determined combination of associated measurement data, there is at most one associated measurement data from the said target object. Under the condition of satisfying the above two conditions, the association between the associated measurement data of a certain radar and the target object can be described by an association event, and the combination of the association events of all radars constitutes the combination of the associated measurement data of the target object.

[0069] If the i-th radar corresponds to two associated measurement data denoted as {mi1, mi2}, then the association relationship between radar i and the target object denoted as ai are respectively {1, 0, 0}, {0, 1, 0}, {0, 0, 1}, and. Among them, {1, 0, 0} indicates that all the associated measurement data of radar i has nothing to do with the target object and is measurement data generated due to false alarms. {0, 1, 0} indicates that the first associated measurement data, namely mi1, is the associated measurement data from the target object. {0, 0, 1} indicates that the second associated measurement data, namely mi2, is the associated measurement data from the target object.

[0070] On this basis, when the number of radars is Ns, all possible combinations of associated measurement data corresponding to the target object (which can be denoted as association events) can be obtained. Among them, a combination of associated measurement data corresponding to a set of associated measurement data and satisfying the above-set rules can be expressed as ams(t) → {{0, 1, …, mk1}, {0, 1, …, mk2}, …, {0, 1, …, mkNs}}.

[0071] If the corresponding relationship between the associated measurement data reflected by the association relationship ai at the k-th moment and the target object is correct, it can be expressed as Hai(k); if the corresponding relationship between the associated measurement data reflected by the association event ams at the k-th moment and the target object is correct, it can be expressed as Hams(k). For each known target object, let the event L = (l1, l2, … lNs), 0 ≤ li ≤ mki, 0 ≤ i ≤ Ns, indicating that the association between the target object t and the association indicated by the event L at the k-th moment is a correct association. In this case, ams = L. Similarly, indicating that the association between the target object t and the associated measurement data of the i-th radar at the k-th moment is a correct association. In this case, ai(t) = li.

[0072] Based on this, is the union of mutually exclusive events Hams(k), is the union of mutually exclusive events Hai(k). Thus, all possible combinations of the associated measurement data of the target object can be determined.

[0073] Sub-step S1082: Perform Kalman filtering based on the predicted positions and association probabilities corresponding to the associated measurement data in the associated measurement data combination, and the estimated position of the target object at the (k - 1)-th moment, to determine the estimated position corresponding to each associated measurement data combination.

[0074] Since the errors between each radar are independent of each other, based on the association probability of each associated measurement data, the probability of each association relationship can be obtained, and the probability of the associated measurement data combination of multiple radars is the product of the probabilities of the association relationships of multiple radars.

[0075] The principle is as follows: Given the set of associated measurement data Z obtained by multiple radars at the k-th moment k the conditional probability of any feasible associated measurement data combination L holding can be expressed as:

[0076] where represents the probability of the occurrence of the associated measurement data combination L in the set of associated measurement data Z k .

[0077] Since the measurement errors between each radar are independent of each other, there is:

[0078] Substituting the above formula gives:

[0079]

[0080] That is to say, the probability of the associated measurement data combination of multiple radars is the product of the associated measurement data combinations of individual radars.

[0081] Based on the probability of the associated measurement data combination, the estimated position of the target object is determined, for example:

[0082] If the k-th moment is the first moment, that is, there is no (k - 1)-th moment, then set the estimated position and covariance at the (k - 1)-th moment to 0, and determine the estimated position according to the associated measurement data at the k-th moment. If the k-th moment is not the first moment, then there is an estimated position at the (k - 1)-th moment.

[0083] When determining the estimated position, the following formula is used:

[0084] where is the estimated position at the k-th moment, is the estimated position at the (k - 1)-th moment, and K i(k) is the Kalman filter coefficient, which is determined in an appropriate manner according to the association probability of the associated measurement data. is the i-th associated measurement data in the feasible combination L of associated measurement data. H i (k) is the transformation matrix, which is preset according to the parameters of the radar.

[0085] Based on this formula, the estimated position of the target object at the k-th moment can be determined when each combination L of associated measurement data holds.

[0086] Sub-step S1083: Sum the estimated positions corresponding to each combination of associated measurement data, and determine the estimated position of the target object at the k-th moment according to the summation result.

[0087] The estimated position at the k-th moment can be determined according to the following formula:

[0088] where, is the estimated position of the target object at the k-th moment. is the estimated position of the target object at the (k - 1)-th moment. Sum the estimated positions corresponding to multiple combinations of associated measurement data, and then sum the summation result with the estimated position at the (k - 1)-th moment as the estimated position of the target object at the k-th moment.

[0089] Optionally, in order to characterize the accuracy of the estimated position, the method in this embodiment may further include step S110 and step S112.

[0090] Step S110: Calculate the covariance of the combination of associated measurement data.

[0091] In a feasible manner, the covariance of the combination L of associated measurement data can be expressed as It can be determined in an existing manner, so it will not be elaborated here.

[0092] Step S112: Determine the covariance corresponding to the estimated position of the target object at the k-th moment according to the covariance of the combination of associated measurement data.

[0093] In a feasible manner, the covariance of the target object at the k-th moment can be denoted as:

[0094]

[0095] where, is the probability corresponding to the combination L of associated measurement data, is the estimated position at the k-th moment, is the transpose of the estimated position at the k-th moment.

[0096] Through this embodiment, the associated measurement data of multiple radars can be fused, and the estimated position and covariance of the target object at the k-th moment can be determined, so as to achieve accurate and timely detection and positioning of the target object.

[0097] The data fusion method in this embodiment can be executed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablets, mobile phones, etc.), and PC machines, etc.

[0098] Embodiment III

[0099] Referring to Figure 3a , a schematic flow chart of the steps of the data fusion method according to Embodiment III of the present application is shown.

[0100] In this embodiment, the data fusion method includes the aforementioned steps S102 to S108. Optionally, it may further include steps S110 and S112. Among them, steps S104 to S108 can adopt any of the aforementioned suitable implementation manners.

[0101] Based on the detection and positioning of the target object, the trajectory tracking and management of the target object can be realized. Among them, the trajectory of the target object is used to indicate the estimated position of the target object at different moments. Since there are errors and false alarms in the associated measurement data when realizing the detection and positioning of the target object based on the radar, it is impossible to determine whether the target object actually exists within the detection range only based on the target objects detected at one moment. Instead, it is necessary to determine whether the target object actually exists based on the detection and positioning results at multiple moments and determine its estimated position.

[0102] For this reason, when determining a new target object based on the measurement data output by the radar, a corresponding trajectory can be created for the target object, and an existence probability can be initialized for the trajectory. Subsequently, the trajectory and the existence probability of the trajectory are updated based on the measurement data at multiple moments, so as to process the trajectory according to the existence probability, such as confirmation processing, deletion processing, and update processing, etc. Among them, the confirmation processing means determining that the target object corresponding to the trajectory actually exists within the detection range, and the estimated position in the trajectory can be output to the vehicle decision-making unit.

[0103] In order to realize the management of the trajectory, the method further includes:

[0104] Step S114: Determine the existence probability of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object.

[0105] In a feasible manner, step S114 includes the following sub-steps:

[0106] Sub-step S1141: Determine the probability of missed detection of the target object at the k-th moment based on the association probability of the associated measurement data corresponding to the target object, combined with the radar to which the associated measurement data belongs, the tracking probability and detection probability of the radar, and the existence probability of the target object at the (k - 1)-th moment.

[0107] For example, the probability of missed detection is the probability that the target object exists in the detection area but there is no position measurement data associated with it, denoted as The probability of missed detection can be determined according to the following formula:

[0108]

[0109] where is the existence probability of the target object at the (k - 1)-th moment. If the k-th moment is the first moment, that is, there is no calculated existence probability at the (k - 1)-th moment, then the existence probability at the (k - 1)-th moment is determined as a default value, such as values less than 0.1 like 0.05, 0.07, etc. If the k-th moment is not the first moment, then the existence probability at the (k - 1)-th moment has been calculated and can be directly obtained.

[0110] p o is the detection probability of the radar, which is an inherent parameter of the radar and is determined by the performance of the radar itself.

[0111] P w is the tracking probability, which can be determined by looking up a table based on the covariance.

[0112] is the false alarm probability, which refers to the probability that all associated measurement data comes from other objects, that is, the probability of the associated measurement data combination L0 = (0, 0,..., 0). It can be determined based on the association probability of the associated measurement data corresponding to the target object. The calculation method is similar to the calculation method of the probability of the aforementioned associated measurement data combination, so it is briefly described as follows: Taking the estimated position at the k-th moment as the mean of the Gaussian distribution and the covariance at the k-th moment as the standard deviation of the Gaussian distribution, determine the probability that none of the associated measurement data comes from the target object.

[0113] In this way, the probability of missed detection can be calculated conveniently and quickly, and then the existence probability of the target track can be determined quickly based on the probability of missed detection, making the number of cycles required for track processing fewer and achieving fast processing.

[0114] Sub-step S1142: Determine the detection probability of the target object according to the association probability corresponding to the associated measurement data corresponding to the target object.

[0115] When the target object is related to at least one associated measurement data, then there must be a corresponding feasible combination L of associated measurement data. Therefore, the detection probability is the sum of the probabilities of other feasible combinations L of associated measurement data except the combination L0 of associated measurement data, that is, it can be expressed as How to calculate the probability of the combination L of associated measurement data has been described in the foregoing process, so it will not be elaborated here.

[0116] Sub-step S1143: Sum the missed detection probability and the detection probability, and use the sum result as the existence probability of the target object at the k-th moment.

[0117] The existence probability can be expressed as The existence probability is the sum of the detection probability and the missed detection probability, and it can be expressed as:

[0118] Step S116: Process the target track corresponding to the target object according to the existence probability and the set track processing threshold.

[0119] Specifically, track processing includes track initiation, track confirmation, and track deletion.

[0120] Among them, for the target objects not saved in the track table, a new track can be initiated for each of them, and this process is track initiation. For example, at the first moment after the radar is started, the measurement data at the first moment of the radar is obtained. Since there are no known target objects at the first moment, each measurement data can be considered to have a corresponding target object, or the measurement data with the distance between the indicated positions less than a set value (the set value can be determined as needed) can be aggregated into a target object, and a new track is initiated for the target object. The newly initiated track can be saved in the track table, and a default existence probability, such as 0.05, etc., is set for the new track, and the track state of the new track is the track initiation state.

[0121] Since it cannot be determined whether the target object of the new track really exists during track initiation, it is necessary to determine whether the target object exists or has left the detection area according to the position measurement data at subsequent moments. For this reason, as described above, the track processing threshold includes, but is not limited to, a confirmation threshold for determining that the target object corresponding to the track is a real target object, and a deletion threshold for indicating that the target object corresponding to the track has left the detection area of the multiple radars.

[0122] In one case, step S116 can be implemented as: if the existence probability is greater than the confirmation threshold and the track state of the target track is the track initiation state, then change the track state of the target track to the track confirmation state.

[0123] The confirmation threshold can be 0.9, for example. When the existence probability is greater than 0.9, it indicates the existence of the target object. Therefore, the track state of the corresponding target track can be updated to the track confirmation state to indicate that the target object is within the detection range of the radar.

[0124] After the target track is confirmed, the position state information of the target track at the k-th moment can be output to the decision-making unit of the vehicle. In this way, the decision-making unit of the vehicle can make decisions based on the position state information of the target object at the k-th moment, such as controlling the vehicle to decelerate or change direction to avoid colliding with the target object.

[0125] Alternatively, in another feasible way, step S116 can be implemented as follows: If the existence probability is less than the deletion threshold and the existence probability at at least one historical moment in the target track is greater than the confirmation threshold, then change the track state of the target track to the track deletion state.

[0126] If the existence probability increases and then quickly decreases below the deletion threshold, it means that the target has left the detection area of the radar. Therefore, the track state of the target track can be directly updated to the track deletion state. In this way, the position state information of the target object does not need to be output to the decision-making unit anymore, so that the decision-making unit can obtain more accurate information about the target objects around the vehicle and make correct and reasonable decisions.

[0127] Multiple radars are used to detect the target object and obtain position measurement data. The existence probability used in processing the target track is determined based on the miss detection probability and the association probability between "measurement - track", so that the existence probability can change quickly according to the association probability, reducing the error of track processing and shortening the convergence time of the target track, effectively improving the timeliness of detection.

[0128] In this embodiment, the tracking of the target object is realized through centralized multi-vehicle millimeter-wave radar fusion. Multiple targets can be tracked for the vehicle carrying the radar. Since the distribution distances of multiple vehicle-mounted radars are not far, real-time processing of multiple radars can be carried out in a timely manner. Therefore, a centralized multi-sensor fusion method is adopted, and a fusion center uniformly processes the measurement data of all radars. Based on the measurement data, the position state information of the target object is filtered and weighted and summed through extended Kalman filtering to obtain the position state information of the target object. In addition, when processing the track, the existence probability of the track is considered, and the determination and deletion of the track can be incorporated into unified processing, which can effectively reduce the estimation error of the initial state and reduce the convergence time of the track.

[0129] Due to the use of multiple radars, the detection areas of different radars are different, and there are cases where the detection areas of multiple radars overlap in certain areas. When the same target object is detected by multiple radars, the position and state information of the target object can be updated by fusing the measurement data of multiple radars. When a single target object moves forward in the front right of the radar vehicle, the tracking error of a single radar and the tracking error of the fusion of two radars with the same accuracy change with the processing cycle, as Figure 3b shown. It can be seen that whether it is single-radar tracking or multi-radar tracking, the tracking error decreases continuously with time. The multi-radar fusion converges faster than single-radar tracking, and the tracking error after convergence is smaller. This is because multiple radars bring more information about the target and improve the tracking accuracy. Based on this, the deficiencies of a single radar in time and space detection capabilities are solved, and while achieving long-term target tracking, the tracking accuracy of the target is improved.

[0130] To compare the long-life cycle tracking of the target brought by multi-radar fusion, a target vehicle moves from the left rear of the radar carrier to the left side of the radar carrier and then to the front left of the radar carrier. The detection situations of different radars at different positions are also different. Due to multiple radars, the detection range of the target has been greatly extended. The simulation results are as Figure 3c shown. It can be seen that the target object first appears within the measurement ranges of the rear radar and the rear left side radar. As the target moves, the front left radar and the front radar also detect the target. Then, the rear radar and the rear left side radar no longer detect the target. The tracking error of the target object decreases continuously with the number of processing cycles. At about 50 cycles, since the high-precision front and rear radars can no longer detect the target object, only the side radars with larger errors provide measurements, so the tracking error increases.

[0131] In this embodiment, as Figure 3d shown, by fusing the measurement data of the forward radar, the candidate radar, and four side radars installed at the corners of the vehicle, stable 360-degree long-life cycle tracking of multiple moving target objects is achieved, and within a certain detection range of the vehicle itself, the motion state of the target object can be predicted in a timely manner to provide sufficient position and state information of the target object, thereby solving the deficiencies of a single radar in time and space detection capabilities, and improving the tracking accuracy of the target while achieving long-term target tracking.

[0132] In addition, by adopting centralized multi-radar fusion, there is only one centralized processing unit (i.e., the fusion center), which uniformly processes the measurement data of multiple radars, and has high estimation accuracy. Since the distribution distances of multiple vehicle-mounted radars are not far, a centralized processing unit can timely perform real-time processing on the measurement data of multiple radars.

[0133] Based on multi-radar joint probabilistic data association, the position and state information of the target object is filtered and weighted and summed through extended Kalman filtering to obtain the track information of the target. Figure 3e (The difference between the speed estimated value and the true value of the Kalman filter is shown). Since the existence probability of the track is considered, the determination and deletion of the track can be incorporated into unified processing, which can effectively reduce the estimation error of the initial state and reduce the convergence time of the track. Simulation and measured data verification show that the proposed data processing can achieve stable tracking of multiple target objects in a dense clutter environment and can achieve 360-degree target tracking of multiple targets with a long life cycle.

[0134] Embodiment 4

[0135] Refer to Figure 4 , which shows the structural block diagram of the data fusion device according to Embodiment 4 of the present application.

[0136] In this embodiment, the data fusion device includes:

[0137] The first acquisition module 402 is configured to acquire measurement data corresponding to the echo signals collected by at least two radars at the k-th moment;

[0138] The selection module 404 is configured to select the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment;

[0139] The first determination module 406 is configured to determine the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object;

[0140] The second determination module 408 is configured to perform fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated position of the target object at the k-th moment.

[0141] Optionally, the selection module 404 includes:

[0142] The second acquisition module 4041 is configured to acquire the inter-frame displacement amount, and determine a reference reference position for predicting the predicted position area of the target object at the k-th moment according to the inter-frame displacement amount and the estimated position of the target object at the (k - 1)-th moment;

[0143] The prediction module 4042 is configured to predict the predicted position area of the target object at the k-th moment according to the reference reference position and the covariance corresponding to the estimated position of the target object at the (k - 1)-th moment according to the Gaussian distribution;

[0144] A third determination module 4043, configured to select measurement data within the predicted position area as the associated measurement data of the target object.

[0145] Optionally, the first determination module 406 is configured to calculate, according to multiple pieces of associated measurement data corresponding to the target object, conditional probabilities that echo signals corresponding to the multiple pieces of associated measurement data come from the target object, as the association probabilities between the associated measurement data and the target object.

[0146] Optionally, the second determination module 408 is configured to determine, according to the radar to which the associated measurement data of the target object belongs, an associated measurement data combination that meets a set rule, where, in an associated measurement data combination, at most one piece of associated measurement data belonging to the same radar is the associated measurement data from the target object; perform Kalman filtering according to the predicted positions and association probabilities corresponding to the associated measurement data included in the associated measurement data combination, and the estimated position of the target object at the (k - 1)-th moment, to determine the estimated position corresponding to each associated measurement data combination; sum up the estimated positions corresponding to each associated measurement data combination, and determine the estimated position of the target object at the k-th moment according to the summation result.

[0147] Optionally, the apparatus further includes:

[0148] A calculation module 410, configured to calculate the covariance of the associated measurement data combination;

[0149] A fourth determination module 412, configured to determine the covariance corresponding to the estimated position of the target object at the k-th moment according to the covariance of the associated measurement data combination.

[0150] Optionally, the apparatus further includes:

[0151] A fifth determination module 414, configured to determine the existence probability of the target object at the k-th moment according to the association probabilities of the associated measurement data corresponding to the target object;

[0152] A track processing module 416, configured to process the target track corresponding to the target object according to the existence probability and a set track processing threshold.

[0153] Optionally, the fifth determination module 414 is configured to determine the probability of missed detection of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object, in combination with the radar to which the associated measurement data belongs, the tracking probability and detection probability of the radar, and the existence probability of the target object at the (k-1)-th moment; determine the detection probability of the target object according to the association probability corresponding to the associated measurement data corresponding to the target object; sum the probability of missed detection and the detection probability, and use the sum result as the existence probability of the target object at the k-th moment.

[0154] Optionally, the track processing threshold includes a confirmation threshold for determining that the target object corresponding to the track is a real target object. The track processing module 416 is configured to change the track state of the target track to a track confirmation state if the existence probability is greater than the confirmation threshold and the track state of the target track is a track start state.

[0155] Optionally, the track processing threshold includes a confirmation threshold for determining that the target object corresponding to the track is a real target object and a deletion threshold for indicating that the target object corresponding to the track has left the detection area of the multiple radars. The confirmation threshold is greater than the deletion threshold. The track processing module 416 is configured to change the track state of the target track to a track deletion state if the existence probability is less than the deletion threshold and the existence probability at at least one historical moment in the target track is greater than the confirmation threshold.

[0156] The data processing device in this embodiment is used to implement the corresponding data processing methods in the foregoing multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be elaborated here. In addition, the function implementation of each module in the data processing device in this embodiment can refer to the corresponding part of the description in the foregoing method embodiments, which will not be elaborated here either.

[0157] Embodiment Five

[0158] Referring to Figure 5 , a schematic structural diagram of an electronic device according to Embodiment Five of the present application is shown. The specific implementation of the electronic device in the specific embodiment of the present application is not limited.

[0159] As Figure 5 shown, the electronic device may include: a processor 502, a communications interface 504, a memory 506, and a communication bus 508.

[0160] Wherein:

[0161] The processor 502, the communication interface 504, and the memory 505 communicate with each other via the communication bus 508.

[0162] The communication interface 504 is used to communicate with other electronic devices or servers.

[0163] The processor 502 is used to execute the program 510, and specifically can execute the relevant steps in the above-mentioned data processing method embodiments.

[0164] Specifically, the program 510 may include program code, and this program code includes computer operation instructions.

[0165] The processor 502 may be a central processing unit (CPU), or a specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. One or more processors included in the intelligent device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0166] The memory 506 is used to store the program 510. The memory 506 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0167] The program 510 is specifically used to cause the processor 502 to perform the operations corresponding to any of the foregoing data fusion methods.

[0168] For the specific implementation of each step in the program 510, reference may be made to the corresponding steps and descriptions in the corresponding units in the above-mentioned data fusion method embodiments, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated here.

[0169] It should be noted that according to the needs of implementation, each component / step described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of the components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0170] The method according to the embodiments of the present application can be implemented in hardware, firmware, or be implemented as software or computer code that can be stored in a recording medium (such as a CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or be implemented as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium and downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored on such a software process on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (such as RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the data fusion method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the data fusion method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the data fusion method shown herein.

[0171] Those of ordinary skill in the art can realize that the units and method steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0172] The above embodiments are only used to illustrate the embodiments of the present application, rather than to limit the embodiments of the present application. Those of ordinary skill in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application, and the patent protection scope of the embodiments of the present application should be defined by the claims.

Claims

1. A data fusion method for tracking and detecting target objects within the detection range of a radar, specifically including: Obtaining measurement data corresponding to the echo signals at the k-th moment collected by at least two radars; Selecting the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment; Determining the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object, and the association probability is the probability corresponding to the conditional probability determined based on the number of the total associated measurement data of the target object for this associated measurement data; Performing fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated position of the target object at the k-th moment; The method further includes: Determining the existence probability of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object; Processing the target track corresponding to the target object according to the existence probability and a set track processing threshold.

2. The method according to claim 1, wherein, The step of selecting the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment includes: Obtaining the inter-frame displacement amount, and determining a reference position for predicting the predicted position area of the target object at the k-th moment according to the inter-frame displacement amount and the estimated position of the target object at the (k - 1)-th moment; Predicting the predicted position area of the target object at the k-th moment according to the reference position and the covariance corresponding to the estimated position of the target object at the (k - 1)-th moment according to the Gaussian distribution; Selecting the measurement data within the predicted position area as the associated measurement data of the target object.

3. The method according to claim 1 or 2, wherein The step of determining the association probability between the associated measurement data and the target object includes: Calculating, respectively, the conditional probabilities that the echo signals corresponding to multiple associated measurement data come from the target object based on the multiple associated measurement data corresponding to the target object, and taking them as the association probabilities between the associated measurement data and the target object.

4. The method according to claim 1, wherein The step of performing fusion calculation on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated position of the target object at the k-th moment includes: Determining a combination of associated measurement data that meets the set rules according to the radar to which the associated measurement data of the target object belongs, where in a combination of associated measurement data, at most one of the associated measurement data belonging to the same radar is the associated measurement data from the target object; Performing Kalman filtering according to the predicted positions and association probabilities of the associated measurement data included in the combination of associated measurement data, and the estimated position of the target object at the (k - 1)-th moment to determine the estimated positions corresponding to each combination of associated measurement data; Sum the estimated positions corresponding to each of the associated measurement data combinations, and determine the estimated position of the target object at the k-th moment according to the summation result.

5. The method according to claim 4, wherein, The method further includes: Calculate the covariance of the associated measurement data combination; Determine the covariance corresponding to the estimated position of the target object at the k-th moment according to the covariance of the associated measurement data combination.

6. The method according to claim 1, wherein, The determining the probability of existence of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object includes: Determine the probability of missed detection of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object, in combination with the radar to which the associated measurement data belongs, the tracking probability and detection probability of the radar, and the probability of existence of the target object at the (k - 1)-th moment; Determine the probability of detection of the target object according to the association probability corresponding to the associated measurement data corresponding to the target object; Sum the probability of missed detection and the probability of detection, and use the summation result as the probability of existence of the target object at the k-th moment.

7. The method according to claim 1, wherein The track processing threshold includes a confirmation threshold for determining that the target object corresponding to the track is a real target object; The processing the target track corresponding to the target object according to the probability of existence and the set track processing threshold includes: If the probability of existence is greater than the confirmation threshold and the track state of the target track is the track start state, change the track state of the target track to the track confirmation state.

8. The method according to claim 1, wherein, The track processing threshold includes a confirmation threshold for determining that the target object corresponding to the track is a real target object and a deletion threshold for indicating that the target object corresponding to the track has left the detection areas of multiple radars, and the confirmation threshold is greater than the deletion threshold; The processing the target track corresponding to the target object according to the probability of existence and the set track processing threshold includes: If the probability of existence is less than the deletion threshold and the probability of existence at at least one historical moment in the target track is greater than the confirmation threshold, change the track state of the target track to the track deletion state.

9. A data fusion device, comprising: A first acquisition module, configured to acquire measurement data corresponding to echo signals at the k-th moment collected by at least two radars; A selection module, configured to select the associated measurement data of the target object from the measurement data according to the predicted position area of the target object at the k-th moment; A first determination module, configured to determine the association probability between the associated measurement data and the target object, where the association probability is used to indicate the possibility that the echo signal corresponding to the associated measurement data comes from the target object, and the association probability is the probability corresponding to the conditional probability of determining the associated measurement data based on the total number of the associated measurement data of the target object; A second determination module, configured to perform fusion calculation based on the predicted position of the target object indicated by the associated measurement data, the association probability corresponding to the associated measurement data, and the estimated position of the target object at the (k - 1)-th moment, to determine the estimated position of the target object at the k-th moment; A processing module, configured to determine the existence probability of the target object at the k-th moment according to the association probability of the associated measurement data corresponding to the target object; and process the target track corresponding to the target object according to the existence probability and a set track processing threshold.

10. An electronic device, comprising: A processor, a memory, a communication interface, and a communication bus, where the processor, the memory, and the communication interface complete communication with each other through the communication bus; The memory is used to store at least one executable instruction, and the executable instruction causes the processor to perform operations corresponding to the data fusion method according to any one of claims 1-8.

11. A computer storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the data fusion method according to any one of claims 1-8.

Images