Tracking Device
The tracking device employs a FMB filter to assign one observation point per tracer, reducing hypotheses and calculations, thus addressing the computational challenges of RFS tracking for in-vehicle applications and ensuring real-time accuracy.
Patent Information
- Application Number
- JP2021081239
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-05-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-05-12
AI Technical Summary
RFS tracking methods face challenges in in-vehicle applications due to exponential increases in the number of hypotheses and calculations required for real-time processing, making them difficult to implement effectively.
A tracking device with an information acquisition unit, tracer generation unit, state prediction unit, and state update unit that reduces the number of hypotheses by assigning one observation point to each tracer, using a Fast Multi-Bernoulli (FMB) filter to manage hypotheses and integrate likelihood calculations.
This approach significantly reduces computational load and maintains tracking accuracy, enabling real-time processing for in-vehicle tracking devices by minimizing the number of hypotheses and calculations.
Smart Images

Figure 0007593877000014 
Figure 0007593877000015 
Figure 0007593877000016
Abstract
Description
[Technical field]
[0001] The present disclosure relates to a tracking device for tracking a target. [Background technology]
[0002] NN tracking and RFS tracking are known as tracking methods that successively update the state of a tracker that represents a target candidate being tracked using information from observation points. NN stands for Nearest Neighbor, and RFS stands for Random Finite Set. In NN tracking, the state of a tracker is updated using information from one observation point closest to the tracker. In RFS tracking, information from all observation points within a certain range from the RFS tracker is used. RFS tracking is disclosed in, for example, Patent Document 1.
[0003] In RFS tracking, multiple hypotheses are generated that represent the correspondence between multiple trackers and multiple observation points using the detection probability of each of the multiple observation points and the existence probability of each of the multiple trackers, the likelihood of each hypothesis is calculated, and tracking is performed taking all hypotheses into consideration. In addition, a method is known that reduces the number of hypotheses propagated to the next processing cycle by convolving the state of the tracker updated according to each hypothesis with the likelihood of each hypothesis. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] US Patent Application Publication No. 2019 / 0035088 Summary of the Invention [Problem to be solved by the invention]
[0005] However, as a result of detailed investigation by the inventors, in RFS tracking, the number of hypotheses, and therefore the amount of calculation required for tracking, especially the amount of calculation required for calculating the likelihood of hypotheses, increases exponentially as the number of observation points and trackers increases. As a result, it was found that RFS tracking is difficult to apply to vehicle-mounted tracking devices that require real-time processing.
[0006] One aspect of the present disclosure provides a technique for reducing the amount of calculation while maintaining tracking accuracy. [Means for solving the problem]
[0007] One aspect of the present disclosure is a tracking device, comprising an information acquisition unit (20: S100), a tracer generation unit (20: S180), a state prediction unit (20: S110), and a state update unit (20: S130 to S170).
[0008] The information acquisition unit is configured to acquire, for each processing cycle, observation information from a sensor that observes the state of a target existing around the vehicle. The tracker generation unit is configured to generate a tracker having a state distribution that probabilistically indicates the state of the observation information for an observation point that satisfies a preset generation condition, among an observation point group that is one or more observation points from which observation information has been acquired by the information acquisition unit. The state prediction unit is configured to generate, for each of the existing trackers that are the trackers generated up to the previous processing cycle, a predictive distribution that is a state distribution that predicts the state of the observation information of the existing tracker in the current processing cycle from the state distribution of the existing tracker. The state update unit is configured to update, for each of the existing trackers, the predictive distribution generated by the state prediction unit and the observation information of the observation point group obtained in the current processing cycle.
[0009] Moreover, the state update unit includes an observation allocation unit (20: S130, S144) and a hypothesis generation unit (20: S310 to S320). The observation allocation unit is configured to exclusively allocate an observation point to each of the existing trackers according to the distance between the observation information acquired by the information acquisition unit and the observation information indicated by the prediction distribution. The hypothesis generation unit is configured to generate, for each of the hypotheses belonging to a hypothesis group including a first hypothesis and a second hypothesis, a hypothesis likelihood which is the likelihood of the hypothesis, and a hypothesis distribution which is a state distribution updated on the assumption that the hypothesis is correct, for each of the hypotheses belonging to the hypothesis group including a first hypothesis and a second hypothesis. The first hypothesis is a hypothesis that the target observation point is a result of observing the target object. The second hypothesis is a hypothesis that the target observation point is not a result of observing the target object. The target tracker is an existing tracker to which an observation point is allocated by the observation allocation unit. The target observation point is an observation point allocated to the target tracker. The target target is a target associated with the target tracker.
[0010] According to such a configuration, one observation point is not assigned to multiple trackers, so that the number of combinations of trackers and observation points, i.e., the number of hypotheses, is reduced, and the computational load of the tracking process can be reduced. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a block diagram showing a configuration of a radar system. [Diagram 2] FIG. 1 is an explanatory diagram showing an overview of processing by a PMB filter. [Diagram 3] FIG. 10 is an explanatory diagram showing that the number of hypotheses to be processed increases rapidly when there are a large number of trackers and observation points. [Figure 4] FIG. 2 is an explanatory diagram showing hypotheses to be processed in an FMB filter; [Diagram 5] FIG. 1 is an explanatory diagram showing an overview of processing by an FMB filter. [Figure 6] 4 is a flowchart showing a tracking process in the first embodiment. [Figure 7] FIG. 13 is an explanatory diagram showing how to set a search range. [Figure 8] 11 is a flowchart showing a tracer update process in the first embodiment. [Figure 9]11 is an explanatory diagram illustrating an example of how a state distribution changes due to a tracker update process when a Gaussian distribution is used as the state distribution; FIG. [Figure 10] 11 is a graph showing the results of calculations of the calculation time required for tracking processing by simulation, in comparison with the results obtained by a conventional method. [Figure 11] 11 is a graph showing the results of calculating the operation time for each type of processing, in comparison with the results obtained by a conventional method. [Figure 12] 1 is a graph showing simulation conditions. [Figure 13] 13 is a flowchart showing a tracking process in the second embodiment. [Figure 14] FIG. 13 is an explanatory diagram showing grouping of tracers using a search range. [Figure 15] FIG. 13 is an explanatory diagram showing hypotheses to be considered in trackers with no overlapping search ranges; [Figure 16] 13 is a flowchart showing a tracking process in the third embodiment. [Figure 17] 13 is a flowchart showing a tracer update process in the third embodiment. [Figure 18] 11 is an explanatory diagram illustrating a state distribution that changes due to tracker processing when a mixed Gaussian distribution is used as the state distribution; FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. [1. First embodiment] [1-1. Configuration] The radar system 1 of this embodiment is mounted on a vehicle, and is used to detect various targets existing around the vehicle on which the radar system 1 is mounted.
[0013] The detection area is set, for example, in front of the vehicle. However, the detection area is not limited to the front of the vehicle, and one or more detection areas may be set so that the detection area is in any of the rear, right side, left side, right front, right rear, left front, and left rear of the vehicle. Furthermore, when there are multiple detection areas, the detection areas may be set to be different from each other, or may be set so that parts of the detection areas overlap each other.
[0014] As shown in FIG. 1, the radar system 1 includes a radar sensor 10 and a tracking device 20. The radar sensor 10 uses a LiDAR that irradiates a detection area with laser light and receives the reflected light to generate position information of an observation point that indicates the point where the laser light is reflected. LiDAR is an abbreviation for Light Detection and Ranging. However, the radar sensor 10 is not limited to a LiDAR, and may be a millimeter wave radar, ultrasonic radar, or the like as long as it can obtain position information of an observation point.
[0015] The detection area is set, for example, in front of the vehicle. However, the detection area is not limited to the front of the vehicle, and one or more detection areas may be set so that the detection area is in any of the rear, right side, left side, right front, right rear, left front, and left rear of the vehicle. When there are multiple detection areas, a radar sensor 10 may be provided for each detection area. The detection areas may be set so as not to overlap each other, or so that parts of the detection areas overlap each other. The range of the detection area is defined by a horizontal angle range θh and a vertical angle range θv.
[0016] The radar sensor 10 scans the detection area with an angular resolution Δθh in the horizontal direction and an angular resolution Δθ in the vertical direction. That is, the radar sensor 10 irradiates the detection area with (θh / Δθh)×(θv / Δθv) light beams for each preset processing cycle, and performs sampling a predetermined number of times for each light beam. The sampling interval is set to satisfy a specified distance resolution ΔR. That is, the radar sensor 10 performs sampling a number of times per second that is calculated by dividing the time it takes for light to travel the maximum detection distance Rmax and back by the sampling interval (i.e., 2×Rmax / c / ΔR times).
[0017] Then, peaks having signal strength equal to or greater than the peak threshold are extracted from the waveform represented by the values sampled for each light ray, and position information of the observation point is generated using the sampling timing (i.e., distance) of the peak and the horizontal and vertical angles of the irradiated light ray. The peak threshold is set so as to obtain a desired S / N ratio. The position information does not necessarily need to be represented in a three-dimensional polar coordinate system using distance, horizontal angle, and vertical angle, and may be represented in a three-dimensional Cartesian coordinate system.
[0018] The tracking device 20 includes a microcomputer having a CPU 21 and a semiconductor memory such as a RAM, a ROM, a flash memory, etc. (hereinafter, referred to as a memory 22). The tracking device 20 may include a coprocessor.
[0019] The tracking device 20 executes at least a tracking process by the CPU 21 executing a program stored in the memory 22. In the tracking process, target information is generated, which is information about targets existing within a detection area, based on position information of observation points obtained from the radar sensor 10.
[0020] The target information generated by the tracking process is used for various processes performed for driving assistance, a process for estimating the axis deviation of the radar sensor 10, etc. Driving assistance includes, for example, following distance control (i.e., ACC), lane keeping assist (i.e., LKA), lane change assist (i.e., LCA), lane departure warning, overtaking assist, intervention control, automatic braking, and automatic driving, etc. ACC is an abbreviation for Auto Cruise Control, LKA is an abbreviation for Lane Keeping Assist, and LCA is an abbreviation for Lane Change Assist.
[0021] [1-2. Tracking Overview] This section explains the outline of processing using the FMB filter (hereinafter referred to as FMB), which is a type of RFS tracking used in tracking processing. FMB is a method to speed up processing by improving the existing processing using the PMB filter (hereinafter referred to as PNB). FMB is an abbreviation for Fast Multi-Bernoulli, and PMB is an abbreviation for Poisson Multi-Bernoulli.
[0022] Here, the target position is used as the state of the target to be estimated. However, the state of the target to be estimated is not limited to the position. For ease of understanding, an example will be described in which two observation points are obtained while two targets are being tracked by trackers A and B.
[0023] In RFS tracking, seven hypotheses are considered, which represent the combination of two trackers A and B and two observation points, as shown in Figure 2. That is, hypothesis i is represented as Ki (the observation point where tracker A was observed, and the observation point where tracker B was observed), the two observation points are represented as "observation 1" and "observation 2", and the case where it does not correspond to any observation point is represented as "not yet". The seven hypotheses are K1 (observation 1, observation 2), K2 (not yet, observation 2), K3 (observation 1, not yet), K4 (observation 2, observation 1), K5 (not yet, observation 1), K6 (observation 2, not yet), and K7 (not yet, not yet).
[0024] Then, in the PMB, a state distribution and a hypothesis likelihood are calculated for each of the hypotheses K1 to K7, and the state distributions of all the hypotheses K1 to K7 are convoluted using the calculated hypothesis likelihood to integrate the hypotheses K1 to K7 into one. The state distribution is a distribution that probabilistically represents the state (i.e., the position) of the tracker. For example, a Gaussian distribution is used for the state distribution. The state distribution is calculated by updating the state distribution obtained in the previous processing cycle using an extended Kalman filter (hereinafter, EKF) or the like based on the observation information obtained by observation. However, for each hypothesis, an updated state distribution is generated for the state distribution obtained in the previous processing cycle, assuming that each hypothesis is correct. The hypothesis likelihood is a value that indicates the plausibility of a hypothesis of interest. The hypothesis likelihood is calculated from the proximity of the tracker to the observation point, the error of the sensor used for the observation, the detection probability, the false detection probability, the existence probability of the tracker in the previous processing cycle, and the like.
[0025] Next, if the above process is generalized to the case where n observations are obtained while tracking a target with n trackers, the number of hypotheses Knum to be considered in PMB is expressed by equation (1).
[0026]
number
[0027] For example, when n = 5, the number of hypotheses is 1545, as shown in Figure 3. In LiDAR observations, n is around 300, and due to combinatorial explosion, the number of hypotheses exceeds the amount of computation that can be processed in real time.
[0028] Therefore, FMB is based on the premise that the road obstacles to be tracked are "stationary" and do not move unpredictably, and that while the distance measurement accuracy of LiDAR is extremely high, there will be a non-negligible amount of undetection and false detection.
[0029] Under these assumptions, as shown in Figure 4, the tracker of interest is always assigned to the nearest observation point, and the likelihoods of hypotheses 4 to 6, in which the tracker is assigned to an observation point other than the nearest observation point, are always close to 0.
[0030] Then, as shown in Figure 5, the hypotheses that need to be considered for each tracker are only two patterns: either it was observed at the nearest observation point or it was not observed. In other words, the number of hypotheses to be considered, Knum, is reduced from the number calculated by equation (1) to 2n.
[0031] [1-3. Processing] Next, the tracking process executed by the tracking device 20 will be described with reference to the flowchart of FIG.
[0032] The tracking process is executed by the radar sensor 10 every time it takes to scan the entire detection area (hereinafter, referred to as a processing cycle). When the tracking process is started, in S100, the tracking device 20 acquires observation information representing the observation results from the radar sensor 10. The observation information includes all observation points (hereinafter, observation point group) whose signal strength exceeds a threshold in the detection area.
[0033] In the next S110, the tracking device 20 predicts the state distribution of the existing trackers T1 to Tn in the current processing cycle, assuming that the number of trackers (hereinafter, existing trackers) generated up to the previous processing cycle is n. Furthermore, the tracking device 20 calculates the existence probability of the target associated with the existing tracker Ti for each of the existing trackers T1 to Tn according to the formula (2).
[0034]
number
[0035] r k|k-1 is the probability of the existence of a current tracker after prediction, r k―1|k-1 is the probability of the existence of an existing tracker before prediction, Ps is a constant representing the survival probability of the target, and a value less than 1 but close to 1 is used.
[0036] The state distribution is predicted using the method used in the Extended Kalman Filter (hereinafter, EKF). The state distribution is represented by a Gaussian distribution. In the EKF, the position of the peak in the state distribution of the tracker in the previous processing cycle (i.e., the average value of the state distribution) is moved by the time elapsed for one cycle according to the speed information held by the tracker. Furthermore, in the EKF, the spread of the state distribution (i.e., the variance of the Gaussian distribution) is diffused by the time elapsed for one cycle, thereby obtaining a predicted value of the state distribution (hereinafter, predicted distribution).
[0037] In the next step S120, the tracking device 20 sets a search range for each of the existing trackers T1 to Tn such that the distance from the position indicated by the average value of the predictive distribution is equal to or less than the upper limit distance. The Mahalanobis distance is used as the distance. However, other distances such as the Euclidean distance may also be used.
[0038] In the next step S130, the tracking device 20 exclusively assigns an observation point belonging to the observation point group to each of the existing trackers T1 to Tn. Specifically, among the observation points existing within the search range set for the existing tracker of interest, the observation point closest to the center of the search range is assigned to the existing tracker of interest. This process is also called gating. As a result of gating, a maximum of one observation point is assigned to each existing tracker. Also, each observation point may be assigned to one of the existing trackers or may not be assigned to any of the existing trackers.
[0039] The upper limit distance that defines the search range is, for example, a distance equivalent to 5σ, which is the range outside which the existence probability is almost 0 in a Gaussian distribution, as shown in FIG. 7. However, the upper limit distance may be set to be smaller than 5σ in order to suppress false positives. In other words, the larger the upper limit distance, the lower the probability of non-detection, in which a tracker cannot be detected despite its existence, but the higher the probability of false positives, in which an observation point that has not observed an existing tracker is associated with an existing tracker of interest. Conversely, the smaller the upper limit distance, the higher the probability of non-detection, but the lower the probability of false positives.
[0040] In the next step S140, the tracking device 20 sets an identifier i for identifying the existing tracers T1 to Tn to 1. In the next step S150, the tracking device 20 executes a tracker update process for updating the state distribution of the existing trackers Ti.
[0041] Details of the tracer update process will now be described with reference to the flowchart of Fig. 8. In the tracer update process, the FMB described above is applied. In S310, the tracking device 20 updates the state of the current tracker Ti based on a hypothesis (hereinafter, detection hypothesis) that the observation point assigned to the current tracker Ti has observed the current tracker Ti. Specifically, the state distribution of the current tracker Ti is updated by the EKF, and the existence probability rr of the target associated with the current tracker Ti is calculated. det is updated according to equation (3).
[0042]
number
[0043] In the next S320, the tracking device 20 updates the state of the existing tracker Ti based on the hypothesis that the observation point assigned to the existing tracker Ti has not observed the existing tracker Ti (hereinafter, the non-detection hypothesis). Specifically, in S120, the predicted distribution calculated in S210 is used as the estimated distribution of the existing tracker Ti, and the existence probability rr of the target corresponding to the existing tracker Ti is calculated. mis is updated according to equation (4).
[0044]
number
[0045] Pd represents the detection probability of the radar sensor 10, that is, the probability of detecting an existing target. The detection probability Pd is a constant determined by the performance of the radar sensor 10. Note that there are two cases of non-detection: when a target corresponding to an existing tracer Ti does not exist in the first place, and when a target corresponding to an existing tracer Ti exists but has not been detected by the radar sensor 10. Equation (4) represents the probability of the latter case.
[0046] In the next step S330, the tracking device 20 generates a new tracker based on the non-detection hypothesis. Specifically, the tracking device 20 generates a state distribution of the new tracker with the position of the observation point as the mean of the distribution and the error variance of the radar sensor 10 as the variance of the distribution. Furthermore, the tracking device 20 calculates the existence probability rr of the target associated with the new tracker. new is calculated according to equation (5).
[0047]
number
[0048] D new is the burst rate, which indicates how many observation points are detected per sample for each beam of light irradiated in a fixed direction within the scanning range. C indicates the false detection probability of the radar sensor 10, i.e., the probability of detecting a non-existent target, and is also called the clutter rate. The burst rate D new and the false positive probability C are both constants that are experimentally determined.
[0049] In the next S340, the tracking device 20 calculates a hypothesis likelihood W1 of the detection hypothesis and a hypothesis likelihood W2 of the undetected hypothesis. Specifically, the hypothesis likelihood WD1 before normalization for the detection hypothesis is calculated by equation (6), and the hypothesis likelihood WD2 before normalization for the undetected hypothesis is calculated by equation (7). Furthermore, the hypothesis likelihoods WD1 and WD2 are normalized by equations (8) and (9), so that the hypothesis likelihoods W1 and W2 whose sum is 1 are calculated.
[0050]
number
[0051] In the next S350, the tracking device 20 updates the state distribution by integrating the two hypotheses generated for the existing tracker Ti. Specifically, the state distribution p(i) of the existing tracker Ti is generated according to equation (10), and the existence probability r of the target associated with the existing tracker Ti is calculated according to equation (11).
[0052]
number
[0053] That is, as shown in Fig. 9, the mean μ1, μ2 of two state distributions based on two hypotheses (hereafter, hypothesis distributions) is synthesized (i.e., convolved) with the hypothesis likelihoods W1, W2 to become the mean of the distribution in the updated state distribution p(i). Also, the variances P1, P2 of the two hypothesis distributions are synthesized (i.e., convolved) with the hypothesis likelihoods W1, W2 to become the variance of the distribution in the updated state distribution p(i).
[0054] In the next S360, the tracking device 20 integrates the existence probabilities of the new trackers generated in S330 according to equation (12), and ends the tracker update process.
[0055]
number
[0056] When the tracker update process is completed, the process returns to FIG. 6, and in the subsequent S160, the tracking device 20 increments the identifier i. In the next S170, the tracking device 20 judges whether the identifier i is larger than the number n of existing trackers. If i>n, the tracking device 20 proceeds to S180 assuming that the tracker update process has been performed on all existing trackers, and if i≦n, there are existing trackers for which the tracker update process has not been performed, and returns the process to S150.
[0057] In S180, the tracking device 20 generates a new tracker based on each of the observation points that satisfy the generation condition, with the observation points not being assigned to the existing trackers T1 to Tn in the previous S130. Specifically, the tracking device 20 generates a state distribution of the new tracker, which is represented by a Gaussian distribution with the positions of the observation points as the distribution mean and the error variance of the radar sensor 10 as the distribution variance. Furthermore, the tracking device 20 calculates the existence probability of the target associated with the new tracker according to equation (5).
[0058] In the next S190, the tracking device 20 deletes the trackers whose target existence probability r corresponding to the tracker is equal to or less than the tracker pruning threshold among the existing trackers T1 to Tn inherited from the previous processing cycle and the new trackers generated in the current processing cycle. The tracker pruning threshold is set according to the processing capacity of the tracking device 20 so that the number of trackers propagated to the next processing cycle is within the number that can be processed by the tracking device 20.
[0059] In the next S200, the tracking device 20 extracts tracers from among the remaining tracers that were not deleted in S190, the tracers having a presence probability r of a target corresponding to the tracer that is equal to or greater than a targeting threshold, and recognizes the targets corresponding to each of the extracted tracers as actually existing targets.
[0060] In the next S210, the tracking device 20 determines whether or not there is a target recognized as actually existing in S200 (hereinafter, a recognized target), and if there is a recognized target, the process proceeds to S220, and if there is no recognized target, the process ends.
[0061] In S220, the tracking device 20 generates target information, which is information about the recognized target, using information about the tracer associated with the recognized target, and ends the process. The target information includes at least position information.
[0062] [1-4. Correspondence of terms] In this embodiment, S100 corresponds to the information acquisition unit in this disclosure, S110 corresponds to the state prediction unit in this disclosure, S120 corresponds to the range setting unit in this disclosure, and S130 to S170 correspond to the state update unit in this disclosure. Also, S180 corresponds to the tracker generation unit in this disclosure, S190 corresponds to the tracker pruning unit in this disclosure, and S130 corresponds to the observation allocation unit in this disclosure. Furthermore, S310 to S320 correspond to the hypothesis generation unit in this disclosure, S340 to S350 correspond to the hypothesis integration unit in this disclosure, and S110, S150, and S180 correspond to the judgment value generation unit in this disclosure. Note that the existing tracker to which the observation point is assigned in S130 corresponds to the target tracker in this disclosure, the observation point assigned to the target tracker corresponds to the target observation point in this disclosure, and the target associated with the target tracker corresponds to the target target in this disclosure. The detection hypothesis corresponds to the first hypothesis in this disclosure, and the non-detection hypothesis corresponds to the second hypothesis in this disclosure.
[0063] [1-5. Effects] According to the first embodiment described above in detail, the following effects are achieved. (1a) In the tracking device 20, when assigning observation points to the existing trackers T1 to Tn, the closest observation point is assigned among the observation points existing within the search range set for the existing tracker Ti of interest. In other words, hypotheses regarding observation points whose distance from the existing tracker Ti is so far that the hypothesis likelihood is almost 0 are not generated and are removed before calculation, thereby reducing the calculation load of the tracking process.
[0064] (1b) In the tracking device 20, for each existing tracker Ti, two hypotheses, a detection hypothesis and a non-detection hypothesis, are generated for one observation point assigned to the existing tracker Ti. In other words, the number of hypotheses for the assignment of observation points is 2n, and the number of hypotheses handled in the tracking process can be significantly reduced compared to PMB, in which the number of hypotheses increases exponentially with the number of existing trackers n. In addition, in the tracking process, the calculation load of the hypothesis likelihood that needs to be calculated for each hypothesis is the heaviest, so by reducing the number of hypotheses, the calculation load can be significantly reduced. As a result, a tracking process can be realized that is lower in load than PMB and has higher performance than NN, which does not consider the non-detection hypothesis.
[0065] (1c) In the tracking device 20, the number of hypotheses, and therefore the number of trackers, is halved by integrating two hypotheses generated for each existing tracker Ti by convolution, and further, among the existing trackers and new trackers, trackers whose existence probability is equal to or less than the tracker pruning threshold are deleted. This reduces the number of trackers propagated to the next processing cycle, thereby further reducing the computational load required for tracking processing.
[0066] (1d) In the tracking device 20, the existence probability of the target associated with the existing tracker Ti is updated using the hypothesis likelihood for each processing cycle, thereby improving the performance of distinguishing whether the target associated with the existing tracker Ti is real or not (i.e., it is noise).
[0067] [1-6. Experiment] The calculation times required for tracking processing for each frame are calculated by simulation in the cases of using FMB corresponding to this embodiment, PMB which is a conventional technology that does not perform hypothesis integration, and PHD which is a conventional technology that performs hypothesis integration, and the results are shown in Figures 10 and 11. PHD is an abbreviation for Probability Hypothesis Density.
[0068] As shown in FIG. 12, the simulation conditions were as follows: the threshold level used for extracting the observation point was 2.40σ, the horizontal viewing angle was 19.4°, the vertical viewing angle was 1.3°, and the horizontal and vertical angular resolutions were both 0.1°. In other words, the number of light beams used for scanning was 194 in the horizontal direction and 13 in the vertical direction. The sampling interval of the A / D conversion for each light beam was set to a time equivalent to a distance resolution of 1.9 m, the maximum detection distance was set to 250 m, and the number of samples per light beam was set to 132. In addition, the application range of this process was set to a range of 103.2 m to 131.7 m, the maximum number of samples within the application range was set to 37364, and the expected noise amount was set to 306.
[0069] As shown in Figure 10, when FMB is applied, the calculation time is reduced to about 1 / 10 compared to when PMB is applied. It can also be seen that the calculation time is reduced compared to when PHD is applied. As shown in Figure 11, the calculation time was measured for prediction, gating, hypothesis generation, update, and pruning. It can be seen that the calculation time for hypothesis generation and update is significantly reduced in comparison with PMB. It is also found that the calculation time is almost the same as that of PHD, but the calculation time for pruning is reduced. S110 corresponds to prediction, S120 to gating, S130 to hypothesis generation, S150 to update, and S190 to pruning.
[0070] [2. Second embodiment] [2-1. Differences from the first embodiment] The second embodiment has a basic configuration similar to that of the first embodiment, and therefore differences will be described below. Note that the same reference numerals as those in the first embodiment indicate the same configuration, and the preceding description will be referred to.
[0071] In the first embodiment described above, FMB is applied to all tracers, whereas in the second embodiment, PMB is applied to tracers for which no combinatorial explosion occurs, which is different from the first embodiment.
[0072] [2-2. Processing] Next, a tracking process executed by the tracking device 20 of the second embodiment in place of the tracking process of the first embodiment shown in FIG. 6 will be described with reference to a flowchart of FIG.
[0073] In steps S100 to S120, the tracking device 20 executes the same processes as in the first embodiment. In the next S140, the tracking device 20 initializes to 1 an identifier i used to identify the existing trackers T1 to Tn.
[0074] In the next S142, the tracking device 20 judges whether the search range of the existing tracer Ti overlaps with the search range of other tracers, and if there is an overlap, the process proceeds to S144, and if there is no overlap, the process proceeds to S152. For example, as shown in Fig. 14, when the tracers A to H and the observation points 1 to 18 are located, it is judged that the search ranges of the tracers A, B, D to G overlap, and the search ranges of the tracers C and H do not overlap.
[0075] In S144, the tracking device 20 assigns the nearest observation point existing within the search range of the existing tracker Ti to the existing tracker Ti. For example, in FIG. 13, "viewpoint 1" is assigned to tracker A, "viewpoint 2" to tracker B, "viewpoint 8" to tracker D, "viewpoint 10" to tracker E, "viewpoint 12" to tracker F, and "viewpoint 14" to tracker G.
[0076] In the next S150, the tracking device 20 executes the tracker update process in the same manner as in the first embodiment, and proceeds to S160. In S152, the tracking device 20 assigns all observation points existing within the search range set for the existing tracker Ti to the existing tracker Ti. For example, in FIG. 13, “View 3”, “View 4”, “View 5”, and “View 6” are assigned to the tracker C, and “View 15” and “View 16” are assigned to the tracker H.
[0077] In the following S154, the tracking device 20 executes a tracer update process using the PMB, and advances the process to S160. For a tracker whose search range does not overlap with that of other trackers, the number of hypotheses generated is m+1 if the number of assigned observation points is m, as shown in Figure 15. In other words, there is no combinatorial explosion, and it is possible to perform calculations using the PMB method, which considers all hypotheses.
[0078] The subsequent processes from S160 to S220 are similar to those in the first embodiment. [2-3. Correspondence of terminology] In this embodiment, S142 corresponds to the allocator in this disclosure, S144 corresponds to the observation allocation unit in this disclosure, and S152 to S154 correspond to the precise update unit in this disclosure.
[0079] [2-4. Effects] According to the second embodiment described above in detail, in addition to the effects (1a) to (1d) of the first embodiment described above, the following effect is also obtained.
[0080] (2a) In this embodiment, by partially applying PMB, provided that the search range does not overlap with others, it is possible to improve the accuracy of the tracking process for the tracker to which PMB is applied, without significantly increasing the processing load.
[0081] [3. Third embodiment] [3-1. Differences from the first embodiment] The second embodiment has a basic configuration similar to that of the first embodiment, and therefore differences will be described below. Note that the same reference numerals as those in the first embodiment indicate the same configuration, and the preceding description will be referred to.
[0082] In the above-described first embodiment, a Gaussian distribution is used as the state distribution of the tracker, whereas in the third embodiment, a mixed Gaussian distribution is used instead of the Gaussian distribution, which is different from the first embodiment.
[0083] [3-2. Processing] Next, a tracking process executed by the tracking device 20 of the third embodiment in place of the tracking process of the first embodiment shown in FIG. 6 will be described with reference to a flowchart of FIG.
[0084] In S100, the tracking device 20 executes the same process as in the first embodiment. In the next step S115, in addition to the process of S110 in the first embodiment, the tracking device 20 updates the likelihood of each maximum value included in the state distribution (hereinafter, mixed likelihood) by using equation (13).
[0085]
number
[0086] That is, the mixed likelihood calculated in the above process cycle, i.e., the mixed likelihood before prediction, is used as it is as the mixed likelihood in the predictive distribution. Note that j in equation (13) is an identifier for identifying the maximum values present in the state distribution (i.e., the peaks of the individual Gaussian distributions forming the mixed Gaussian distribution). Here, the number of maximum values present in the state distribution is m.
[0087] In the following steps S120 to S140, the tracking device 20 executes the same processes as in the first embodiment. In the following S155, the tracking device 20 executes a tracker update process using a Gaussian mixture distribution as the state distribution.
[0088] Here, details of the tracer update process will be described with reference to the flowchart of Fig. 17. Regarding the tracer update process, differences from the tracer update process of the first embodiment shown in Fig. 8 will be described. In S310, the tracking device 20 executes the same process as in the first embodiment.
[0089] In the next step S315, the tracking device 20 calculates the likelihood of each maximum value in the state distribution (i.e., the hypothesis distribution) updated with the detection hypothesis, using equations (14) and (15).
[0090]
number
[0091] Equation (14) means that the mixed likelihood of each maximum value is updated according to the distance between the observation point assigned to the current tracker Ti and the maximum value of interest in the predictive distribution of the current tracker Ti. Equation (15) means that each mixed likelihood is normalized so that the sum of the mixed likelihoods of multiple maximum values belonging to the same predictive distribution becomes 1.
[0092] In the next step S320, the tracking device 20 executes the same process as in the first embodiment. In the next S325, if the tracking device 20 adopts the undetected hypothesis, the tracking device 20 uses the prediction distribution as it is as the updated state distribution (i.e., the hypothesis distribution), and calculates the mixed likelihood of each maximum value in the hypothesis distribution by using the formula (16).
[0093]
number
[0094] In other words, the mixed likelihood of the hypothesis distribution based on the undetected hypothesis is also the mixed likelihood of the predictive distribution calculated in S115. In the following steps S330 to S340, the tracking device 20 executes the same processes as in the first embodiment.
[0095] In the next step S352, the tracking device 20 integrates the hypotheses generated for the existing tracker Ti. Specifically, the tracking device 20 generates an updated state distribution p(i) of the existing tracker Ti according to equation (17), and calculates the existence probability r of the target associated with the existing tracker Ti according to equation (18).
[0096]
number
[0097] Equation (17) means that the updated state distribution becomes a distribution obtained by simply adding up m Gaussian distributions that individually represent each maximum value in the hypothesis distribution of the detection hypothesis and m Gaussian distributions that individually represent each maximum value in the hypothesis distribution of the non-detection hypothesis, as shown in Fig. 18. Equation (18) is the same as equation (11).
[0098] In the next step S354, the tracking device 20 calculates the mixed likelihood of each maximum value in the state distribution updated in S350 by using the equations (19) and (20).
[0099]
number
[0100] Equation (19) is the mixed likelihood of the maximum value based on the hypothesis distribution of the detection hypothesis, and equation (20) is the mixed likelihood of each maximum value based on the hypothesis distribution of the non-detection hypothesis. In other words, in this embodiment, every time the tracking update process is performed, the number of maximum values included in the state distribution after the update becomes twice the number of maximum values included in the state distribution before the update.
[0101] In the next S360, the tracking device 20 executes the same process as in the first embodiment, and ends the tracker update process. Returning to FIG. 16, when the tracker update process is completed, in the following steps S160 to S190, the tracking device 20 executes the same processes as in the first embodiment.
[0102] In the next step S195, the tracking device 20 deletes the maximal values whose mixed likelihoods are equal to or smaller than the maximal pruning threshold in the state distribution updated in step S155. Furthermore, for the state distribution from which even one maximal value has been deleted, the tracking device 20 recalculates the mixed likelihoods of the maximal values in the state distribution so that the sum of the mixed likelihoods becomes 1 (i.e., normalized).
[0103] In the following steps S200 to S220, the tracking device 20 executes the same processes as in the first embodiment, and then ends the tracking process. [3-3. Correspondence of terminology] In this embodiment, S354 corresponds to a mixed likelihood calculation unit in this disclosure, and S195 corresponds to a local maximum pruning unit in this disclosure.
[0104] [3-4. Effects] According to the third embodiment described above in detail, in addition to the effects (1a) to (1d) of the first embodiment described above, the following effect is also obtained.
[0105] (3a) In this embodiment, a mixed Gaussian distribution is used as the state distribution instead of a simple Gaussian distribution, so that loss of information due to hypothesis integration can be suppressed and robustness can be improved.
[0106] (3b) In this embodiment, a mixed likelihood is calculated for each maximum value included in the state distribution, and maximum values whose mixed likelihood is equal to or less than the maximum pruning threshold are removed. This prevents the state distribution from becoming unnecessarily complicated, and thus prevents an increase in the computational load due to an increase in the number of maximum values.
[0107] [4. Other embodiments] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and can be implemented in various modified forms.
[0108] (3a) In the above embodiment, the position of the observation point is used as the state to be updated by the tracking process. However, the present disclosure is not limited to the position of the observation point, and any information obtained from the target, such as the speed and acceleration of the observation point, may be used.
[0109] (3b) In the above embodiment, the state of the tracker is updated using a PMB filter, but the state of the tracker may be updated using a filter other than the PMB filter used in the theory of RFS, such as a PHD filter, an LMB filter, a PMBM filter, etc. LMB is an abbreviation for Labeled Multi-Bernoulli, and PMBM is an abbreviation for Poisson Multi-Bernoulli Mixture.
[0110] (3c) In the above embodiment, the presence probability expressed by a value between 0 and 1 is used as the presence determination value used to determine whether or not a target corresponding to a tracker exists, but the present disclosure is not limited to the presence probability. For example, cardinal information that allows a value of 1 or more may be used as the presence determination value.
[0111] (3d) The tracking device 20 and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor and a memory programmed to execute one or more functions embodied in a computer program. Alternatively, the tracking device 20 and the method described in the present disclosure may be realized by a dedicated computer provided by configuring a processor with one or more dedicated hardware logic circuits. Alternatively, the tracking device 20 and the method described in the present disclosure may be realized by one or more dedicated computers configured by a combination of a processor and a memory programmed to execute one or more functions and a processor configured with one or more hardware logic circuits. In addition, the computer program may be stored in a computer-readable non-transitory tangible recording medium as instructions executed by a computer. The method for realizing the functions of each unit included in the tracking device 20 does not necessarily need to include software, and all of the functions may be realized using one or more hardware.
[0112] (3e) Multiple functions possessed by one component in the above embodiments may be realized by multiple components, or one function possessed by one component may be realized by multiple components. Also, multiple functions possessed by multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Also, part of the configuration of the above embodiments may be omitted. Also, at least part of the configuration of the above embodiments may be added to or substituted for the configuration of another of the above embodiments.
[0113] (3f) In addition to the tracking device 20 described above, the present disclosure can be realized in various forms, such as a system including the tracking device 20 as a component, a program for causing a computer to function as the tracking device 20, a non-transient physical recording medium such as a semiconductor memory on which the program is recorded, and a target tracking method. [Explanation of symbols]
[0114] 1... radar system, 10... radar sensor, 20... tracking device, 21... CPU, 22... memory.
Claims
1. An information acquisition unit (20: S100) configured to acquire observation information from a sensor that observes the state of a target existing around the vehicle for each processing cycle; a tracer generation unit (20: S180) configured to generate a tracer having a state distribution probabilistically indicating a state of the observation information for the observation points that satisfy a preset generation condition among a group of observation points that is a plurality of observation points from which the observation information is acquired by the information acquisition unit; a state prediction unit (20: S110) configured to generate, for each of the existing tracers, which are the tracers generated up to the previous processing cycle, a predictive distribution, which is the state distribution obtained by predicting the state of the observation information of the existing tracer in the current processing cycle from the state distribution of the existing tracer; a state update unit (20: S130 to S170) configured to update the state distribution of each of the existing trackers by using the predicted distribution generated by the state prediction unit and the observation information of the observation point group obtained in the current processing cycle; Equipped with The state update unit is an observation allocation unit (20: S130, S144) configured to exclusively allocate the observation points to each of the existing trackers according to a distance between the observation information acquired by the information acquisition unit and the observation information indicated by the predictive distribution; a hypothesis generation unit (20: S310 to S320) configured to generate, for each of hypotheses belonging to a hypothesis group including a first hypothesis that the target observation point is a result of observing the target object and a second hypothesis that the target observation point is not a result of observing the target object, a hypothesis likelihood that is the likelihood of the hypothesis and a hypothesis distribution that is the state distribution updated assuming that the hypothesis is correct; Equipped with the target tracker is the existing tracker to which the observation point is assigned by the observation assignment unit, the first hypothesis is a hypothesis that the target observation point is a result of observing the target object, the second hypothesis is a hypothesis that the target observation point is not a result of observing the target object, the target observation point is the observation point assigned to the target tracker, and the target object is a target associated with the target tracker; Tracking device.
2. 2. The tracking device of claim 1, The state update unit is The method further includes a hypothesis integration unit (20: S340 to S350) configured to generate the updated state distribution by integrating the hypothesis distributions generated for each hypothesis belonging to the hypothesis group by the hypothesis generation unit through convolution using the hypothesis likelihood. Tracking device.
3. The tracking device according to claim 1 or 2, the observation assigner is configured to assign at most one of the observation points to each of the existing trackers; Tracking device.
4. A tracking device according to any one of claims 1 to 3, a judgment value generating unit (20: S110, S150, S180) configured to calculate, for each of the tracers, an existence judgment value used to judge whether or not a target corresponding to the tracer exists; A tracer pruning unit (20: S190) configured to remove the tracer whose existence determination value is equal to or less than a pruning threshold; The tracking device further comprises:
5. A tracking device according to any one of claims 1 to 4, The observation allocation unit further includes a range setting unit (20: S120) configured to set a search range for each of the existing tracers; The observation allocation unit is configured to allocate the observation points that exist within the search range. Tracking device.
6. 6. The tracking device of claim 5, A precise update unit (20: S152 to S154) configured to update the state of the tracker using all of the observation points present within the search range; a distribution unit (20: S142) configured to distribute the trackers such that the state update unit is applied to the trackers whose search ranges overlap with other trackers, and the precise update unit is applied to the trackers whose search ranges do not overlap with other trackers; Further comprising Tracking device.
7. A tracking device according to any one of claims 1 to 6, The state distribution is represented by a Gaussian mixture distribution. Tracking device.
8. 8. The tracking device of claim 7, a mixed likelihood calculation unit (20: S354) configured to calculate a mixed likelihood, which is the likelihood of a Gaussian distribution, for each of a plurality of Gaussian distributions that are components of the mixed Gaussian distribution representing the state distribution; A maximum pruning unit (20: S195) that removes the Gaussian distribution whose mixed likelihood is equal to or smaller than a maximum pruning value from the state distribution; The tracking device further comprises:
Citation Information
Patent Citations
Dead corner area estimating device and program
JP2011123551A
Mobile object position estimation device, mobile object position estimation method and mobile object position estimation program
JP2013072858A
State estimation device, program and integrated circuit
JP2016114988A
Data-driven delta-generalized labeled multi-bernoulli tracker
US20190035088A1
Tag sensor system and sensor device, and object position estimating device and object position estimating method
WO2009113265A1
