Target tracking device and target tracking method

Abstract

The target point cloud data detection unit (402) includes a target point cloud data detection unit (2) that detects point cloud data of a target, a feature detection unit (3) that detects observed features, which are features of the target's surface, based on the point cloud data detected by the target point cloud data detection unit (2), a feature prediction unit (401) that predicts features at a current time as predicted features from observed features at past times, a feature correlation unit (402) that generates a hypothesis indicating a correlation between the observed features detected by the feature detection unit (3) and the predicted features predicted by the feature prediction unit (401), a feature filter unit (403) that filters the observed features detected by the feature detection unit (3) based on the hypothesis generated by the feature correlation unit (402), a sub-optimization unit (404) that integrates similar tracks based on the observed features filtered by the feature filter unit (403), and a track establishment unit (405) that establishes tracks based on the tracks integrated by the sub-optimization unit (404).

Description

[Technical field]

[0001] The present disclosure relates to a target tracking device and a target tracking method for tracking a target. [Background technology]

[0002] Target monitoring is required for the purpose of preventive maintenance against critical target situations and for the optimization of associated guidance. As a means for continuously monitoring a target, there is a technique for detecting and tracking the target using a non-contact sensor such as a camera, radar or laser (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 171498 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when targets obtained as a group of multiple observation information by a sensor are in a crowded and cluttered environment, it is difficult to accurately track individual targets and obtain their exact positions.

[0005] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a target tracking device that is capable of tracking individual targets with high precision and obtaining the exact position of the targets. [Means for solving the problem]

[0006] The target tracking device according to the present disclosure is characterized in that it includes a target point cloud data detection unit that detects point cloud data of the target, a feature detection unit that detects observed features, which are features of the target's surface, based on the point cloud data detected by the target point cloud data detection unit, a feature prediction unit that predicts features at the current time as predicted features from observed features at past times, a feature correlation unit that generates a hypothesis indicating a correlation between the observed features detected by the feature detection unit and the predicted features predicted by the feature prediction unit, a feature filter unit that filters the observed features detected by the feature detection unit based on the hypothesis generated by the feature correlation unit, a semi-optimization unit that integrates similar tracks based on the observed features after filtering by the feature filter unit, and a track establishment unit that establishes tracks based on the tracks after integration by the semi-optimization unit. Effect of the Invention

[0007] According to the present disclosure, with the above-mentioned configuration, it is possible to accurately track individual targets and obtain the exact positions of the targets. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating an example of the configuration of a target tracking device according to a first embodiment. [Diagram 2] 4 is a diagram illustrating an example of the configuration of a feature amount correlation unit in the first embodiment. FIG. [Diagram 3] 5 is a flowchart showing an example of the operation of the target tracking device according to the first embodiment. [Figure 4] 4 is a diagram illustrating an example of the operation of a feature amount prediction unit in the first embodiment. FIG. [Diagram 5] 5 is a diagram illustrating an example of the operation of a feature amount correlation unit in the first embodiment. FIG. [Figure 6] 5 is a diagram illustrating an example of the operation of a feature amount correlation unit in the first embodiment. FIG. [Figure 7] 4 is a diagram illustrating an example of the operation of a sub-optimization unit in the first embodiment. FIG. [Figure 8]8A and 8B are diagrams illustrating an example of a hardware configuration of a target tracking device according to the first embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the drawings. Embodiment 1 FIG. 1 is a diagram showing an example of the configuration of a target tracking device according to the first embodiment. The target tracking device is a device that tracks a target. As shown in Fig. 1, the target tracking device includes a sensor data acquisition unit 1, a target point cloud data detection unit 2, a feature detection unit 3, a target tracking unit 4, and a display unit 5.

[0010] The sensor data acquisition unit 1 is a sensor that acquires sensor data. The sensor data is point cloud data for each frame. An example of the sensor data acquisition unit 1 is a LIDAR.

[0011] The target point cloud data detection unit 2 detects target point cloud data from the sensor data acquired by the sensor data acquisition unit 1 based on the sensor data.

[0012] The feature amount detection unit 3 detects the feature amount of the target surface as an observed feature amount by clustering the point cloud data based on the point cloud data detected by the target point cloud data detection unit 2. At this time, the feature amount detection unit 3 estimates the surface of the object by clustering the point cloud data, and detects the feature amount of the surface. The feature amount of the surface may include the position (including the position on the pixel) and size of the surface, or the positions (including the positions on the pixel) of the vertices that constitute the surface. In addition to the above, the feature amount of the surface may also include the orientation of the surface.

[0013] The target tracking unit 4 tracks the target based on the observed feature detected by the feature detection unit 3. At this time, the target tracking unit 4 tracks the target by updating the track based on the correlation between the track (predicted feature based on the track) and the observed feature. Note that the track is data in which the observed feature at past times (past frames), the state of the target, and quality information are arranged in chronological order. An example of the configuration of this target tracking unit 4 will be described later.

[0014] The processing by the target tracking unit 4 may be processing that holds multiple (correlation) hypotheses based on MHT (Multiple Hypothesis Tracking), or processing that holds a single correlation such as GNN (Global Nearest Neighbor).

[0015] The display unit 5 is a display that displays information indicating the flight path obtained by the target tracking unit 4. The display unit 5 may be a higher-level device that performs fusion and filtering with observation information from other sensors before displaying the information. In this case, the display unit 5 may display the information indicating the above-mentioned track, and the track score or quality information for each surface.

[0016] 1 shows a case where the sensor data acquisition unit 1 and the display unit 5 are provided inside the target tracking device. However, this is not limiting, and the sensor data acquisition unit 1 and the display unit 5 may be provided outside the target tracking device.

[0017] Next, a configuration example of the target tracking unit 4 will be described. As shown in FIG. 1, the target tracking unit 4 includes a feature prediction unit 401, a feature correlation unit 402, a feature filter unit 403, a semi-optimization unit 404, and a track establishment unit 405.

[0018] The feature amount prediction unit 401 predicts a feature amount at the current time (current frame) based on the track established by the track establishment unit 405. Note that the feature amount predicted by the feature amount prediction unit 401 is also called a predicted feature amount.

[0019] The feature correlation unit 402 generates a hypothesis indicating a correspondence relationship between the observed feature and the predicted feature, based on the observed feature detected by the feature detection unit 3 and the predicted feature predicted by the feature prediction unit 401 .

[0020] At this time, for example, the feature amount correlation unit 402 may generate a plurality of hypotheses indicating the corresponding relationships for each face. Also, for example, the feature amount correlating unit 402 may generate one hypothesis indicating a plausible correspondence between the faces. Also, for example, the feature amount correlating unit 402 may determine a plurality of hypotheses including a hypothesis indicating a corresponding relationship for each face. Also, for example, the feature correlation unit 402 may generate multiple hypotheses in which the object correspondence is the same but the surface correspondence is different, and generate a hypothesis that integrates the multiple generated hypotheses by weighting them by reliability.

[0021] Also, for example, the feature amount correlating unit 402 may generate hypotheses by performing object correlation and then surface correlation. In this case, for example, the feature amount correlation unit 402 has a first correlation unit and a second correlation unit as shown in FIG. The first correlation unit extracts candidates for object correspondence relationships based on the observed feature amounts detected by the feature amount detection unit 3 and the predicted feature amounts predicted by the feature amount prediction unit 401 . The second correlation unit generates hypotheses by extracting correspondences between faces based on the candidates extracted by the first correlation unit.

[0022] The feature amount filter unit 403 filters the observed feature amount detected by the feature amount detector 3 based on the hypothesis generated by the feature amount correlator 402 . At this time, the feature amount filtering unit 403 performs filtering based on the quality information for each surface. The filter used by the feature amount filtering unit 403 may be a simple filter such as an alpha-beta filter, or a time series filter based on statistical estimation such as a Kalman filter or a particle filter.

[0023] The sub-optimization unit 404 integrates similar tracks based on the observed features after filtering by the feature filter unit 403 . At this time, for example, the sub-optimization unit 404 may combine tracks that have similar positions, sizes, and speeds based on the positions and sizes of targets included in the tracks, as well as speeds based on the positions. Furthermore, for example, the sub-optimization unit 404 may integrate tracks having similar observation features based on the observation features after filtering by the feature filter unit 403 and the number of frames in which the observation features correlate as a common track in the time series direction.

[0024] The track establishment unit 405 establishes tracks based on the reliability based on the quality information included in the tracks after integration by the sub-optimization unit 404. That is, the track establishment unit 405 discards tracks after integration by the sub-optimization unit 404 whose reliability based on the quality information included in the tracks is equal to or less than a threshold. The threshold can be set as appropriate.

[0025] Next, an example of the operation of the target tracking device according to the first embodiment shown in FIG. 1 will be described with reference to FIG. In an operation example of the target tracking device according to the first embodiment shown in FIG. 1, as shown in FIG. 3, the target tracking device first detects an observed feature amount (step ST101, feature amount detection process). That is, first, the sensor data acquisition unit 1 acquires sensor data which is point cloud data for each frame. Then, the target point cloud data detection unit 2 detects point cloud data of the target from the sensor data acquired by the sensor data acquisition unit 1 based on the sensor data. Then, the feature amount detection unit 3 detects the feature amount of the target surface as an observed feature amount by clustering the point cloud data based on the point cloud data detected by the target point cloud data detection unit 2. At this time, the feature amount detection unit 3 estimates the surface of the object by clustering the point cloud data, and detects the feature amount of the surface. The feature amount of the surface may include the position (including the position on the pixel) and size of the surface, or the positions (including the positions on the pixel) of the vertices that constitute the surface. In addition to the above, the feature amount of the surface may also include the orientation of the surface.

[0026] In this manner, in the feature amount detection process, the feature amount detection unit 3 detects the feature amount of the face, which is the area of ​​the surface portion of the object, for each frame. Specifically, the feature detection unit 3 determines the main surface with high quality among the estimated surfaces as the main surface, and the other surfaces as sub-surfaces. The main surface is the surface with the largest cross-sectional area as viewed from the reference position of the sensor. The sub-surface is discarded if its quality is not high. A sub-surface whose quality is not high refers to a case where the angle between the vector connecting the reference position of the sensor and the center of the surface and the vector connecting the vertices of the surface when viewed from a two-dimensional bird's-eye view of the sub-surface is smaller than a threshold value. The threshold value can be set appropriately. The feature detection unit 3 detects the feature values ​​of the main surface and the sub-surface, or the feature value of the main surface.

[0027] Next, the feature amount prediction unit 401 predicts the feature amount at the current time (current frame) based on the track established by the track establishment unit 405 (step ST102, feature amount prediction process).

[0028] Examples of models used in prediction by the feature prediction unit 401 include a constant change model in which the position, size, or tilt changes uniformly, a fixed model in which the position, size, or tilt is fixed, or a turning model or a lane-following movement prediction model in which the specific trajectory of the target is known.

[0029] Here, the prediction of the feature amount by the feature amount predicting unit 401 immediately after the start of tracking will be described with reference to FIG. In this figure, O j krepresents the feature amount of the jth (j = 1, , P) surface at time k. Note that when the target tracking device tracks an object that can be approximated as a rectangular parallelepiped, such as a vehicle, P = 2.

[0030] As shown in Fig. 4, immediately after the start of tracking, there are few past tracks to use for prediction. 1 k-2 ,O 2 k-2 A parallelogram is created by folding back the vector x, and its center position x is set as the temporary center position of the target, and the feature amount at time k is predicted. Here, specific examples of the state of the target include, when the target tracking device tracks an object that can be approximated as a rectangular parallelepiped such as a vehicle, the central position X of the target, the central position Y, the length W of the main surface, the length L perpendicular to the main surface, and the inclination θ (the angle with the X-axis when the target is viewed on a two-dimensional overhead coordinate system (X, Y axes)).

[0031] Next, the feature correlation unit 402 generates a hypothesis indicating a correspondence between the observed features and the predicted features based on the observed features detected by the feature detection unit 3 and the predicted features predicted by the feature prediction unit 401 (step ST103, feature correlation processing).

[0032] At this time, for example, the feature amount correlation unit 402 may generate a plurality of hypotheses indicating the corresponding relationships for each face. Also, for example, the feature amount correlating unit 402 may generate one hypothesis indicating a plausible correspondence between the faces. Also, for example, the feature amount correlating unit 402 may determine a plurality of hypotheses including a hypothesis indicating a corresponding relationship for each face. Also, for example, the feature correlation unit 402 may generate multiple hypotheses in which the object correspondence is the same but the surface correspondence is different, and generate a hypothesis that integrates the multiple generated hypotheses by weighting them by reliability. Also, for example, the feature amount correlating unit 402 may generate hypotheses by performing object correlation and then surface correlation.

[0033] For example, the feature correlation unit 402 generates a hypothesis indicating a likely correspondence between objects based on the proximity between the center position of the predicted feature and the center position of the observed feature. Then, the feature correlation unit 402 generates a hypothesis in which the correspondence between faces is changed from the generated hypothesis. Then, the feature correlation unit 402 may hold these generated hypotheses. Alternatively, the feature correlation unit 402 may extract one hypothesis with the maximum likelihood from the generated hypotheses and hold the one hypothesis. In this case, for example, the feature amount correlation unit 402 may generate hypotheses by listing all combinations of objects and faces, and then narrowing them down to the top C. Alternatively, for example, the feature amount correlation unit 402 may generate hypotheses by listing D hypotheses indicating object correspondences with provisionally high reliability based on the likelihood calculated from the center positions of the feature amounts, and then changing the correspondences of faces to add detailed hypotheses.

[0034] A specific example of the operation of the feature amount correlation unit 402 will be described with reference to FIG. The concept of multiple target tracking is shown in Fig. 5. In Fig. 5, three tracks TRK1 to TRK3 and two observation feature amounts OBS1 and OBS2 at the current time are shown. Then, when the feature correlation unit 402 considers combinations of these tracks (predicted features based on them) and observed features as hypotheses, it may consider, for example, a first hypothesis that TRK1 and OBS1 correspond to each other and TRK3 and OBS2 correspond to each other, and a second hypothesis that TRK1 and OBS1 correspond to each other and TRK2 and OBS2 correspond to each other.

[0035] 5, the feature amount correlation unit 402 generates hypothesis A from the first hypothesis, and also generates hypothesis C, which is a hypothesis obtained by changing the correspondence between faces with respect to hypothesis A. The feature amount correlation unit 402 also generates hypothesis B from the second hypothesis. The target tracking device may then perform multiple hypothesis tracking using these hypotheses A, B, and C. Alternatively, the feature amount correlation unit 402 may select only the correspondence relationship of the surface having a high reliability, which is the accumulation of the likelihood, from among the hypotheses A and C. In this case, the target tracking device may perform multiple hypothesis tracking using the hypotheses A and B, or the hypotheses C and B. Alternatively, the feature correlation unit 402 may generate a hypothesis A' by weighting the hypotheses A and C by their reliability and integrating them. In this case, the target tracking device may perform multiple hypothesis tracking using the hypotheses A', B, and C.

[0036] Next, the surface correlation process performed by the feature amount correlating unit 402 will be described with reference to FIG. As shown in Fig. 6, the feature correlation unit 402 calculates a track score from the similarity between the predicted feature and the observed feature to determine whether the principal surface at the past time coincides with the principal surface at the current time, and uses the track score as an index for generating one surface correlation or for ranking the surface correlations. k i indicates the observed feature amount, vic1 to vic4 indicate the target states (position, size, and orientation) after filtering, and vicP1 to vicP4 indicate the predicted target states (position, size, and orientation). In Figure 6, vicP1 and O k 1 and vicP4 and O k 2 The surface correlation pattern (first correlation pattern) shows that vicP1 and O k 2 and correspond and vicP2 and Ok 1 In response to this, the feature amount correlating unit 402 determines the first correlation pattern having a higher surface similarity as the first surface correlation pattern.

[0037] The track score is expressed, for example, by the following formulas (1) to (3). Score1 in formula (1) represents the track score when the principal surface of the current time coincides with the principal surface of the past time. Score2 in formula (2) represents the track score when the principal surface of the current time correlates with the adjacent surface to the right of the principal surface of the past time. Score3 in formula (3) represents the track score when the principal surface of the current time correlates with the adjacent surface to the left of the principal surface of the past time. In addition, in formulas (1) to (3), vici_pos_dif represents the difference in the center position of the surface between the predicted feature and the observed feature to be compared, vici_ang_dif represents the difference in the slope of the surface between the predicted feature and the observed feature to be compared, and vici_len_dif represents the difference in the surface length between the predicted feature and the observed feature to be compared. Furthermore, Spos represents a fixed parameter for covariance or normalization of the difference in the center position of the surface between the predicted feature and the observed feature to be compared, Sang represents a fixed parameter for covariance or normalization of the difference in the slope of the surface between the predicted feature and the observed feature to be compared, and Slen represents a fixed parameter for covariance or normalization of the difference in the surface length between the predicted feature and the observed feature to be compared. Then, the feature amount correlation section 402 may adopt one hypothesis with the smallest track score, or may adopt multiple hypotheses in ascending order of track score. TIFF0007634803000001.tif53166

[0038] As for the quality of each surface, in the case of Fig. 6, the surface of vicP1 has a correlation with the surface of the observed feature for all three times in the past, but the surface of vicP4 has not had a correlation with the surface of the observed feature for even one time in the past. Then, the feature filter unit 403 in the latter stage performs filtering for each surface based on such correlation count (Q) or track score.

[0039] Next, the feature amount filter unit 403 filters the observed feature amount detected by the feature amount detector 3 based on the hypothesis generated by the feature amount correlator 402 (step ST104, feature amount filtering process). At this time, the feature quantity filtering unit 403 performs filtering based on the quality information for each surface. Note that as the filter used by the feature quantity filtering unit 403, a simple filter such as an alpha-beta filter may be used, or a time-series filter based on statistical estimation such as a Kalman filter or a particle filter may be used.

[0040] Specifically, for example, the feature quantity filter unit 403 may perform filtering using a Kalman filter with respect to the central position, size, or inclination among the feature quantities. Also, for example, the feature quantity filter unit 403 may perform filtering using a Kalman filter with respect to the central position among the feature quantities and using an alpha-beta filter with respect to the size or inclination among the feature quantities.

[0041] Here, the filter formula for the size in the case of using an alpha-beta filter is represented by the following formulas (4) and (5). In formula (4), Wx represents the length of the main surface after filtering, Wp represents the length of the main surface indicated by the predicted feature quantity, Wo represents the length of the main surface indicated by the observed feature quantity, and α represents a parameter. Also, in formula (5), Lx represents the length of the sub-surface after filtering, Lp represents the length of the sub-surface indicated by the predicted feature quantity, and Lo represents the length of the sub-surface indicated by the observed feature quantity. TIFF0007634803000002.tif22166

[0042] Then, the target tracking device integrates the predicted feature quantity of the surface and the observed feature quantity of the surface by weighting with α according to formulas (4) and (5), and estimates the accurate size of the object. Here, α is changed according to the number of times of the past correlated surfaces or the track score. For example, α is basically set to a fixed value (for example, α = 5), and for a surface with insufficient correlation (when α < Q), α = Q may be set, etc., and the weight may be changed according to the number of correlation times with the main surface and the sub-surface. In this way, the feature amount filter unit 403 may perform filtering by weighting with an alpha-beta filter based on the correlation count or the track score for each surface.

[0043] When there is no observed feature to be filtered, the feature filter unit 403 sets the predicted feature as the feature after filtering. The feature filter unit 403 acquires quality information as the track in addition to the feature after filtering and the state of the target. The quality information includes information indicating at least one of the correlation function and the track score. The quality information may also include information indicating the error covariance, the proximity track information, and the presence or absence of a memory track. The error covariance is the error covariance for each surface calculated based on the correlation function by a Kalman filter or an alpha-beta filter. The memory track indicates the track in the absence of correlation.

[0044] Next, the semi-optimization unit 404 integrates similar tracks based on the observed feature values ​​filtered by the feature value filter unit 403 (step ST105, semi-optimization process). At this time, the semi-optimization unit 404 optimizes the tracks by performing N scan pruning or N scan limit processing. At this time, for example, the sub-optimization unit 404 may combine tracks that have similar positions, sizes, and velocities based on the positions and sizes of targets included in the tracks, as well as velocities based on the positions. Furthermore, for example, the sub-optimization unit 404 may integrate tracks having similar observation features based on the observation features after filtering by the feature filter unit 403 and the number of frames in which the observation features correlate as a common track in the time series direction.

[0045] In addition to the function of integrating tracks that have the same observation features constituting the most recent N tracks, as in the conventional N scan pruning, this semi-optimization unit 404 may also have an added function of integrating tracks taking into account the size of the volume (rigid body). This additional function takes into account the feature quantities of multiple surfaces for one target based on the feature quantities of the surface of the volume target. In this function, for example, as shown in FIG. 7, for a plurality of tracks generated within the distance of the assumed target size, even if the M (M < N) observed feature quantities out of the N most recent ones are not the same, for tracks with close positions and similar speeds or directions of speed, track integration is performed.

[0046] Next, the track establishment unit 405 establishes a track based on the reliability based on the quality information included in the track, from the integrated track by the sub-optimization unit 404 (step ST106, track establishment process). That is, the track establishment unit 405 rejects a track in which the reliability based on the quality information included in the track is below the threshold among the integrated tracks by the sub-optimization unit 404.

[0047] After that, the display unit 5 displays information indicating the track obtained by the target tracking unit 4. Also, the information indicating the above track, and the track score or quality information for each surface may be displayed.

[0048] As described above, in the target tracking device according to Embodiment 1, the feature quantity of the surface is detected as the observed feature quantity from the point cloud data of the target, and the tracking process is performed using the observed feature quantity. Thereby, in the target tracking device according to Embodiment 1, low-load and high-speed calculation are possible compared with the prior art. In addition, in the target tracking device according to Embodiment 1, by managing the tracking quality for each surface, it is possible to perform accurate correlation processing and filtering processing for each surface. Therefore, improvement in tracking performance such as higher accuracy of target position estimation and reduction of tracking handover is possible.

[0049] In addition, in the target tracking device according to Embodiment 1, even when a single lidar is used as the sensor data acquisition unit 1, by taking surface correlation, if multiple surfaces can be observed in time series due to movement, the position and size of the target can be accurately estimated. Moreover, the target tracking device according to the first embodiment allows dynamic and accurate merging with other sensors by taking into account the observation quality for each surface.

[0050] As described above, according to the first embodiment, the target tracking device includes the target point cloud data detection unit 2 that detects point cloud data of the target, the feature detection unit 3 that detects an observation feature, which is a feature of the surface of the target, based on the point cloud data detected by the target point cloud data detection unit 2, the feature prediction unit 401 that predicts a feature at the current time as a predicted feature from the observation feature at the past time, the feature correlation unit 402 that generates a hypothesis indicating a correlation between the observation feature detected by the feature detection unit 3 and the predicted feature predicted by the feature prediction unit 401, the feature filter unit 403 that filters the observation feature detected by the feature detection unit 3 based on the hypothesis generated by the feature correlation unit 402, the semi-optimization unit 404 that integrates similar tracks based on the observation feature filtered by the feature filter unit 403, and the track establishment unit 405 that establishes tracks based on the tracks integrated by the semi-optimization unit 404. As a result, the target tracking device according to the first embodiment can accurately track individual targets and obtain the accurate position of the targets. That is, the target tracking device according to embodiment 1 is capable of tracking based on the surface features of the target obtained as point cloud data and their quality information, so that even when the target is in a crowded, multi-target, cluttered environment, it is possible to track individual targets with high accuracy, and also to obtain the exact position of the target from, for example, its center position or surface position.

[0051] Finally, with reference to FIG. 8, an example of the hardware configuration of the target tracking device according to the first embodiment will be described. The functions of the target point cloud data detection unit 2, the feature detection unit 3, and the target tracking unit 4 in the target tracking device are realized by a processing circuit 51. The processing circuit 51 may be dedicated hardware as shown in Fig. 8A, or may be a CPU (also called a Central Processing Unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a processor, or a DSP (Digital Signal Processor)) 52 that executes a program stored in a memory 53 as shown in Fig. 8B.

[0052] When the processing circuit 51 is a dedicated hardware, the processing circuit 51 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. The functions of each of the target point cloud data detection unit 2, the feature detection unit 3, and the target tracking unit 4 may be realized by the processing circuit 51, or the functions of each unit may be realized by the processing circuit 51 collectively.

[0053] When the processing circuit 51 is a CPU 52, the functions of the target point cloud data detection unit 2, the feature amount detection unit 3, and the target tracking unit 4 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 53. The processing circuit 51 realizes the functions of each unit by reading and executing the programs stored in the memory 53. That is, the target tracking device includes a memory 53 for storing a program that, when executed by the processing circuit 51, results in the execution of each step shown in FIG. 3. It can also be said that these programs cause a computer to execute the procedures and methods of the target point cloud data detection unit 2, the feature amount detection unit 3, and the target tracking unit 4. Here, examples of the memory 53 include non-volatile or volatile semiconductor memories such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable ROM), and EEPROM (Electrically EPROM), magnetic disks, flexible disks, optical disks, compact disks, mini disks, and DVDs (Digital Versatile Discs).

[0054] It should be noted that the functions of the target point cloud data detection unit 2, the feature detection unit 3, and the target tracking unit 4 may be partially realized by dedicated hardware and partially realized by software or firmware. For example, the target point cloud data detection unit 2 may be realized by a processing circuit 51 as dedicated hardware, and the feature detection unit 3 and the target tracking unit 4 may be realized by the processing circuit 51 reading and executing a program stored in a memory 53.

[0055] Thus, the processing circuitry 51 can realize each of the above-mentioned functions by hardware, software, firmware, or a combination of these.

[0056] Any of the components of the embodiments may be modified or omitted. [Industrial Applicability]

[0057] The target tracking device according to the present disclosure is capable of tracking individual targets with high precision and obtaining the exact position of the target, and is suitable for use as a target tracking device that tracks targets. [Explanation of symbols]

[0058] 1 sensor data acquisition unit, 2 target point cloud data detection unit, 3 feature detection unit, 4 target tracking unit, 5 display unit, 51 processing circuit, 52 CPU, 53 memory, 401 feature prediction unit, 402 feature correlation unit, 403 feature filter unit, 404 semi-optimization unit, 405 track establishment unit.

Claims

1. A target point cloud data detection unit that detects target point cloud data; a feature amount detection unit that detects an observation feature amount, which is a feature amount of a surface of a target, based on the point cloud data detected by the target point cloud data detection unit; a feature prediction unit that predicts a feature at a current time from an observed feature at a past time as a predicted feature; a feature correlation unit that generates a hypothesis indicating a correspondence relationship between the observed feature detected by the feature detection unit and the predicted feature predicted by the feature prediction unit; a feature amount filter unit that filters the observed feature amount detected by the feature amount detector based on the hypothesis generated by the feature amount correlation unit; a sub-optimization unit that integrates similar trajectories based on the observation feature values ​​filtered by the feature value filter unit; a track establishment unit that establishes a track based on the track integrated by the sub-optimization unit; A target tracking device comprising:

2. The feature correlation unit generates a plurality of hypotheses indicating corresponding relationships for each face.

2. The target tracking device according to claim 1.

3. The feature correlation unit generates a hypothesis indicating a plausible correspondence between the faces.

2. The target tracking device according to claim 1.

4. The feature correlation unit determines a plurality of hypotheses including a hypothesis indicating a correspondence relationship for each face.

2. The target tracking device according to claim 1.

5. The feature amount correlation unit generates a plurality of hypotheses in which the object correspondence is the same but the surface correspondence is different, and generates a hypothesis by weighting the generated plurality of hypotheses by reliability.

2. The target tracking device according to claim 1.

6. The feature amount correlation unit generates hypotheses by performing object correlation and then surface correlation.

2. The target tracking device according to claim 1.

7. The track contains the target's position and size in time series, The sub-optimization unit integrates tracks having similar positions, sizes, and velocities based on the positions and sizes of targets included in the tracks, and the velocities based on the positions.

2. The target tracking device according to claim 1.

8. The sub-optimization unit integrates tracks having similar observation features based on the observation features filtered by the feature filter unit and the number of frames in which the observation features correlate as a common track in a time series direction.

2. The target tracking device according to claim 1.

9. The feature amount filter unit performs filtering based on quality information for each surface.

2. The target tracking device according to claim 1.

10. The feature amount filter unit performs filtering by weighting with an alpha-beta filter based on the number of correlations for each surface or the track score.

2. The target tracking device according to claim 1.

11. A step of detecting point cloud data of a target by a target point cloud data detection unit; A feature detection unit detects an observation feature, which is a feature of a surface of a target, based on the point cloud data detected by the target point cloud data detection unit; A feature prediction unit predicts a feature at a current time from an observed feature at a past time as a predicted feature; a feature correlation unit generating a hypothesis indicating a correlation between the observed feature detected by the feature detection unit and the predicted feature predicted by the feature prediction unit; a feature amount filter unit filtering the observed feature amount detected by the feature amount detection unit based on the hypothesis generated by the feature amount correlation unit; A sub-optimization unit integrates similar trajectories based on the observation feature values ​​filtered by the feature value filter unit; A track establishment unit establishes a track based on the track after integration by the sub-optimization unit. A target tracking method comprising the steps of:

Images