Object Tracking Device

Abstract

An observation point extraction unit (31) of an object tracking device (1) extracts an observation point obtained from a target (3) of a current cycle, on the basis of a state amount of the target (3) estimated in the previous cycle. A prediction point generation unit (33) generates a prediction point at a prediction position where the observation point can be expected, on the basis of the state amount of the target (3) estimated in the previous cycle. An alignment unit (35) aligns, through scan matching, the observation point obtained from the target (3) of the current cycle with the prediction point generated by the prediction point generation unit (33). A residual error calculation unit (37) obtains a correspondence point of an object at the prediction position corresponding to the observation point from the prediction point aligned with the observation point obtained from the current cycle, and calculates a residual error between the observation point and the correspondence point.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This international application claims priority based on Japanese Patent Application No. 2022-107182, filed with the Japan Patent Office on July 1, 2022, the entire contents of which are incorporated by reference into this international application. [Technical field]

[0002] The present disclosure relates to techniques for tracking objects such as vehicles. [Background technology]

[0003] Conventionally, as a technology for tracking (i.e., tracking) objects (i.e., targets) such as other vehicles around a vehicle using a sensor such as a radar, a mass point tracking technology is known in which the target is tracked assuming that it is a single mass point, as shown in the upper left part of Figure 17.

[0004] In recent years, in order to improve the problem of mass point tracking (i.e., the problem of unstable tracking), extended object tracking (i.e., EOT) has become known, which tracks the target by incorporating the target's shape in addition to the driving state of the target into a model, as shown in the upper right of Fig. 17. EOT is an abbreviation for Extended Object Tracking, and this technology is disclosed in, for example, Non-Patent Document 1.

[0005] However, with this extended object tracking technology, as shown in the center diagram of Figure 17, when the movement of the target or the vehicle changes suddenly, or when the observation information of the target is insufficient, such as in the early stages of tracking, it is sometimes difficult to track the target accurately.

[0006] As a countermeasure against this, for example, the technology described in Patent Document 1 below has been proposed. This technology, as shown in the lower diagram of Fig. 17, defines the likelihood of an observation point as a likelihood function from the state distribution of the tracked target predicted at a certain time, corrects the likelihood function (i.e., adjusts the weight) in accordance with the number of observation points, and then updates the state distribution, thereby suppressing the deterioration of tracking accuracy and tracking failures due to sparse observation information.

[0007] The publication of Patent Document 1 discloses that a prediction error is dealt with by varying the prediction distribution that can be taken by a particle filter. In other words, it describes that the tracking performance (for example, improvement of tracking loss, etc.) is improved by generating multiple hypotheses of the predicted state distribution. [Prior art documents] [Patent documents]

[0008] [Non-Patent Document 1] K. Granstrom, M. Baum, and S. Reuter, “Extended object tracking: Introduction, overview, and applications,” Journal of Advances in Information Fusion, vol. 12, no. 2, 2017. [Patent Document 1] Patent Publication No. 2021-67482 Summary of the Invention

[0009] As a result of detailed investigation by the inventors, the following problems were found in the conventional techniques. Specifically, the technique described in Patent Document 1 generates multiple hypotheses for the predicted state distribution, which results in a large computational load.

[0010] That is, since the computing power of an electronic control device such as a computer of the vehicle is limited, there is a problem that the computational load is large when multiple hypotheses of the predicted state distribution are generated as described above.

[0011] That is, when performing calculations in an electronic control device, the number of hypotheses that can be generated is limited under the constraint of a limited amount of calculation, so that the range of prediction error that can be handled is limited, and it is not easy to perform tracking with high accuracy. In particular, when the state vector indicating the state quantity of the target is high-dimensional, there is a problem that the range of prediction error becomes significantly narrower.

[0012] It is desirable for one aspect of the present disclosure to provide a technology that can track an object with high accuracy and reduce the computational load when tracking the object. One aspect of the present disclosure relates to an object tracking device (1) that senses the surrounding environment using a sensor (5) in a predetermined cycle and tracks a target object (3) based on information obtained by the sensing.

[0013] The object tracking device includes a tracking processing unit (9). The tracking processing unit is configured to estimate a state quantity indicating the state of the object in the current cycle using information on an observation point indicating the object obtained by a sensor, and uses, as the state quantity of the object, at least state quantities related to the position and shape of the object.

[0014] The tracking processing unit (9) includes an observation point extraction unit (31), a predicted point generation unit (33), a registration unit (35), a residual calculation unit (37), and a state update unit (39). The tracking processing unit is configured to estimate a state quantity indicating a state of the object in the current cycle, using information on an observation point indicating the object obtained by the sensor.

[0015] The observation point extraction unit is configured to extract observation points obtained from the object in the current cycle based on the state quantity of the object estimated in the previous cycle (i.e., the cycle immediately before the current cycle). Note that in the case of an initial cycle, it may be configured to extract all the obtained observation points.

[0016] The predicted point generating unit is configured to generate at least one predicted point at a predicted position where the observation point can be expected based on the state quantity of the object estimated in the previous cycle. The alignment unit is configured to align the observation points obtained from the object in the current cycle with the predicted points generated by the predicted point generation unit by scan matching.

[0017] The residual calculation unit is configured to obtain a corresponding point of the object at a predicted position corresponding to the observation point from the predicted point aligned with the observation point obtained from the current cycle, and to calculate a residual between the observation point and the corresponding point.

[0018] The state update unit is configured to update the state quantity of the object based on the residual calculated by the residual calculation unit. With this configuration, the object tracking device according to one aspect of the present disclosure can track an object with high accuracy and reduce the computational load required to track an object.

[0019] In other words, in the present disclosure, instead of making multiple predictions as in the past, it is possible to efficiently perform calculations for tracking objects by using residuals that calculate the deviation (i.e., the correspondence) between the observation point and the predicted shape (i.e., the predicted point). Therefore, it is possible to reduce the calculation load when tracking an object, and to suitably track the object.

[0020] Furthermore, the symbols in parentheses in this column and in the claims indicate a correspondence with the specific means described in the embodiments described below as one aspect, and do not limit the technical scope of the present disclosure. [Brief description of the drawings]

[0021] [Figure 1] 1 is a block diagram showing an overall configuration of an object tracking device according to a first embodiment. [Diagram 2] 1 is a block diagram functionally illustrating an object tracking device according to a first embodiment. [Diagram 3]4 is a flowchart showing a control process of the object tracking device of the first embodiment. [Figure 4] FIG. 4 is an explanatory diagram showing the procedure in S100 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Diagram 5] FIG. 4 is an explanatory diagram showing the procedure in S110 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 6] FIG. 4 is an explanatory diagram showing the procedure in S120 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 7] FIG. 4 is an explanatory diagram showing the procedure in S130 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 8] FIG. 4 is an explanatory diagram showing the procedure in S140 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 9] FIG. 1 is an explanatory diagram showing a scan matching method. [Figure 10] FIG. 4 is an explanatory diagram showing the procedure in S150 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 11] FIG. 4 is an explanatory diagram showing the procedure in S160 of FIG. 3 out of the procedures for object tracking in the first embodiment. [Figure 12] FIG. 11 is an explanatory diagram showing a method of calculating a residual vector. [Figure 13] FIG. 11 is an explanatory diagram showing a method of updating a state quantity. [Figure 14] FIG. 14A is an explanatory diagram showing a scan matching method for object tracking in the second embodiment, and FIG. 14B is an explanatory diagram showing an undesirable scan matching method. [Figure 15] FIG. 13 is an explanatory diagram showing a state in which the spread of observation points is small in the procedure of object tracking in the third embodiment. [Figure 16] FIG. 13 is an explanatory diagram showing the relationship between the area of ​​a rectangle and the covariance of observation points in the procedure of object tracking in the third embodiment. [Figure 17] FIG. 1 is an explanatory diagram of a conventional technique. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. [1. First embodiment] In the first embodiment, an object tracking device mounted on a vehicle (e.g., a vehicle itself) is taken as an example to explain the present invention. That is, an object tracking device that scans (i.e., senses) the surrounding environment of the vehicle itself in a predetermined cycle using a sensor and tracks a target object based on information obtained by the sensing is taken as an example to explain the present invention.

[0023] [1-1. Overall composition] First, the overall configuration of an object tracking device 1 according to the first embodiment will be described with reference to FIG. As shown in FIG. 1, an object tracking device 1 is a device that is mounted on a vehicle (eg, a host vehicle) and tracks an object (ie, a target) 3 such as another vehicle around the host vehicle.

[0024] In other words, the object tracking device 1 is a device that realizes EOT, which recognizes the spread target 3 and performs tracking processing of the target 3 from at least one observation point where the target 3 is observed, as will be described in detail later.

[0025] In this example, a vehicle such as an automobile is taken as an example of the target 3, but other moving objects such as a motorcycle or a pedestrian can also be used. Stationary objects having a certain shape can also be used.

[0026] The object tracking device 1 includes a surrounding sensor, such as a distance measuring sensor 5, that senses the surroundings of the host vehicle, a group of behavior sensors 7 that detect the state of the host vehicle, a tracking processing unit 9 that performs processing for tracking the target 3, and the like.

[0027] The distance measuring sensor 5 may be any of various sensors that can obtain information about the position of the target 3, such as the distance between the object tracking device 1 and the target 3, and information about the direction of the target 3 relative to the object tracking device 1.

[0028] For example, at least one of the well-known radar, lidar, sonar, stereo camera, and monocular camera may be used. Note that the sensor may be capable of observing three-dimensional information (x, y, z) indicating a position in space, or may be capable of observing Doppler velocity or signal strength.

[0029] This distance measurement sensor 5 repeats sensing of the target 3 (i.e., scanning and detecting with a radar or the like) and observation of a reflection point group indicating the portion reflected by the target 3 at a predetermined time interval (i.e., a predetermined cycle). Then, as described later, in the tracking processing unit 9, at a specific time k, the reflection point group is recognized by the distance measurement sensor 5 as at least one observation point for the target 3 at the same time k.

[0030] As the behavior sensor group 7, various sensors capable of detecting the behavior of the host vehicle, such as the amount of movement, etc., such as an inertial measurement unit (i.e., IMU) or odometry, can be used. Information detected by the behavior sensor group 7 includes, for example, the amount of operation of the accelerator pedal, the amount of operation of the brake pedal, the steering angle, the vehicle speed, the vehicle acceleration, etc.

[0031] The tracking processing unit 9 is an electronic control device mainly composed of a microcomputer including a control unit 11 that performs various processes such as tracking of the target 3, and a storage unit 13 that stores various data and programs.

[0032] In detail, the control unit 11 includes a well-known processor (for example, a CPU 11a), and the storage unit 13 includes, for example, a semiconductor memory such as a RAM or a ROM (hereinafter, referred to as a memory 13a).

[0033] The functions of the control unit 11 are realized by the CPU 11a executing a program stored in a non-transient substantial recording medium (i.e., the memory 13a). In addition, by executing this program, a method corresponding to the program is executed.

[0034] The method of realizing the various functions of the control unit 11 is not limited to software, and some or all of the elements may be realized by one or more pieces of hardware. For example, when the above functions are realized by electronic circuits that are hardware, the electronic circuits may be realized by digital circuits including a large number of logic circuits, or analog circuits, or a combination of these.

[0035] <Functional configuration of the tracking processing unit> Here, a description will be given of the functional configuration of the tracking processing unit 9. Note that a control unit 11 is a part that performs calculation processing for tracking.

[0036] As shown in FIG. 2, the tracking processing unit 9 (more specifically, the control unit 11) functionally has an observation point extraction unit 31, a predicted point generation unit 33, an alignment unit 35, a residual calculation unit 37, a state update unit 39, and a state prediction unit 41.

[0037] The tracking processing unit 9 is configured to estimate a state quantity (i.e., a state vector) indicating the state of the target 3 in the current cycle, using information on the observation point indicating the target 3 obtained by the distance measurement sensor 5. Note that as the state quantity, at least state quantities related to the position of the target 3 and the shape of the target 3 can be used.

[0038] The observation point extraction unit 31 is configured to extract an observation point obtained from the target 3 in the current cycle, based on the state quantity of the target 3 estimated in the previous cycle (that is, the cycle immediately before the current cycle).

[0039] The predicted point generating unit 33 is configured to generate at least one predicted point at a predicted position where an observation point can be expected, based on the state quantity of the target 3 estimated in the previous cycle.

[0040] The alignment unit 35 is configured to align the observation points obtained from the target 3 in the current cycle with the predicted points generated by the predicted point generation unit 33 by scan matching.

[0041] The residual calculation unit 37 is configured to obtain a corresponding point of the object at a predicted position corresponding to the observation point from the predicted point aligned with the observation point obtained from the current cycle, and to calculate the residual (i.e., residual vector) between the observation point and the corresponding point.

[0042] The state update unit 39 is configured to update the state quantity of the target 3 based on the residual calculated by the residual calculation unit 37 . The state prediction unit 41 is configured to predict the state quantity of the next cycle (that is, the cycle immediately following the current cycle) based on the state quantity updated by the state update unit 39.

[0043] [1-2. Overall processing by the tracking processing unit] Next, a schematic procedure of the overall processing of the object tracking device 1 of the first embodiment will be described based on the flowchart in Fig. 3. The contents of each step will be described in detail later.

[0044] First, in step (hereinafter, S) 100 in FIG. 3, initial values ​​of the motion and shape of the target 3 are set based on observation points obtained in the most recent predetermined period. In the next step S110, the motion and shape of the target 3 at the current time are predicted.

[0045] In the next step S120, observation points that are expected to be on the contour (that is, the actual contour) of the target 3 are extracted from the associated observation points. In the next step S130, expected observation points (ie, predicted points) are sampled from the contour of the predicted shape of the target 3 (ie, predicted contour).

[0046] In the next step S140, the predicted points and the observed points are aligned by scan matching. In the next step S150, a corresponding point on the predicted contour that corresponds to the observed point is calculated.

[0047] In the next S160, a residual (that is, a residual vector) is found from the observation point and the corresponding point, and the state quantity (that is, a state vector) at the current time is updated. Thereafter, the process returns to S110 using the updated state quantities and the same processing is repeated.

[0048] [1-3. Contents of each process by the tracking processing unit] Next, the contents of each process (ie, S100 to S160) in the tracking processing unit 9 in FIG. 3 will be explained in detail in due order.

[0049] (1) Contents of S100 processing In this step, initial values ​​for the motion and shape of the target 3 are set based on observation points obtained in the most recent predetermined period.

[0050] Here, the motion includes the position and velocity (i.e., the magnitude and direction of the velocity) of the mass point of the target 3, and the shape includes the size of the target 3 (i.e., the spatial extent of the target 3) and orientation (i.e., the direction in which the target 3 faces in the shape). Since the size of the target 3 is constant, the size (e.g., width, depth, height, etc.) and orientation of the target 3 may be considered as the shape. The position may be coordinates in a predetermined coordinate system (e.g., a coordinate system that defines north, south, east, west, etc., or a coordinate system based on a predetermined vehicle such as the subject vehicle).

[0051] As shown in Fig. 4, suppose that a distance measuring sensor 5 such as a radar is used to obtain, for example, multiple observation points on the target 3 in the lower left of the figure by sensing in the first cycle, and multiple observation points on the target 3 in the upper right of the figure by sensing in the second cycle. Note that the observation points are reflection points on the target 3 of radar waves, etc. In Fig. 4, the observation points are indicated by circles with fine meshes inside. Note that in other drawings, similar circles with meshes inside indicate observation points.

[0052] In this case, for example, the position of the target 3 in the first cycle can be known from the center of gravity of the multiple observation points in the first cycle (i.e., center of gravity 1), and the position of the target 3 in the second cycle can be known from the center of gravity of the multiple observation points in the second cycle (i.e., center of gravity 2). In addition, the velocity of the target 3 (i.e., the motion of the target 3) can be known from the movement vector from center of gravity 1 to center of gravity 2.

[0053] Separately, a case will be described in which the outer peripheral shape of the target 3 when viewed from above (i.e., in a plan view) (i.e., the outline including the size of the target 3) is known in advance. For example, as shown by the rectangular frame in the figure, when the outline of the target 3 is rectangular, the shape of the target 3 in a plan view (i.e., size and orientation) can be known by arranging the multiple observation points so as to match the outline of the target 3 (i.e., fitting). That is, as shown by the rectangular frame in the figure, the shape of the target in a plan view can be known, and the center of gravity of each target 3 can be known from the shape of each target 3.

[0054] In this way, the position of the target 3 can be determined from the positions of the center of gravity of the target 3 in the first cycle and the second cycle, and the movement of the target 3 can be determined from the movement vector of the center of gravity.

[0055] Therefore, for example, the center of gravity of the target 3 in the first cycle can be set as the initial value of the position of the target 3, and the velocity obtained from the movement vector can be set as the initial value of the velocity of the target 3 in the second cycle.

[0056] In the following description, the planar shape of the target 3 will be described as, for example, a rectangle that matches the outer peripheral shape (that is, the outline) of the vehicle. (2) Contents of processing at S110 In this step, the motion and shape of target 3 at the current time are predicted.

[0057] As shown in Fig. 5, if the position and speed of the target 3 at the time one cycle before (i.e., the previous time) are known, it is possible to predict the position and speed of the target 3 in the current cycle. In the figure, the contour of the predicted target is shown by a dashed line.

[0058] In other words, the predicted position of the target 3 can be determined by predicting that the target 3 has moved from its position at the previous time at the same speed as the target 3 at the previous time. Note that the speed at the predicted position of the target 3 may be the same as the speed at the previous time.

[0059] In addition, in the same figure, the movement state of the actual target 3 is indicated by a solid arrow, the observation points actually observed at the current time are indicated by large and small circles, and the outline of the actual target 3 at the current time (i.e., the outline at the tip of the arrow) is indicated by a solid line.

[0060] Of the multiple observation points actually observed, the observation points located on the contour of the target 3 are shown larger than the observation points located inside the target 3. Usually, many observation points are obtained from the contour (i.e., the outer periphery).

[0061] (3) Contents of processing at S120 In this step, observation points that are expected to be on the contour of the target 3 are extracted from the corresponding observation points.

[0062] As shown in FIG. 6, since there may be a discrepancy between the actual position of the target 3 and the predicted position of the target 3, the actual position of the target 3 is usually estimated to be within a connection range in the vicinity of the predicted position of the target 3, for example, within a predetermined distance from the predicted target 3.

[0063] Then, a plurality of observation points within the connection range (i.e., the observation points represented by large and small circles in FIG. 6) are selected as the actual observation points of the target 3 at the current time. Furthermore, from these observation points, the observation points expected to be on or near the actual contour (i.e., actual contour) of the target 3 (i.e., the observation points represented by large circles with meshes inside in Fig. 6) are extracted. In other words, since the contour of the target 3 is rectangular, the observation points expected to be on or near the rectangular frame are extracted.

[0064] (4) Contents of processing at S130 In this step, expected observation points (ie, predicted points) are sampled from the predicted contour of the target 3 (ie, predicted contour).

[0065] As shown in Fig. 7, here, prediction points are set at predetermined intervals on the contour of the predicted rectangular frame-shaped target 3. In Fig. 7, the prediction points are shown as circles with multiple dots inside. Note that in other drawings, circles with similar dots show the prediction points.

[0066] (5) Contents of processing at S140 In this step, the predicted points of the predicted contour and the observation points extracted on the actual contour (i.e., the observation points along the actual contour) are aligned by scan matching.

[0067] As shown in FIG. 8, predicted points and observed points are aligned by a known scan matching method, so that the predicted contour and the actual contour are overlapped as much as possible. For example, as shown in FIG. 9, the prediction points are translated (i.e., moved in parallel) or rotated so that the prediction points and the observation points have a one-to-one correspondence, and processing is performed to bring the multiple observation points and the multiple prediction points as close as possible (i.e., geometric transformation estimation and transition processing).

[0068] In other words, the multiple prediction points are moved while maintaining the shape resulting from the arrangement of the multiple prediction points so that the shape resulting from the arrangement of the multiple extracted observation points (e.g., an L-shape) matches as closely as possible.

[0069] Here, the basics of scan matching will be described. As a scan matching method, for example, the ICP algorithm, which is widely used as a basic method for aligning 3D shapes, can be used. The ICP algorithm is a method for performing final alignment by alternately repeating two processes, calculating correspondence using nearest neighbors and estimating geometric transformation from corresponding points, when the predicted point cloud is the source point cloud and the observed points are the target point cloud, until the convergence condition is satisfied.

[0070] This scan matching technique is described in the following documents [1] and [2]. Note that the following document [1] shows the original paper on the ICP algorithm, which describes the detailed steps. Also, document [2] is a tutorial that provides an overview of the ICP algorithm. Reference [1]: Besl, Paul J. and Neil D. McKay. “A Method for Registration of 3-D Shapes.” IEEE Trans. Pattern Anal. Mach. Intell.14 (1992): 239-256. References [2]: Ken Masuda, “Icp Algorithm,” Information Processing Society of Japan Technical Report, vol. 2009, no. 2, pp. 1-8, 2009. (6) Contents of processing at S150 In this step, the corresponding points on the predicted contour that correspond to the observed points are calculated.

[0071] As shown in FIG. 10, for the observation points and predicted points obtained by scan matching, predicted points (that is, corresponding points) on the predicted contour corresponding to the observation points are obtained by, for example, radial projection. For example, a straight line is drawn from the predicted position of the mass point of the target 3 as the center of radiation toward the observation point on the actual contour, and the observation point on that line or closest to that line is placed in one-to-one correspondence with the predicted point.

[0072] The radial projection method is a known technique, as described in, for example, the following document [3]. Reference [3]: M.Michaelis, P.Berthold, T.Luettel, D.Meissner, and HJWuensche, “Extended object tracking with an improved measurement-to-contour association,” Proc.2020 23rd Int.Conf.Inf.Fusion,FUSION 2020,no.July,2020,doi:10.23919 / FUSION45008.2020.9190439. The predicted point associated with this observation point is the corresponding point on the predicted contour of the predicted target 3, as shown in FIG.

[0073] (7) Contents of processing at S160 In this step, as shown in FIG. 11, a residual vector is obtained from the observation point and the corresponding point (that is, the predicted point corresponding to the observation point), and the state quantity of the target 3 at the current time is updated.

[0074] As is well known, the state quantity of the target 3 can be expressed as a state vector. For example, when the state quantities of the target 3 are expressed by the lateral position x, the longitudinal position y, the lateral velocity vx, and the longitudinal velocity vy, the state vector expressed as a vector with these state quantities as elements can be expressed as X=[x, y, vx, vy]T. Note that, for example, the lateral direction is the vehicle width direction, and the longitudinal direction is the direction perpendicular to the vehicle width direction.

[0075] Here, the residual vector will be described. As shown in FIG. 12, the residual (i.e., Δ) expressed as a matrix is ​​the difference Δ=ZH(xt) between the predicted value (i.e., predicted estimated value) xt of the state quantity of target 3 at the current time and the value indicating the observation point (i.e., observed value) Z.

[0076] H is the observation model. The i-th element of the residual δ becomes the residual vector δi. Note that a tilde "~" is added above the x in the predicted value xt as shown in Figure 12, but it is omitted here.

[0077] For example, let P be the predicted point (group), pi be the i-th point belonging to it, Z be the observed point (group), and zj be the j-th point belonging to it. Also, let us assume that pi and zj correspond to each other based on the correspondence information obtained in advance. In this case, the residual vector δi is calculated as δi=zj-pi (i.e., the difference between the information of points zj and pi: the difference between vector zj and vector pi).

[0078] Here, the information held by pi and zj is, for example, the position information (x, y) of the point. Note that other dimensions such as the velocity information (vx, vy) of the point may be added. The following document [4] describes a specific calculation method for determining a residual vector from correspondence information in a B-spline model.

[0079] Reference [4]: ​​H. Kaulbersch, J. Honer, and M. Baum, “A Cartesian B-Spline Vehicle Model for Extended Object Tracking,” 2018 21st Int. Conf. Inf. Fusion, FUSION 2018, no. December 2019, pp.1771-1778,2018,doi:10.23919 / ICIF.2018.8455717. In addition, the following document [5] describes a residual formulation that takes into account point velocity information (Doppler Range Rate) in the star-convex model.

[0080] Reference [5]: K.Thormann, M.Baum, and J.Honer, “Extended Target Tracking Using Gaussian Processes with High-Resolution Automotive Radar,”2018 21st Int.Conf.Inf.Fusion,FUSION 2018,no.December 2019,pp.1764-1770,2018, doi:10.23919 / ICIF.2018.8455630. Next, a method for updating the state quantity at the current time will be explained. However, since this is a publicly known technique (see, for example, JP 2021-4737 A, etc.), a brief explanation will be given.

[0081] As shown in FIG. 13, the residual Δ=ZH(xt) is calculated from the predicted value xt of the state quantity of the target 3 at the current time and the observed value Z of the observation point, and the state quantity of the object is updated using a well-known nonlinear Kalman filter (e.g., an extended Kalman filter or a sigma-point Kalman filter).

[0082] Then, after updating the state quantity at the current time, the process returns to step 110 and repeats the same process using the updated state quantity. In other words, the state quantity of the target 3 in the next cycle is predicted using the updated information quantity.

[0083] The process of predicting the state quantity of the target 3 in the next cycle is publicly known. For example, the predicted value xt of the state quantity of the target 3 in the next cycle (time t) can be calculated using the state quantity x(t-1) of the target 3 in the previous cycle (time t-1) and a motion model (see the following formula [Math 1]) that represents the state transition of the target 3.

[0084]

number

[0085] [1-4. Effects] According to the first embodiment, the following effects can be obtained. In the first embodiment, the tracking processing unit 9 estimates a state quantity (i.e., a state vector) indicating the state of the target 3 in the current cycle using information on the observation point indicating the target 3 obtained by the distance measurement sensor 5. The observation point extraction unit 31 extracts an observation point obtained from the target 3 in the current cycle based on the state quantity of the target 3 estimated in the previous cycle. The predicted point generation unit 33 generates at least one predicted point at a predicted position where the observation point can be expected based on the state quantity of the target 3 estimated in the previous cycle. The position matching unit 35 aligns the observation point obtained from the target 3 in the current cycle with the predicted point generated by the predicted point generation unit 33 by scan matching. The residual calculation unit 37 obtains a corresponding point of the target 3 at a predicted position corresponding to the observation point from the predicted point aligned with the observation point obtained from the current cycle, and calculates a residual (i.e., a residual vector) between the observation point and the corresponding point. The state update unit 39 updates the state quantity of the target 3 based on the residual. The state prediction unit 41 predicts the state quantity of the next cycle based on the updated state quantity.

[0086] With this configuration, the object tracking device 1 of the first embodiment can track the target 3 with high accuracy, and can reduce the computation load when tracking the target 3.

[0087] That is, in the first embodiment, instead of making multiple predictions as in the conventional method, the calculation can be performed efficiently by calculating the deviation (i.e., the correspondence) between the observation point and the predicted shape (i.e., the predicted point). Therefore, the calculation load when tracking the target 3 can be reduced, and the target 3 can be tracked with high accuracy.

[0088] [1-5. Correspondence] Next, the relationship between the first embodiment and the present disclosure will be described. The object tracking device 1 corresponds to the object tracking device, the target 3 corresponds to the target, the ranging sensor 5 corresponds to the sensor, the tracking processing unit 9 corresponds to the tracking processing unit, the observation point extraction unit 31 corresponds to the observation point extraction unit, the prediction point generation unit 33 corresponds to the prediction point generation unit, the alignment unit 35 corresponds to the alignment unit, the residual calculation unit 37 corresponds to the residual calculation unit, and the state update unit 39 corresponds to the state update unit.

[0089] [2. Second embodiment] Since the second embodiment has a basic configuration similar to that of the first embodiment, the following description will mainly focus on the differences from the first embodiment. Note that the same reference numerals as those in the first embodiment indicate the same configuration, and the preceding description will be referred to.

[0090] The hardware configuration of the second embodiment is the same as that of the first embodiment, so a description thereof will be omitted. In the second embodiment, the scan matching technique is unique, so the scan matching technique will be described.

[0091] In the second embodiment, in the process of alignment in scan matching, a process is performed in which points that can be observed from a sensor (for example, distance measurement sensor 5) are sequentially reviewed among the predicted points generated by the predicted point generation unit 33. This will be described in detail below.

[0092] [2-1. Contents of processing] Consider a case where observation points along an actual contour are obtained by sensing and predicted points along a predicted contour (e.g., a rectangular frame) are calculated, as shown in the leftmost diagram of Fig. 14A. Of all the predicted points along the rectangular frame (i.e., predicted points indicated by circles), predicted points within a range where reflected waves are expected to be obtained from the sensor, for example, within the irradiation range of the radar wave (i.e., predicted points indicated by circles with multiple dots inside) are selected.

[0093] Here, as shown in the leftmost diagram in Figure 14A, if the observation points along the actual contour (i.e., circles with a fine mesh inside) and the predicted points of the predicted contour (i.e., circles with multiple dots inside) are aligned directly using the scan matching method described in the above-mentioned document [3], in some cases, appropriate alignment may not be achieved, as shown in Figure 14B.

[0094] In other words, the observation points of the actual contour are the observation points of the two sides on the lower right side of the actual contour, but the predicted points of the predicted contour are the predicted points of the two sides on the left side of the predicted contour, so even if scan matching is performed between these observation points and predicted points, it may not be possible to correctly match the observation points and predicted points.

[0095] Therefore, in the second embodiment, first, as shown in the second diagram from the left in FIG. 14A, a process of re-obtaining prediction points to be used in scan matching is performed. Specifically, a process is performed to retake predicted points in a sector-shaped range (e.g., the range of radar waves) that can be directly detected by the sensor as predicted points to be used in scan matching. In other words, of all predicted points along the predicted contour, points that can be observed by the sensor are sequentially reviewed.

[0096] Then, as shown in the rightmost diagram in FIG. 14A, scan matching is performed using the observed points of the actual contour and the re-obtained predicted points of the predicted contour. [2-2. Effects] The second embodiment provides the same effects as the first embodiment.

[0097] Furthermore, in the second embodiment, it is possible to prevent the scan matching from being associated with an incorrect side surface of the target 3. This enables appropriate residual calculation, thereby improving tracking accuracy.

[0098] The use of observable positions of the predicted shape (i.e., predicted contour) of the target 3 for the correspondence calculation is described in the above-mentioned document [3]. However, the second embodiment is characterized in that the predicted points are retaken (i.e., sequentially reviewed) during the scan matching process.

[0099] [3. Third embodiment] Since the third embodiment has a basic configuration similar to that of the first embodiment, the following description will mainly focus on the differences from the first embodiment. Note that the same reference numerals as those in the first embodiment indicate the same configurations, and the preceding description will be referred to.

[0100] The hardware configuration of the third embodiment is the same as that of the first embodiment, so a description thereof will be omitted. The third embodiment shows processing when there are a small number of observation points.

[0101] [3-1. Processing Contents 1] In one example of the third embodiment, the spatial spread of points (i.e., the observation point group) observable from a sensor (e.g., the distance measurement sensor 5) is determined, and if the spread is within a certain narrow range, scan matching is not performed. Note that the spatial spread may be the spread in a planar view.

[0102] In other words, as shown in Figure 15, when the spread of the observation point group is narrow, it is difficult to match the observation points with the prediction points, and there is a possibility that scan matching cannot be performed appropriately, so in such cases, scan matching is not performed.

[0103] This makes it possible to suppress deterioration in object trackability caused by inappropriate scan matching. [3-2. Processing Contents 2] In another example of the third embodiment, the spatial spread of points observable from the sensor (i.e., the group of observation points) is determined, and if the spread is within a narrow range below a certain level, the weighting of the observation information, which is information about the observation points, is reduced when updating the state quantity.

[0104] In other words, it is determined that geometric features such as contours (e.g., the shape of a rectangular frame) cannot be obtained, and it is assumed that scan matching may fail. In such cases, the weight of the observation information is reduced.

[0105] The details will be explained below. Assume that the observation noise can be expressed by a covariance matrix Rt and follows a zero-mean multivariate normal distribution. When the observations are independent, the covariance matrix Rt is defined as a diagonal matrix as shown in the following equation [Mathematical formula 2]. Note that here, the observation noise means, for example, an error in the distance measurement information obtained from the distance measurement sensor 5 (for example, an error in distance or an error in Doppler velocity).

[0106]

number

[0107] If there is only one observation point, it can be determined that there is no spatial spread (i.e., area is zero), so the components of the covariance matrix have very large values ​​(for example, 10σ 2 In the case of two or more points, for example, the area of ​​a rectangle circumscribing the point group is calculated, and the components of the covariance matrix are adjusted according to the area, for example, in the relationship shown in FIG.

[0108] In this way, when it is determined that geometric features cannot be obtained and scan matching fails, the observation noise is increased, the weight of the observation information is relatively reduced, and prediction-oriented estimation is performed (i.e., emphasis is placed on estimation using predicted points), thereby improving robustness.

[0109] For example, by performing processing using the following equation [Mathematical formula 3] shown in "Proposition 2 (Nonlinear Kalman Filter)" of "Basics of Nonlinear Kalman Filter" in the following document [6] (see the URL below), it is possible to reduce the weight of observed information and perform estimation that emphasizes prediction.

[0110] Reference [6]:https: / / www.jstage.jst.go.jp / article / sicejl / 56 / 9 / 56_638 / _pdf

[0111]

number

[0112] That is, the Kalman gain K in the extended Kalman filter is calculated by using the above-mentioned covariance matrix Rt in the well-known formula described in the above-mentioned [Mathematical formula 3]. Here, since the covariance matrix Rt is included in the denominator term, if the covariance matrix Rt is large, the Kalman gain K becomes small. And, as the Kalman gain K becomes small, the observation weight decreases.

[0113] In this way, by placing more importance on estimation using prediction points than estimation using observation points, it is possible to improve the object tracking ability compared to the case where this is not the case. [4. Other embodiments] Although the embodiments of the present disclosure have been described above, it goes without saying that the present disclosure is not limited to the above-described embodiments and can take various forms.

[0114] (4a) The object tracking device described in the present disclosure can be applied to autonomous driving systems for vehicles, surveillance systems, and the like. (4b) The object tracking device described in the present disclosure may be implemented by a special purpose computer provided by configuring a processor and memory programmed to perform one or more functions embodied in a computer program.

[0115] Alternatively, the object tracking device described in the present disclosure may be implemented by a special purpose computer provided by configuring a processor with one or more special purpose hardware logic circuits.

[0116] Alternatively, the object tracking device described in the present disclosure may be implemented by one or more special purpose computers configured by a combination of a processor and memory programmed to perform one or more functions and a processor configured by one or more hardware logic circuits.

[0117] The computer program may be stored in a computer-readable non-transitory recording medium as instructions executed by a computer. The method for realizing the functions of each unit included in the object tracking device does not necessarily need to include software, and all of the functions may be realized using one or more pieces of hardware.

[0118] (4c) In addition to the object tracking device described above, a program for operating the computer of the object tracking device, a non-transient tangible recording medium such as a semiconductor memory on which the program is recorded, The present disclosure may be implemented in a variety of forms, such as a tracking method.

[0119] (4d) Multiple functions possessed by one component in each of the above embodiments may be realized by multiple components, or one function possessed by one component may be realized by multiple components. Also, multiple functions possessed by multiple components may be realized by one component, or one function realized by multiple components may be realized by one component. Also, part of the configuration of each of the above embodiments may be omitted. Also, at least part of the configuration of each of the above embodiments may be added to or substituted for the configuration of another embodiment. [Technical idea disclosed in this specification] [Item 1] An object tracking device (1) that senses a surrounding environment by a sensor (5) in a predetermined cycle and tracks a target object (3) based on information obtained by the sensing, a tracking processing unit (9) configured to estimate a state quantity indicating a state of the object in a current cycle using information of an observation point indicating the object obtained by the sensor; The tracking processing unit is As the state quantity of the object, at least a state quantity related to a position of the object and a state quantity related to a shape of the object are used, an observation point extraction unit (31) configured to extract the observation points obtained from the object in the current cycle based on a state quantity of the object estimated in a previous cycle; a predicted point generating unit (33) configured to generate at least one predicted point at a predicted position where the observation point is expected based on the state quantity of the object estimated in the previous cycle; a registration unit (35) configured to register the observation points obtained from the object in the current cycle and the predicted points generated by the predicted point generation unit by scan matching; a residual calculation unit (37) configured to obtain a corresponding point of the object at the predicted position corresponding to the observation point from the observation point obtained from the current cycle and the aligned predicted point, and to calculate a residual between the observation point and the corresponding point; a state update unit (39) configured to update a state quantity of the object based on the residual calculated by the residual calculation unit; An object tracking device comprising:

[0120] [Item 2] The object tracking device according to item 1, The alignment unit is configured to extract, from the observation points obtained from the object in the current cycle, the observation points that are on a contour of the object or in the vicinity of a predetermined range of the contour, and use the extracted observation points for the scan matching. Object tracking device.

[0121] [Item 3] The object tracking device according to item 1 or 2, The alignment unit is configured to review the predicted points generated by the predicted point generation unit during the alignment process of the scan matching so as to select the predicted points that can be observed by the sensor. Object tracking device.

[0122] [Item 4] An object tracking device according to any one of items 1 to 3, The alignment unit is configured to determine a spatial spread of the point cloud of the observation points that can be observed by the sensor, and not to perform the scan matching when the spatial spread is equal to or smaller than a predetermined range. Object tracking device.

[0123] [Item 5] An object tracking device according to any one of items 1 to 4, The alignment unit is configured to determine a spatial spread of the point cloud of the observation points observable by the sensor, and when the spatial spread is equal to or smaller than a predetermined range, reduce a weight of the observation points when updating the state quantity using the observation points, compared to when the spatial spread is not equal to or smaller than the predetermined range. Object tracking device.

Claims

1. An object tracking device (1) that senses a surrounding environment by a sensor (5) in a predetermined cycle and tracks a target object (3) based on information obtained by the sensing, a tracking processing unit (9) configured to estimate a state quantity indicating a state of the object in a current cycle using information of an observation point indicating the object obtained by the sensor; The tracking processing unit is As the state quantity of the object, at least a state quantity related to a position of the object and a state quantity related to a shape of the object are used, an observation point extraction unit (31) configured to extract the observation points obtained from the object in the current cycle based on a state quantity of the object estimated in a previous cycle; a predicted point generating unit (33) configured to generate at least one predicted point at a predicted position where the observation point can be expected based on the state quantity of the object estimated in the previous cycle; a registration unit (35) configured to register the observation points obtained from the object in the current cycle and the predicted points generated by the predicted point generation unit by scan matching; a residual calculation unit (37) configured to obtain a corresponding point of the object at the predicted position corresponding to the observation point from the observation point obtained from the current cycle and the aligned predicted point, and to calculate a residual between the observation point and the corresponding point; a state update unit (39) configured to update a state quantity of the object based on the residual calculated by the residual calculation unit; An object tracking device comprising:

2. The object tracking device according to claim 1 , The alignment unit is configured to extract, from the observation points obtained from the object in the current cycle, the observation points that are on a contour of the object or in the vicinity of a predetermined range of the contour, and use the extracted observation points for the scan matching. Object tracking device.

3. The object tracking device according to claim 1 , The alignment unit is configured to review the predicted points generated by the predicted point generation unit during the alignment process of the scan matching so as to select the predicted points that can be observed by the sensor. Object tracking device.

4. The object tracking device according to claim 1 , The alignment unit is configured to determine a spatial spread of the point cloud of the observation points that can be observed by the sensor, and not to perform the scan matching when the spatial spread is equal to or smaller than a predetermined range. Object tracking device.

5. The object tracking device according to claim 1 , The alignment unit is configured to determine a spatial spread of the point cloud of the observation points observable by the sensor, and when the spatial spread is equal to or smaller than a predetermined range, reduce a weighting of the observation points when updating the state quantity using the observation points, compared to when the spatial spread is not equal to or smaller than the predetermined range. Object tracking device.

Images