Object tracking method and apparatus using lidar sensor, vehicle, and recording medium

Abstract

This invention discloses an object tracking method using a LiDAR sensor. The method includes: clustering LiDAR data including multiple points of an object acquired by the LiDAR sensor; generating information on multiple segment boxes for each channel using the clustering results; and selecting, from the multiple segment boxes, the associated segment box of the target object at the current time point. The step of selecting the associated segment box includes: obtaining the correlation index between the current representative point of each segment box at the current time point and each of the tracking representative point and the previous representative point; using the correlation index, selecting candidate segment boxes for the associated segment box; and selecting the associated segment box at the current time point.

Description

Technical Field

[0001] Embodiments of the present invention relate to an object tracking method and apparatus using a LiDAR sensor, a vehicle including the apparatus, and a recording medium storing a program for executing the method. Background Technology

[0002] The vehicle's Highway Driving Driver (HDP) system provides the following function: when the driver sets the vehicle speed and the distance between the vehicle and the vehicle in front in its lane, it can maintain the vehicle speed according to the conditions set by the driver, even without operating the accelerator pedal or brake pedal.

[0003] For example, information about a target vehicle can be obtained using a LiDAR (Light Detection and Ranging) sensor, and the acquired information can be used to perform HDP (High-Reliability Vehicle) functions. However, if the information about the target vehicle obtained using the LiDAR sensor is incorrect, the HDP function may be executed incorrectly, leading to a decrease in vehicle reliability. Summary of the Invention

[0004] Therefore, embodiments of the present invention relate to an object tracking method and apparatus using a LiDAR sensor that substantially eliminates one or more problems caused by the limitations and disadvantages of the prior art, a vehicle including the apparatus, and a recording medium storing a program for performing the method.

[0005] The embodiments may provide an object tracking method and apparatus using a LiDAR sensor with stable tracking performance, a vehicle including the apparatus, and a recording medium storing a program for executing the method.

[0006] However, the objectives achieved by the embodiments are not limited to those described above, and other objectives not mentioned herein will be clearly understood by those skilled in the art from the following description.

[0007] An object tracking method using a LiDAR sensor according to an embodiment may include the following steps: clustering LiDAR data consisting of multiple points of an object acquired by the LiDAR sensor; generating information on multiple segment boxes for each channel using the clustering results; and selecting, among the multiple segment boxes, the associated segment box of the target object at the current time t. The step of selecting the associated segment box may include: obtaining a correlation index between the current representative point of each of the multiple segment boxes at the current time t and each of the tracking representative point and the previous representative point; using the correlation index, selecting candidate segment boxes for the associated segment box among the multiple segment boxes; and selecting the associated segment box at the current time t from the selected candidate segment boxes. The tracking representative point may correspond to the representative point of the tracking box of the target object at the current time t estimated using historical information. The previous representative point may correspond to the representative point of the segment box at the previous time t-1 that was selected as the associated segment box.

[0008] For example, the current representative point may include: a first outer representative point located at a corner of the fragment box; and a first center representative point located at the center of the fragment box. The tracking representative point may include: a second outer representative point located at a corner of the tracking box; and a second center representative point located at the center of the tracking box. The previous representative point may include: a third outer representative point located outside the associated fragment box selected at a previous time point.

[0009] For example, the correlation index may include a first correlation index between the first peripheral representative point and the second peripheral representative point, a second correlation index between the first central representative point and the second central representative point, and a third correlation index between the first peripheral representative point and the third peripheral representative point.

[0010] For example, the first correlation index, the second correlation index, and the third correlation index can be calculated using the following mathematical formula.

[0011]

[0012]

[0013]

[0014] Here, γ1 represents the first correlation index, γ2 represents the second correlation index, γ3 represents the third correlation index, and x mo and x t mo These represent the x-axis coordinates of the first outermost representative point, y and y respectively. mo and y t mo Let x and y represent the ordinates of the first outermost representative points, respectively. o The x-axis coordinate of the second outermost point is represented by y. oσ represents the ordinate of the second outermost representative point. 2 xm σ represents the variance value on the horizontal axis of the first outermost representative point. 2 ym σ represents the variance of the vertical axis of the first outermost representative point. 2 x σ represents the variance of the second outermost representative point on the horizontal axis. 2 y The vertical variance of the second outermost representative point is represented by x. mc The x-axis coordinate of the first central point is represented by y. mc σ represents the ordinate of the first central point. 2 xmc σ represents the variance of the first central point on the horizontal axis. 2 ymc The vertical variance of the first central point is represented by x. c The x-axis coordinate of the second center point is represented by y. c The x-axis represents the ordinate of the second center point. t-1 mo The x-axis coordinate of the third outermost point is represented by y. t-1 mo This represents the ordinate of the third outermost representative point.

[0015] For example, in the step of selecting candidate fragment boxes, a fragment box that meets at least one of the following three conditions is selected as a candidate fragment box of the associated fragment box.

[0016] γ1<C1

[0017] γ2<C2

[0018] γ3 <C3

[0019] Here, C1, C2, and C3 represent preset critical indices.

[0020] For example, a fragment box that does not meet the three conditions but overlaps with the tracking box is selected as a candidate fragment box.

[0021] For example, the step of selecting a related fragment box may include the following steps: assigning a first score to the distance fit of each candidate fragment box; assigning a second score to the reliability fit of each candidate fragment box; assigning a third score to the correlation between each candidate fragment box and the related fragment box selected at a previous time point; summing the first to third scores assigned to each candidate fragment box to calculate a final score; and selecting the candidate fragment box with the highest final score as the related fragment box at the current time point t.

[0022] For example, in the step of assigning the first score, the first score can be assigned to a candidate fragment box whose smallest correlation index among the first to third correlation indices is equal to or less than the first threshold.

[0023] For example, the step of assigning a second score may include the following steps: obtaining the reliability level of each candidate fragment box and assigning the second score to candidate fragment boxes that have a reliability level greater than a second threshold.

[0024] For example, the step of obtaining the reliability level may include the following steps: a shape reliability level determination step, obtaining a preset second-1 reliability level corresponding to the shape of the candidate fragment box; an attribute reliability level determination step, obtaining a preset second-2 reliability level corresponding to at least one of the ratio of the number of points included in the candidate fragment box to the size of the candidate fragment box or the dispersion of the points included in the candidate fragment box; a geometric reliability level determination step, obtaining a preset second-3 reliability level corresponding to the position of the candidate fragment box and the distance from the reference point to the candidate fragment box; and adding the preset second-1, second-2 and second-3 reliability levels to determine the reliability level to be compared with a second threshold.

[0025] For example, in the step of assigning the third score, the third score is assigned proportionally to the ratio of the area overlapping the associated fragment box selected at a previous time point with the total area of ​​each candidate fragment box.

[0026] For example, the method may further include the following steps: determining whether there is an associated segment box selected at the current time point t; when it is determined that an associated segment box exists, updating the historical information of the channel including the associated segment box; and when it is determined that there is no associated segment box, deleting the historical information of the channel without an associated segment box.

[0027] An object tracking apparatus utilizing a LiDAR sensor according to another embodiment may include: a clustering unit configured to group LiDAR data consisting of multiple points of an object acquired by the LiDAR sensor; a shape analysis unit configured to generate information on multiple segment boxes for each channel using the clustering results; and an object tracking unit configured to select, among the multiple segment boxes, the associated segment box of the target object being tracked at the current time t. The object tracking unit may include: a storage unit configured to store historical information for each channel; a correlation index calculation unit configured to calculate a correlation index between the current representative point of each of the multiple segment boxes at the current time t and each of the tracking representative point and the previous representative point; a candidate selection unit configured to select candidate segment boxes of the associated segment box among the multiple segment boxes using the correlation index; and a final selection unit configured to select the associated segment box of the current time t from the selected candidate segment boxes. The tracking representative point may correspond to the representative point of the tracking box of the target object at the current time t estimated using historical information. The previous representative point can correspond to the representative point of the fragment frame that was selected as the associated fragment frame at the previous time point t-1.

[0028] For example, the correlation index calculation unit may include: a first index calculation unit configured to calculate a first correlation index between a first outer representative point and a second outer representative point; a second index calculation unit configured to calculate a second correlation index between a first center representative point and a second center representative point; and a third index calculation unit configured to calculate a third correlation index between the first outer representative point and a third outer representative point. The current representative point may include: a first outer representative point located at a corner of the fragment frame; and a first center representative point located at the center of the fragment frame. The tracking representative point may include: a second outer representative point located at a corner of the tracking frame; and a second center representative point located at the center of the tracking frame. The previous representative point may include: a third outer representative point located outside the associated fragment frame selected at a previous time point.

[0029] For example, the candidate selection unit may include: a first comparison unit configured to compare a first correlation index of each of the plurality of fragment boxes with a first critical index; a second comparison unit configured to compare a second correlation index of each of the plurality of fragment boxes with a second critical index; a third comparison unit configured to compare a third correlation index of each of the plurality of fragment boxes with a third critical index; and a box selection unit configured to select a candidate fragment box of the associated fragment box in response to the comparison results of the first to third comparison units.

[0030] For example, the candidate selection unit may further include an overlap determination unit configured to, in response to a control signal, determine, using a correlation index, whether a segment box not selected as a candidate segment box among a plurality of segment boxes overlaps with the tracking box. The box selection unit may generate a control signal in response to the comparison results of the first to third comparison units, and select candidate segment boxes in response to the determination result of the overlap determination unit.

[0031] For example, the final selection unit may include: a score allocation unit configured to allocate a first score to the distance fitness of each candidate fragment box, a second score to the reliability fitness of each candidate fragment box, and a third score to the correlation between each candidate fragment box and the associated fragment box selected at a previous time point; a score calculation unit configured to add the first to third scores allocated to each candidate fragment box to calculate a final score; and a score comparison unit configured to select the candidate fragment box with the highest final score as the associated fragment box at the current time point t.

[0032] According to another embodiment, a vehicle may include a LiDAR sensor and an object tracking device utilizing the LiDAR sensor.

[0033] According to another embodiment, a recording medium storing a program for performing an object tracking method using a LiDAR sensor may store a program for performing the following functions: a clustering function for grouping LiDAR data consisting of multiple points of an object acquired by the LiDAR sensor; a function for generating information on multiple segment frames for each channel using the clustering results; and a function for selecting the associated segment frame of the target object at the current time t among the multiple segment frames. The selection function may include: a function for calculating a correlation index between the current representative point of each of the multiple segment frames at the current time t and each of the tracking representative point and the previous representative point; a function for selecting candidate segment frames for associated segment frames among the multiple segment frames using the correlation index; and a function for selecting the associated segment frame at the current time t among the selected candidate segment frames. The tracking representative point may correspond to the representative point of the tracking frame of the target object at the current time t estimated using historical information. The previous representative point may correspond to the representative point of the segment frame selected as the associated segment frame at the previous time t-1. Attached Figure Description

[0034] The arrangement and embodiments can be described in detail with reference to the following figures, wherein the same reference numerals refer to the same elements, wherein:

[0035] Figure 1 This is a flowchart illustrating an object tracking method using a LiDAR sensor according to an embodiment;

[0036] Figure 2This is a block diagram illustrating an object tracking device utilizing a LiDAR sensor according to an embodiment;

[0037] Figures 3A to 3C It is a diagram used to illustrate the format of the data processed by the object tracking unit;

[0038] Figure 4 It is used for explanation Figure 1 A flowchart of an embodiment of step 400 is shown;

[0039] Figure 5 It is used for explanation Figure 2 A block diagram of an embodiment of the object tracking unit shown;

[0040] Figure 6 (a) and (b) are diagrams used to illustrate the concepts of the current representative point, the tracking representative point, and the previous representative point;

[0041] Figure 7 yes Figure 5 A block diagram of an embodiment of each of the relevant index calculation unit and candidate selection unit shown;

[0042] Figure 8 yes Figure 4 A flowchart of an embodiment of step 420 is shown;

[0043] Figure 9 This is an illustration showing an example of the overlap between the current frame and the tracking frame;

[0044] Figure 10 (a) through (i) are illustrations showing various examples of fragment boxes and tracking boxes overlapping each other;

[0045] Figure 11 yes Figure 5 A block diagram of an embodiment of the final selection unit shown;

[0046] Figure 12 This is a flowchart illustrating an object tracking method using a LiDAR sensor according to another embodiment;

[0047] Figure 13 (a) is a diagram showing the target object and guardrail being tracked using a LiDAR sensor, and (b) is a diagram illustrating the process of tracking the target object according to the second comparative example.

[0048] Figure 14 Figures (a) to (c) are illustrations used to explain the second comparative example; and

[0049] Figure 15 This is a diagram illustrating the tracking results when association is performed using the object tracking method and apparatus according to the embodiment. Detailed Implementation

[0050] This disclosure will now be described more fully below with reference to the accompanying drawings, which illustrate various embodiments. However, examples may be embodied in many different forms and should not be construed as limiting to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will more fully convey the scope of this disclosure to those skilled in the art.

[0051] What will be understood is that when an element is referred to as being "on" or "below" another element, it can be directly on / below that element, or there may be one or more intermediate elements.

[0052] When an element is referred to as "on" or "below", the term "on" can include both "below the element" and "on the element".

[0053] In addition, relational terms such as “first,” “second,” “upper / upper / above,” and “lower / lower / below” are used only to distinguish one subject or element from another, and do not necessarily require or involve any physical or logical relationship or order between the subjects or elements.

[0054] In the following description, an object tracking method and apparatus 600 utilizing a LiDAR (Light Detection and Ranging) sensor 500, and a vehicle 1000 utilizing the apparatus 600, according to an embodiment, will be described with reference to the accompanying drawings. For ease of description, the object tracking method and apparatus 600 utilizing the LiDAR sensor 500 and the vehicle 1000 utilizing the apparatus 600 will be described using a Cartesian coordinate system (x-axis, y-axis, z-axis). However, the embodiments are not limited thereto. In a Cartesian coordinate system, the x-axis, y-axis, and z-axis are perpendicular to each other, but the embodiments are not limited thereto. That is, the x-axis, y-axis, and z-axis may intersect each other at an angle.

[0055] Figure 1 This is a flowchart illustrating an object tracking method using a LiDAR sensor 500 according to an embodiment. Figure 2 This is a block diagram illustrating an object tracking device 600 utilizing a LiDAR sensor 500 according to an embodiment.

[0056] For ease of description, Figure 1 The object tracking method shown will be described as being by Figure 2 The object tracking device 600 shown performs this operation, but the embodiment is not limited thereto. That is, according to another embodiment, Figure 1 The object tracking method shown can be used by having and Figure 2 The object tracking device 600 shown is executed using object tracking devices with different configurations. Additionally, Figure 2 The object tracking device 600 shown will be described as performing... Figure 1The object tracking method shown is not limited to this embodiment. That is, according to another embodiment, Figure 2 The object tracking device 600 shown can perform operations with... Figure 1 The object tracing methods shown are object tracing methods for different processes.

[0057] Figure 2 The object tracking device 600 utilizing the LiDAR sensor 500 shown may include a clustering unit 620, a shape analysis unit (or fragment unit) 630, and an object tracking unit (tracking unit, tracking and classification unit, or object detection unit) 640. Additionally, the object tracking device 600 may further include a preprocessing unit 610. Furthermore, the vehicle 1000 according to the embodiment may include the LiDAR sensor 500, the object tracking device 600, and a vehicle device 700.

[0058] The LiDAR sensor 500 can radiate a single circular laser pulse with a wavelength of, for example, 905 nm to 1550 nm, towards an object within its measurement range. It can then measure the return time of the laser pulse reflected from the object to detect information about the object, such as the distance from the LiDAR sensor 500 to the object, the object's orientation, its velocity, temperature, material distribution, and concentration characteristics. Here, the object can be, for example, other vehicles, people, or objects located outside the vehicle 1000 (hereinafter referred to as "this vehicle") on which the LiDAR sensor 500 is installed. However, the embodiment is not limited to any particular type of object.

[0059] The LiDAR sensor 500 may include: a transmitter (not shown) that emits laser pulses; and a receiver (not shown) that receives laser light reflected from the surface of an object present within the detection range. The receiver has a predetermined field of view (FOV), which is the range that the LiDAR sensor 500 can observe simultaneously without moving or rotating.

[0060] Because LiDAR sensor 500 exhibits higher detection accuracy in the vertical / horizontal direction compared to radar (RaDAR, radio detection and ranging) sensors, it can provide accurate vertical / horizontal position information, thus facilitating its application in obstacle detection and vehicle location identification. Examples of LiDAR sensor 500 include two-dimensional (2D) LiDAR sensors and three-dimensional (3D) LiDAR sensors. 2D LiDAR sensors are configured to tilt or rotate and are used to acquire LiDAR data including 3D information by tilting or rotating. 3D LiDAR sensors can acquire multiple 3D points to predict obstacle height information, thus aiding in accurate and precise object detection and tracking. 3D LiDAR sensors can be composed of multiple layers of 2D LiDAR sensors, thereby generating LiDAR data including 3D information.

[0061] The LiDAR sensor 500 outputs point cloud data (hereinafter referred to as "LiDAR data") consisting of multiple points of a single object.

[0062] The object tracking method and apparatus 600 according to the embodiments is not limited to any particular shape, location, or type of LiDAR sensor 500.

[0063] The object tracking device 600 can receive LiDAR data and use the LiDAR data to determine the presence or absence of an object, to start, continue or stop tracking the object, to update, store or delete the object's information, and to classify the object's type.

[0064] The preprocessing unit 610 can preprocess the LiDAR data (step 100). To this end, the preprocessing unit 610 can perform calibration to match the coordinates between the LiDAR sensor 500 and the vehicle 1000. That is, the preprocessing unit 610 can convert the LiDAR data into data matched to a reference coordinate system based on the position angle at which the LiDAR sensor 500 is mounted on the vehicle 1000. Additionally, the preprocessing unit 610 can perform filtering using the intensity or confidence information of the LiDAR data to remove points with low intensity or low reflectivity.

[0065] In addition, the preprocessing unit 610 can remove data reflected from the body of the vehicle 1000. That is, since there are areas that are obscured by the body of the vehicle 1000 depending on the installation position and field of view of the LiDAR sensor 500, the preprocessing unit 610 can use a reference coordinate system to remove data reflected from the body of the vehicle 1000.

[0066] In the object tracking method according to the embodiment, step 100 can be omitted. In this case, according to the embodiment, the preprocessing unit 610 can be omitted from the object tracking device 600.

[0067] Following step 100, clustering unit 620 groups the point cloud data, which consists of multiple points of an object acquired by LiDAR sensor 500, into meaningful units according to predetermined criteria (step 200). Without omitting step 100 and preprocessing unit 610 as preprocessing steps, clustering unit 620 can group the LiDAR data preprocessed by preprocessing unit 610. For example, clustering unit 620 can apply vehicle modeling or guardrail modeling to group the point cloud data and then cluster the objects to determine their shapes. The result detected by LiDAR sensor 500 consists of multiple points, each with only location information. Therefore, clustering unit 620 is used to group the multiple points detected by LiDAR sensor 500 into meaningful shape units.

[0068] As an example of clustering unit 620, there are 2D clustering units and 3D clustering units. A 2D clustering unit is a unit that performs clustering on a point or specific structure basis by projecting data onto the XY plane without considering height information. A 3D clustering unit is a unit that performs clustering in the XYZ plane taking into account height information Z.

[0069] Following step 200, shape analysis unit 630 generates information on multiple fragment boxes for each channel using the clustering results from clustering unit 620 (step 300). Here, the fragment box can be the result of converting the clustering results into a geometric box shape. Additionally, the fragment box information can be at least one of the following: width, length, position, or orientation (or heading). Channels will be described later.

[0070] The following description of step 400 according to the embodiments is not limited to the presence or absence of step 100, or the specific process of performing the preprocessing process in step 100, the clustering process in step 200, or the process of generating fragment frame information in step 300. Similarly, the following description of the object tracking unit 640 according to the embodiments is not limited to the presence or absence of preprocessing unit 610, or the specific type of operation performed by preprocessing unit 610, clustering unit 620, or shape analysis unit 630. That is, when preprocessing unit 610 is omitted (i.e., when step 100 is omitted), when preprocessing unit 610 performing step 100 processes LiDAR data in a manner different from that described above, when clustering unit 620 performing step 200 clusters LiDAR data in a manner different from that described above, or when shape analysis unit 630 performing step 300 generates fragment frame information in a manner different from that described above, step 400 and object tracking unit 640 according to the embodiments can also be applied.

[0071] Following step 300, the object tracking unit 640 selects, from among multiple segment frames in each channel, the segment frame associated with the object being tracked at the current time point t (hereinafter referred to as the "target object") (final segment frame or associated segment frame) (step 400). Multiple segment frames can be acquired for the same object based on the visibility of the LiDAR sensor 500 and the shape of the object. Here, the term "association" refers to the process of selecting the segment frame from multiple segment frame information to continue tracking the currently tracked target object. This association can be performed over a predetermined time period.

[0072] In order to select an associated fragment box from the plurality of fragment boxes provided by the shape analysis unit 630 for each channel, the object tracking unit 640 can convert the information of each of the plurality of fragment boxes into a predetermined format, and can select the associated fragment box from the plurality of fragment boxes (or fragment boxes of meta-objects) with the converted format.

[0073] Figures 3A to 3C This is a diagram illustrating the format of the data (i.e., fragment frame information) processed by the object tracking unit 640.

[0074] The object tracking method and apparatus 600 according to the embodiment can track "M" target objects. Here, "M" is a positive integer greater than or equal to 1. That is, the number M of target objects that can be tracked is... Figure 3A The number of tracks (Trk) shown is M. Additionally, the unit storing historical information about a single target object is called a "channel," and the number of channels is the same as the number of tracks (Trk). In this case, the historical information can be the accumulated information about the target object being tracked in each channel up to the current time point t. The historical information can include, for example, periodically measured position and velocity information of the target object.

[0075] Furthermore, for a single target object, at the current time point t, the shape analysis unit 630 can generate "N" fragment boxes Seg#1 to Seg#N and provide these "N" fragment boxes Seg#1 to Seg#N to the object tracking unit 640. Here, "N" is a positive integer greater than 1 and can be the same as or different from "M". In the following text, "N" will be described as a positive integer of 2 or greater, but the following description also applies to the configuration where "N" is 1. That is, as... Figure 3C As shown, the “N” fragment frames Seg#1 to Seg#N can be included in each Trk#m (1≤m≤M) of the first to Mth channels Trk#1 to Trk#M.

[0076] The object tracking device 600 selects the associated segment frame of the target object currently being tracked at the current time point t in each of the N segment frames Seg#1 to Seg#N included in each of the first to M channels (step 400).

[0077] In the following text, for ease of description, the description will include... Figure 3A The process of selecting the associated fragment frame of the currently tracked target object at current time t from among the "N" fragment frames Seg#1 to Seg#N in the m-th channel Trk#m is illustrated. However, the following description can also be applied to the process of selecting the associated fragment frame of the currently tracked target object at current time t from among the "N" fragment frames Seg#1 to Seg#N included in every other channel.

[0078] Figure 4 It is used for explanation Figure 1 The flowchart of embodiment 400A showing step 400 is as follows. Figure 5 It is used for explanation Figure 2 The block diagram shown is of embodiment 640A of the object tracking unit 640.

[0079] For ease of description, Figure 4 Step 400A shown will be described by Figure 5 The object tracking unit 640A shown performs this operation, but the embodiment is not limited thereto. That is, according to another embodiment, Figure 4 Step 400A shown can also be performed by having a connection with Figure 5 The object tracking unit 640A shown is configured to perform object tracking tasks with different configurations. Additionally, Figure 5 The object tracking unit 640A shown will be described as performing... Figure 4 Step 400A is shown, but the embodiment is not limited thereto. That is, according to another embodiment, Figure 5The object tracking unit 640A shown can perform operations with... Figure 4 The steps shown in step 400A are different processes.

[0080] Figure 5 The object tracking unit 640A shown may include a storage unit 642, a correlation index calculation unit 644, a candidate selection unit 646, and a final selection unit 648.

[0081] The correlation index calculation unit 644 calculates the correlation index between the current representative point and each of the tracking representative point and the previous representative point in each of the multiple (i.e., "N") fragment frames Seg#1 to Seg#N included in the m-th channel Trk#m at the current time point t, and outputs the calculated correlation index to the candidate selection unit 646 (step 410). That is, the correlation index calculation unit 644 can calculate the correlation index between the current representative point and the tracking representative point, as well as the correlation index between the current representative point and the previous representative point.

[0082] Figure 6 (a) and (b) are diagrams used to illustrate the concepts of current representative point, tracking representative point, and previous representative point.

[0083] Figure 6 (a) shows each of the multiple fragment frames B at the current time t. t , Figure 6 (b) shows the associated fragment box B selected at time t-1 before the current time t. t-1 .in addition, Figure 6 (b) shows the tracking box TB of the target object estimated using historical information at the current time t. For example, the tracking box TB can be generated by using historical information to estimate tracking information such as the current position, shape, and velocity of the target object being tracked.

[0084] Storage unit 642 can store Figure 3A The historical information for each channel, from Trk#1 to Trk#M, is shown.

[0085] To select the associated fragment box of the target object at the current time t, select box B. t B t-1 The representative points for each element in the TB are crucial for accurately performing "associations." This is because the selection of the associated fragment box is accomplished through comparisons between points.

[0086] The current representative point is each B of multiple fragment frames at the current time t. t The representative point is provided to the relevant index calculation unit 644 by the shape analysis unit 630 via the input terminal IN1. For example, the current representative point may include the point located in box B. tThe representative point at the periphery (or edge) (hereinafter referred to as the "first periphery representative point") and the point located in box B t The representative point at the center (hereinafter referred to as the "first central representative point"). For example, such as Figure 6 As shown in (a), the figure label P m0 P m1 P m2 and P m3 Starting from the bottom left corner and proceeding clockwise, assign each B to one of the multiple fragment frames at the current time t. t The first outermost representative point, and the attached figure labeled P. mc Assigned to the first central representative point located at the center.

[0087] The tracking box is estimated by reading historical information about the target object being tracked from storage unit 642 and using this historical information to determine the location of the associated fragment box of the target object at the current time point t. For this purpose, the tracking box TB information can be stored in storage unit 642, or it can be generated in the correlation index calculation unit 644 using historical information stored in storage unit 642.

[0088] A tracking representative point is a representative point of the tracking frame TB at the current time point t. For example, a tracking representative point can include a representative point located on the periphery (or edge) of the tracking frame TB (hereinafter referred to as the "second periphery representative point") and a representative point located at the center of the tracking frame TB (hereinafter referred to as the "second center representative point"). Figure 6 As shown in (b), the reference numerals P0, P1, P2, and P3 are assigned to the second outer representative points of the tracking frame TB at the current time point t, starting from the lower left corner and proceeding clockwise. The reference numeral P... c Assigned to the second center representative point located at the center.

[0089] The previously represented point was the fragment box B selected as the associated fragment box at time t-1 before the current time t. t-1 The representative point. For example, a previously selected associated fragment box B could be a representative point. t-1 The representative points on the periphery (hereinafter referred to as "third periphery representative points"). For example... Figure 6 As shown in (b), the figure label P is... t-1 m0 P t-1 m1 P t-1 m2 and P t-1 m3 Starting from the bottom left corner and proceeding clockwise, each fragment frame B selected as the associated fragment frame at time t-1 before the current time t is assigned a corresponding fragment frame. t-1 The third peripheral representative point.

[0090] Figure 7 yes Figure 5 The block diagrams shown are of embodiment 644A of the relevant index calculation unit 644 and embodiment 646A of the candidate selection unit 646.

[0091] The correlation index calculation unit 644A may include first, second, and third index calculation units 720, 730, and 740. Additionally, the correlation index calculation unit 644A may also include an overlap determination unit 710.

[0092] The first index calculation unit 720 can use, for example, the following mathematical formula 1 to calculate the segment frame B at the current time t, which is output from the shape analysis unit 630 and provided through the input terminal IN1. t The first correlation index between the first peripheral representative point and the second peripheral representative point can be calculated and output to the candidate selection unit 646 or 646A.

[0093] [Mathematical Expression 1]

[0094]

[0095] Here, γ1 represents the first correlation index, x mo The x-axis coordinate of the first outermost representative point is represented by y. mo The x-axis represents the ordinate of the first outermost representative point. o The x-axis coordinate of the second outermost point is represented by y. o σ represents the ordinate of the second outermost representative point. 2 xm σ represents the variance of the first outermost representative point on the horizontal axis. 2 ym σ represents the variance of the vertical axis of the first outermost representative point. 2 x σ represents the variance of the second outermost representative point on the horizontal axis. 2 y This represents the vertical variance of the second outermost representative point.

[0096] Therefore, the first index calculation unit 720 can receive the second peripheral representative point from the storage unit 642 through the input terminal IN2, or it can generate the second peripheral representative point using historical information received from the storage unit 642 through the input terminal IN2.

[0097] The second index calculation unit 730 can use, for example, the following mathematical formula 2 to calculate the segment frame B at the current time t, which is output from the shape analysis unit 630 and provided through the input terminal IN1. t The first central representative point P mc The second center representative point P of the tracking box TB cThe second correlation index between them can be calculated and output to candidate selection unit 646 or 646A.

[0098] [Mathematical Expression 2]

[0099]

[0100] Here, γ2 represents the second correlation index, x mc The first central representative point P is indicated. mc x-axis coordinate, y mc The first central representative point P is indicated. mc The vertical coordinate, σ 2 xmc The first central representative point P is indicated. mc The variance of the horizontal axis, σ 2 ymc The first central representative point P is indicated. mc The variance of the vertical axis, x c The x-axis coordinate of the second center point is represented by y. c This represents the ordinate of the second central point.

[0101] Therefore, the second index calculation unit 730 can receive the second center representative point from the storage unit 642 via the input terminal IN2, or it can generate the second center representative point using historical information received from the storage unit 642 via the input terminal IN2.

[0102] The third index calculation unit 740 can use, for example, the following mathematical formula 3 to calculate the segment frame B at the current time t, which is output from the shape analysis unit 630 and provided through the input terminal IN1. t The third correlation index is calculated between the first peripheral representative point and the third peripheral representative point provided by the storage unit 642 through the input unit IN2, and the calculated third correlation index can be output to the candidate selection unit 646 or 646A.

[0103] [Mathematical Expression 3]

[0104]

[0105] Here, γ3 represents the third correlation index, x t mo The x-axis coordinate of the first outermost representative point at the current time t is y. t mo This represents the ordinate of the first outermost representative point at the current time t. In equations 1 and 2 above, the coordinate x without the superscript "t" is... mo y mo x o and y oIt represents the coordinates of the current time point t. That is, x in mathematical formula 1... mo With x in mathematical formula 3 t mo Similarly, y in mathematical formula 1 mo With y in mathematical formula 3 t mo Same. However, in mathematical expression 3, x mo Represented as x t mo y mo Represented as y t mo This is to distinguish between the current time point t and the previous time point t-1. Furthermore, x t-1 mo The x-axis coordinate of the third outermost point is represented by y. t-1 mo This represents the ordinate of the third outermost representative point.

[0106] After step 410, the candidate selection unit 646 or 646A can use the correlation index calculated by the correlation index calculation unit 644, namely the first to the third correlation index, to select the candidate fragment box of the associated fragment box from multiple fragment boxes at the current time point t, and can output the information of the selected candidate fragment box to the final selection unit 648 (step 420).

[0107] Figure 8 yes Figure 4 The flowchart of embodiment 420A of step 420 is shown.

[0108] For example, such as Figure 7 As shown, the candidate selection unit 646 or 646A may include first to third comparison units 810, 820 and 830 and box selection unit 840.

[0109] After step 410, the first comparison unit 810 compares the first correlation index γ1 of each of the multiple fragment boxes at the current time point t with the first critical index C1. That is, as shown in the following mathematical formula 4, the first comparison unit 810 determines whether the first correlation index γ1 is less than the first critical index C1, and outputs the determination result to the box selection unit 840 (step 421).

[0110] [Mathematical Expression 4]

[0111] γ1<C1

[0112] When the comparison result of the first comparison unit 810 determines that the first correlation index γ1 is less than the first critical index C1, the box selection unit 840 selects the box with the first correlation index γ1 less than the first critical index C1 from the multiple box segments at the current time point t as the candidate box segment for the associated box segment, and outputs the information of the selected candidate box segment segment to the final selection unit 648 (step 428).

[0113] Additionally, the second comparison unit 820 compares the second correlation index γ2 of each of the multiple fragment boxes at the current time point t with the second critical index C2. That is, as shown in the following mathematical formula 5, the second comparison unit 820 determines whether the second correlation index γ2 is less than the second critical index C2, and outputs the determination result to the box selection unit 840 (step 422).

[0114] [Mathematical Expression 5]

[0115] γ2<C2

[0116] When the comparison result of the second comparison unit 820 determines that the second correlation index γ2 is less than the second critical index C2, the box selection unit 840 selects the box with the second correlation index γ2 less than the second critical index C2 from the multiple box segments at the current time point t as the candidate box segment for the associated box segment, and outputs the information of the selected candidate box segment segment to the final selection unit 648 (step 428).

[0117] Additionally, the third comparison unit 830 compares the third correlation index γ3 of each of the multiple fragment boxes at the current time t with the third critical index C3. That is, as shown in the following mathematical formula 6, the third comparison unit 830 determines whether the third correlation index γ3 is less than the third critical index C3, and outputs the determination result to the box selection unit 840 (step 423).

[0118] [Mathematical Expression 6]

[0119] γ3<C3

[0120] In mathematical formulas 4, 5, and 6, the first, second, and third critical indices C1, C2, and C3 can be preset.

[0121] When the comparison result of the third comparison unit 830 determines that the third correlation index γ3 is less than the third critical index C3, the box selection unit 840 selects the box with the third correlation index γ3 less than the third critical index C3 from the multiple box segments at the current time point t as the candidate box segment for the associated box segment, and outputs the information of the selected candidate box segment segment to the final selection unit 648 (step 428).

[0122] Despite Figure 8As shown, if step 421 is not satisfied, step 422 is executed; if step 422 is not satisfied, step 423 is executed. However, the embodiment is not limited to this. That is, according to another embodiment, steps 421, 422, and 423 can be executed in any order. According to yet another embodiment, steps 421, 422, and 423 can be executed simultaneously. In the case of executing steps 421, 422, and 423 simultaneously, Figure 7 The candidate selection unit 646A shown corresponds to an embodiment of candidate selection unit 646.

[0123] The box selection unit 840 can select candidate fragment boxes of the associated fragment boxes at the current time point t in response to the comparison results of the first to third comparison units 810, 820 and 830. Thus, after step 410, among the multiple fragment boxes, a fragment box that satisfies at least one of the three conditions shown in mathematical formulas 4, 5 and 6 can be selected as a candidate fragment box of the associated fragment box at the current time point t.

[0124] In addition, such as Figure 7 As shown, the candidate selection unit 646A may further include an overlap determination unit 710.

[0125] After steps 421, 422, and 423, determine whether a fragment box that does not satisfy any of the three conditions shown in mathematical formulas 4, 5, and 6 overlaps with the tracking box TB (step 424). If a fragment box overlaps with the tracking box TB, it can be selected as a candidate fragment box (step 428).

[0126] Figure 9 This is the frame B that shows the current time point t. t An example illustration of an overlap with the tracking box TB.

[0127] Steps 424 and 428 can be performed by the overlap determination unit 710 and the box selection unit 840, respectively.

[0128] The box selection unit 840 can generate a control signal CS in response to the comparison results of the first to third comparison units 810, 820, and 830. When it is determined that there is a segment box that has not been selected as a candidate segment box when multiple segment boxes are determined using the correlation index in response to the control signal CS, the overlap determination unit 710 can determine that segment box B. t Whether it overlaps with the tracking box TB, and the determination result can be output to the box selection unit 840 (step 424). For example, as Figure 9 As shown, fragment box B t The tracking box (TB) can overlap with each other.

[0129] The box selection unit 840 can select candidate fragment boxes in response to the determination result of the overlap determination unit 710. That is, when fragment box B is determined by the determination result of the overlap determination unit 710... t When overlapping with the tracking box TB, the box selection unit 840 can select the fragment box B that overlaps with the tracking box TB. t As a candidate fragment box (step 428).

[0130] Figure 10 (a) through (i) are illustrations showing various examples of fragment boxes Bt and tracking boxes TB overlapping each other.

[0131] like Figure 10 As shown in (a) and (b), fragment box B t The tracking box (TB) and the bounding box (TB) can overlap at a single point. For example... Figure 10 As shown in (c), fragment box B t The tracking box (TB) can overlap at two points. For example... Figure 10 As shown in (d), fragment box B t The tracking box (TB) can overlap with each other at three points. For example... Figure 10 As shown in (e), fragment box B t The tracking box (TB) can overlap with each other at four points. For example... Figure 10 As shown in (f), fragment box B t The tracking box (TB) can overlap with each other at five points. For example... Figure 10 As shown in (g), fragment box B t The tracking box (TB) can overlap with each other at six points. For example... Figure 10 As shown in (h), fragment box B t The tracking box (TB) can overlap with each other at seven points. For example... Figure 10 As shown in (i), fragment box B t The tracking box TB can overlap with each other at eight points. Regardless of the number of overlapping points, the fragment box B with points overlapping with the tracking box TB can be... t Selected as candidate fragment boxes. Optionally, according to another embodiment, fragment boxes B whose number of points overlapping only with the tracking box TB is more than a predetermined number. t It can be selected as a candidate fragment box.

[0132] The first to third correlation indices mentioned above are distance-related factors and can be obtained by Euclidean distance, or by Mahalanobis distance as shown in mathematical formulas 1 to 3 above.

[0133] After steps 421, 422 and 423, when a segment frame that does not satisfy any of the three conditions shown in mathematical formulas 4, 5 and 6 does not overlap with the tracking frame among the multiple segment frames at the current time t, it can be determined whether steps 421 to 424 (step 426) have been performed on all segment frames in the first to Nth segment frames included in the m-th channel.

[0134] If steps 421 to 424 have not yet been performed on all fragment frames in the first to Nth fragment frames included in the m-th channel, as described above, steps 421 to 424 and 428 can be performed on the fragment frames for which steps 421 to 424 have not been performed.

[0135] For example, step 426 can be performed by box selection unit 840. That is, whenever steps 421 to 424 are performed, box selection unit 840 can count to determine which of the “N” fragment boxes has performed steps 421 to 424.

[0136] Refer again Figure 4 and Figure 5 After step 420, the final selection unit 648 can select the associated fragment frame at the current time point t from the candidate fragment frames selected by the candidate selection unit 646, and can output the information of the selected associated fragment frame through the output terminal OUT1 (step 430).

[0137] Figure 11 yes Figure 5 The block diagram shown is of embodiment 648A of the final selection unit 648.

[0138] Figure 11 The final selection unit 648A shown may include a fraction allocation unit 910, a fraction calculation unit 920, and a fraction comparison unit 930.

[0139] The score allocation unit 910 can assign a first score SCORE1 to the distance fit of each candidate fragment box, assign a second score SCORE2 to the reliability fit of each candidate fragment box, and assign a third score SCORE3 to each candidate fragment box and the previously selected associated fragment box B. t-1 The correlation between them. To this end, the score allocation unit 910 receives information about candidate segment frames from the candidate selection unit 646 or 646A via the input terminal IN4.

[0140] Regarding the distance fit according to the embodiment, the minimum correlation index among the first to third correlation indices of each candidate fragment box can be compared with a first threshold TV1 to determine the distance fit. That is, among the candidate fragment boxes, the first score SCORE1 can be assigned to the candidate fragment box whose minimum correlation index among the first to third correlation indices is equal to or less than the first threshold TV1.

[0141] Regarding the reliability suitability according to the embodiment, the reliability suitability can be determined using the reliability level of each candidate segment frame. That is, among the candidate segment frames, the second score SCORE2 can be assigned to candidate segment frames with a reliability level greater than the second threshold TV2.

[0142] For example, the steps to obtain the reliability level of each candidate fragment box may include steps to determine the shape reliability level, steps to determine the attribute reliability level, and steps to determine the geometric reliability level.

[0143] In the step of determining the shape reliability level, a second-1 reliability level corresponding to the shape of the candidate fragment frame, such as its size, can be obtained. In the step of determining the attribute reliability level, a second-2 reliability level corresponding to at least one of the ratio of the number of points included in the candidate fragment frame to the size of the candidate fragment frame or the dispersion of the points included in the candidate fragment frame can be obtained. In the step of determining the geometric reliability level, a second-3 reliability level corresponding to the position of the candidate fragment frame and the distance from the reference point to the candidate fragment frame can be obtained. For example, in the case where the LiDAR sensor 500 and the object tracking device 600 are mounted in the vehicle 1000, the reference point can be the position of the vehicle 1000. That is, the second-3 reliability level can be obtained based on the distance from the vehicle 1000 to the candidate fragment frame and whether the candidate fragment frame is located at the boundary of the field of view of the LiDAR sensor 500. The second-1, second-2, and second-3 reliability levels can be preset.

[0144] Subsequently, the reliability levels 2-1, 2-2, and 2-3 can be added together to determine the reliability level to be compared with the second threshold TV2.

[0145] Regarding the correlation according to the embodiment, the correlation can be determined using the ratio of the area of ​​each candidate fragment box overlapping with the previously selected associated fragment box to the total area of ​​each candidate fragment box. Specifically, a third score can be assigned to the correlation in proportion to the ratio of the area of ​​each candidate fragment box overlapping with the previously selected associated fragment box to the total area of ​​each candidate fragment box. That is, the higher the ratio, the higher the third score can be assigned.

[0146] The score calculation unit 920 can add the first to third scores SCORE1, SCORE2 and SCORE3 assigned to each candidate fragment box to calculate the final score TSCORE.

[0147] The score comparison unit 930 can select the candidate segment frame with the highest final score TSCORE from the candidate segment frames as the associated segment frame at the current time point t, and can output the selected associated segment frame through the output terminal OUT1.

[0148] Figure 12 This is a flowchart illustrating an object tracking method 440 using a LiDAR sensor 500 according to another embodiment.

[0149] Figure 1 The step 400 shown may further include Figure 12 The object tracing method shown is 440.

[0150] First, determine whether there is an associated fragment frame for the current time point t (step 442). Step 442 can be executed after step 430, or it can be executed in step 420. If it is determined in step 420 that there is no candidate fragment frame, it can be determined that there is no associated fragment frame for the current time point t. Optionally, when an associated fragment frame for the current time point t is selected in step 430, it can be determined that there is an associated fragment frame for the current time point t.

[0151] When an associated fragment frame exists, the historical information of the m-th channel to which the associated fragment frame belongs can be updated (step 444). For example, the final selection unit 648 can output the information of the associated fragment frame of the target object being tracked in the m-th channel at the current time t to the storage unit 642 through the output terminal OUT1 to update the historical information of the m-th channel.

[0152] However, when no associated fragment box exists, the historical information for the m-th channel that does not have an associated fragment box can be deleted (step 446). For example, when none of the "N" fragment boxes determined to belong to the m-th channel are selected as candidate fragment boxes, Figure 7 The block selection unit 840 shown can output a deletion request signal to the storage unit 642 through the output terminal OUT3, and the storage unit 642 can delete the historical information of the m-th channel in response to the deletion request signal.

[0153] Furthermore, when it is determined that at least one of the N fragment frames belonging to the m-th channel is a fragment frame associated with a new target object, the object tracking unit 640 can store the information of the associated fragment frame of the new target object in an empty channel of the storage unit 642, provided there is an empty channel in the storage unit 642. In this way, the object tracking unit 640 can distinguish between the currently tracked target object and the newly identified target object, and can assign a new identifier (ID) number to the new target object. When the tracked target object disappears, the object tracking unit 640 can reclaim the identifier (ID) number assigned to the disappeared target object.

[0154] in addition, Figure 2 The object tracking unit 640 shown can use information from associated frame frames to distinguish whether the target object is an obstacle, a vehicle, or a person.

[0155] Reference Figure 2 For example, when the object tracking method and apparatus 600 described above are used in vehicle 1000, vehicle 1000 may further include vehicle apparatus 700. Vehicle apparatus 700 can control vehicle 1000 based on information about the target object tracked by object tracking unit 640 and the type of the target object.

[0156] Vehicle device 700 can control vehicle 1000 by receiving object determination information from object tracking device 600. For example, vehicle device 700 may include a lane keeping assist system that maintains a distance between the vehicle and the vehicle in front while preventing the vehicle from deviating from its lane, an obstacle detection system for detecting obstacles around the vehicle, a collision avoidance system for detecting collision risks, an automatic driving system that detects obstacles in front of the vehicle and controls the vehicle to perform automatic driving, and a safety driving system that issues a warning and controls the vehicle to assist in safe driving when another vehicle adjacent to the vehicle approaches.

[0157] The LiDAR sensor 500 may be located at at least one of the front, side, or rear of the vehicle 1000. The object tracking method and apparatus 600 according to the embodiments, and the vehicle 1000 including the apparatus, are not limited to the specific location in which the LiDAR sensor 500 is installed in the vehicle 1000.

[0158] Furthermore, the recording medium containing a program for executing an object tracking method using the LiDAR sensor 500 can store a program that performs the following functions: a clustering function to group LiDAR data consisting of multiple points of an object acquired by the LiDAR sensor 500; a function to generate information on multiple segment frames for each channel using the clustering results; and a function to select the associated segment frame of the target object at the current time t from among the multiple segment frames. The function of selecting the associated segment frame at the current time t can include the following functions: calculating the correlation index between the current representative point and each of the tracking representative point and the previous representative point in each of the multiple segment frames at the current time t; selecting candidate segment frames for associated segment frames from among the multiple segment frames using the correlation index; and selecting the associated segment frame for the current time t from among the selected candidate segment frames. The recording medium can be read by a computer system.

[0159] Computer-readable recording media include all types of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include read-only memory (ROM), random access memory (RAM), optical disc ROM (CD-ROM), magnetic tape, floppy disk, and optical data storage. Computer-readable recording media can also be distributed across network-connected computer systems, thereby storing and executing computer-readable code in a distributed manner. Furthermore, programmers in the art to which this disclosure pertains can readily derive functional programs, code, and code segments for implementing object tracing methods.

[0160] In the following description, the object tracking method according to the comparative example and the object tracking method according to the embodiment will be described with reference to the accompanying drawings.

[0161] In the first and second comparative examples performing "association," it can be determined whether there is a correlation between the predicted tracking box and multiple fragment boxes of the target object being tracked, and information on the correlated fragment boxes can be selected. For example, the presence or absence of correlation can be determined by comparing the points between the boxes. The correlation between the tracking box and the fragment boxes can be determined using Euclidean distance or Mahalanobis distance. When the determined distance meets the allowable boundary values, a correlation can be determined to exist.

[0162] In the first comparative example, the center of the fragment frame is used as a representative point. In this case, the information of the fragment frame cannot accurately indicate the head orientation of the fragment frame, resulting in a head orientation error. The head orientation error in the first comparative example is smaller than the head orientation error in the second comparative example, which will be described later. However, due to the characteristics of the LiDAR sensor 500, when the size of the fragment frame changes to ensure the visibility and shape of the object, the changed size of the fragment frame can be reflected in the error.

[0163] Figure 13 (a) is a diagram showing the target object 10 and the guardrail 12 being tracked using the LiDAR sensor 500. Figure 13 (b) is a diagram illustrating the process of tracking target object 10 according to the second comparative example.

[0164] Figure 14 (a) is a diagram of the tracking frame, where the arrow indicates the direction the head is facing. Figure 14 (b) is an exemplary diagram showing two candidate fragment frames CB1 and CB2 selected by the second comparison example. Figure 14 (c) is a diagram showing the tracking box estimated using historical information updated by the associated fragment box incorrectly selected by the second comparison example.

[0165] In the second comparative example, the center of the rear side of the fragment frame is used as a representative point. For example, according to the second comparative example, the centers of the rear sides of the candidate fragment frames CB1 and CB2 used for association are used as representative points RP1 and RP2. Since the density of the point cloud is higher at the center of the rear side of the fragment frame relative to the mounting position of the LiDAR sensor 500, the second comparative example can provide the measured position in the length direction more stably than the first comparative example when the size of the fragment frame varies according to the shape of the target object. However, since the second comparative example cannot correctly understand the head orientation when generating the fragment frame information, there is a problem of rear-side reference change (e.g., the width and length of the fragment frame switching with each other), which can cause a large error in the measured position. Thus, according to the second comparative example that performs "association" based on the rear side of the fragment frame, when the association is determined using distance, tracking loss may occur due to incorrect association.

[0166] exist Figure 13 In the curved section shown in (a), the guardrail 12 with its curved shape is identified as the larger segment frame CB2. However, this segment frame CB2 is not substantially necessary for maintaining tracking of the target object 10 being tracked. However, when unnecessary segment frames CB2 are selected as associated segment frames and used to update historical information, it is possible to... Figure 14 As can be seen in (c), when the tracking box is predicted based on updated historical information, the head orientation indicated by the arrow does not match the actual head orientation of the target object 10.

[0167] Figure 15 This is a diagram illustrating the tracking results when "association" is performed using the object tracking method and apparatus according to the embodiment.

[0168] According to an embodiment, among multiple fragment frames CB1 and CB2, the best associated fragment frame CB1 that matches the target object 10 being tracked is selected, and the historical information of the target object 10 is updated using this best associated fragment frame CB1. In this case, from Figure 15 It can be seen that when the tracking bounding box (CSB) of the target object 10 is generated using updated historical information, errors in the head orientation, position, and shape of the target object 10 can be minimized. Specifically, according to the embodiment, candidate fragment boxes are first selected using a distance-related factor. At this time, it is determined whether fragment boxes not initially selected as candidate fragment boxes overlap with the tracking box TB, i.e., whether they contact the surface of the tracking box TB, in order to select candidate fragment boxes a second time. Therefore, it is possible to prevent the erroneous selection of associated fragment boxes due to using only a distance factor to select candidate fragment boxes.

[0169] It is evident from the above description that the object tracking method and apparatus using a LiDAR sensor according to the embodiments, the vehicle including the apparatus, and the recording medium storing the program to perform the method are capable of tracking target objects with no error or with minimal error using a LiDAR sensor.

[0170] However, the effects that can be obtained through this disclosure are not limited to those described above, and based on the above description, those skilled in the art will clearly understand other effects not mentioned herein.

[0171] Without departing from the purpose of this disclosure, the various embodiments described above can be combined with each other, unless they are incompatible with each other. Furthermore, for any element not described in detail in any embodiment, reference may be made to the description of an element having the same reference numerals in another embodiment.

[0172] Although this disclosure has been specifically shown and described with reference to exemplary embodiments, these embodiments are presented for illustrative purposes only and are not intended to limit the disclosure. It will be apparent to those skilled in the art that various changes in form and detail may be made without departing from the essential characteristics of the embodiments set forth herein. For example, the various configurations set forth in the embodiments may be modified and applied. Furthermore, such modifications and differences in application should be interpreted as falling within the scope of this disclosure as defined by the appended claims.

Claims

1. An object tracking method using a LiDAR sensor, the method comprising the following steps: Clustering is performed on LiDAR data including multiple points of an object acquired by the LiDAR sensor; Using the clustering results, information on multiple fragment frames for each channel is generated, including historical information; as well as Among the multiple segment boxes, select the segment box associated with the current time point of the target object being tracked. The steps for selecting the associated fragment box include the following: Obtain the correlation index between the current representative point and the tracking representative point and the previous representative point of each of the multiple fragment frames at the current time point; Using the correlation index, candidate fragment boxes of the associated fragment boxes are selected from the plurality of fragment boxes; and Among the selected candidate fragment frames, select the fragment frame associated with the current time point. The tracking representative point corresponds to the representative point of the tracking box of the target object at the current time point estimated using the historical information. The previously representative point corresponds to the representative point of the fragment frame that was selected as the associated fragment frame at a previous time point. The unit that stores historical information about the target object is called the channel. The fragment box is the result of converting the clustering result into a geometric box shape, and The information of the fragment frame includes at least one of the following: the width, length, position, and orientation of the fragment frame.

2. The method according to claim 1, wherein, The current representative point includes: The first outermost representative point is located at the corner of the fragment frame; and The first center point is located at the center of the segment frame. The tracking representative points include: The second outermost representative point is located at the corner of the tracking frame; and The second center point is located at the center of the tracking frame. The previously representative points include: The third outermost point is located outside the associated fragment frame selected at a previous time point.

3. The method according to claim 2, wherein, The relevant indices include: The first correlation index between the first peripheral representative point and the second peripheral representative point; The second correlation index between the first central representative point and the second central representative point; and The third correlation index between the first peripheral representative point and the third peripheral representative point.

4. The method according to claim 3, wherein, The first correlation index, the second correlation index, and the third correlation index are obtained using the LiDAR sensor and calculated using the following mathematical formula. Where γ1 represents the first correlation index, γ2 represents the second correlation index, γ3 represents the third correlation index, and x mo and x t mo The x-axis coordinates of the first outer representative point are represented by y and y respectively. mo and y t mo Let x and y represent the ordinates of the first outermost representative point, respectively. o The x-axis coordinate of the second outer representative point is represented by y. o σ represents the ordinate of the second outermost representative point. 2 xm σ represents the variance of the first outermost representative point on the horizontal axis. 2 ym σ represents the variance of the vertical axis of the first outermost representative point. 2 x σ represents the x-axis variance of the second outer representative point. 2 y The vertical variance of the second outer representative point is represented by x. mc The x-axis coordinate of the first center point is represented by y. mc σ represents the ordinate of the first center point. 2 xmc σ represents the variance of the first center point on the horizontal axis. 2 ymc x represents the variance of the vertical axis of the first center point. c The x-axis coordinate of the second center point is represented by y. c The x-axis represents the ordinate of the second center point. t-1 mo The x-axis coordinate of the third outer representative point is represented by y. t-1 mo The vertical axis coordinate of the third outer representative point is represented, t represents the current time point, and t-1 represents the previous time point.

5. The method according to claim 4, wherein, In the step of selecting the candidate fragment boxes, a fragment box that satisfies at least one of the following three conditions is selected as a candidate fragment box for the associated fragment box. C1, C2, and C3 represent preset critical indices.

6. The method according to claim 5, wherein, The fragment frames that do not meet the three conditions but overlap with the tracking frame are selected as candidate fragment frames.

7. The method according to claim 4, wherein, The steps for selecting the associated fragment box include the following: The first score is assigned to the distance fit of each of the candidate fragment boxes; The second score is assigned to the reliability fit of each candidate fragment box; A third score is assigned to the correlation between each candidate fragment box and the associated fragment box selected at the previous time point; The final score is calculated by summing the first through third scores assigned to each candidate fragment box; and Among the candidate fragment frames, the candidate fragment frame with the highest final score is selected as the associated fragment frame at the current time point.

8. The method according to claim 7, wherein, In the step of assigning the first score, the first score is assigned to a candidate segment box whose smallest correlation index among the first to third correlation indices is equal to or less than the first threshold.

9. The method according to claim 8, wherein, The steps for allocating the second score include the following: Obtain the reliability level of each candidate fragment box; and The second score is assigned to candidate fragment frames that have a reliability level greater than the second threshold.

10. The method according to claim 9, wherein, The steps to obtain the reliability level include the following: The shape reliability level determination step involves obtaining the 2-1 reliability level corresponding to the shape of the candidate fragment box; The attribute reliability level determination step involves obtaining a second-second reliability level corresponding to at least one of the ratio of the number of points included in the candidate fragment frame to the size of the candidate fragment frame or the dispersion of the points included in the candidate fragment frame. The geometric reliability level determination step involves obtaining a second-to-third reliability level corresponding to the position of the candidate fragment box and the distance from the reference point to the candidate fragment box; and The reliability levels 2-1, 2-2, and 2-3 are added together to determine the reliability level to be compared with the second threshold.

11. The method according to claim 9, wherein, In the step of allocating the third score, the third score is allocated proportionally to the ratio of the area overlapping each candidate fragment frame with the associated fragment frame selected at the previous time point to the total area of ​​each candidate fragment frame.

12. The method of claim 1, further comprising the following steps: Determine if there is an associated fragment box selected at the current time. When the existence of the associated fragment frame is determined, the historical information of the channel to which the associated fragment frame belongs is updated; as well as When it is determined that the associated fragment frame does not exist, the historical information of the channel for which the associated fragment frame does not exist is deleted.

13. An object tracking device utilizing a LiDAR sensor, the device comprising: Clustering unit, grouping LiDAR data including multiple points of an object acquired by the LiDAR sensor; The shape analysis unit uses the clustering results to generate information on multiple fragment boxes for each channel, including historical information. as well as The object tracking unit selects the relevant segment frame for the current time point of the target object being tracked from among the multiple segment frames. The object tracking unit includes: Storage unit, storing historical information for each channel; The correlation index calculation unit calculates the correlation index between the current representative point and each of the tracking representative point and the previous representative point for each of the multiple fragment frames at the current time point. The candidate selection unit, using the correlation index, selects a candidate segment box of the associated segment box from among the plurality of segment boxes; and The final selection unit selects the relevant segment frame at the current time point from the selected candidate segment frames. The tracking representative point corresponds to the representative point of the tracking box of the target object at the current time point estimated using the historical information. The previously representative point corresponds to the representative point of the fragment frame that was selected as the associated fragment frame at a previous time point. The unit that stores historical information about the target object is called the channel. The fragment box is the result of converting the clustering result into a geometric box shape, and The information of the fragment frame includes at least one of the following: the width, length, position, and orientation of the fragment frame.

14. The apparatus according to claim 13, wherein, The relevant index calculation unit includes: The first index calculation unit calculates the first correlation index between the first outer representative point and the second outer representative point; The second index calculation unit calculates the second correlation index between the first central representative point and the second central representative point; and The third index calculation unit calculates the third correlation index between the first peripheral representative point and the third peripheral representative point. The current representative point includes: The first outer representative point is located at the corner of the fragment frame; and The first center point is located at the center of the segment frame. The tracking representative points include: The second peripheral representative point is located at the corner of the tracking frame; and The second center point is located at the center of the tracking frame. The previously representative points include: The third peripheral representative point is located outside the associated fragment frame selected at a previous time point.

15. The apparatus according to claim 14, wherein, The candidate selection unit includes: The first comparison unit compares the first correlation index of each of the plurality of fragment frames with the first critical index; The second comparison unit compares the second correlation index of each of the plurality of fragment frames with the second critical index; The third comparison unit compares the third correlation index of each of the plurality of fragment frames with the third critical index; and The box selection unit selects a candidate fragment box of the associated fragment box in response to the comparison results of the first to third comparison units.

16. The apparatus according to claim 15, wherein, The candidate selection unit further includes: The overlap determination unit, in response to the control signal, uses the correlation index to determine whether a fragment box not selected as a candidate fragment box among the plurality of fragment boxes overlaps with the tracking box; and The box selection unit generates the control signal in response to the comparison results of the first to third comparison units, and selects the candidate fragment box in response to the determination result of the overlap determination unit.

17. The apparatus according to claim 16, wherein, The final selection unit includes: The score allocation unit assigns a first score to the distance fit of each candidate fragment box, a second score to the reliability fit of each candidate fragment box, and a third score to the correlation between each candidate fragment box and the associated fragment box selected at the previous time point. The score calculation unit adds the first to third scores assigned to each candidate fragment box to calculate the final score; and The score comparison unit selects the candidate segment box with the highest final score as the associated segment box at the current time point.

18. A vehicle comprising: LiDAR sensor; as well as The object tracking device according to claim 13 utilizes the LiDAR sensor.

19. A computer-readable recording medium having a program recorded thereon for performing an object tracking method using a LiDAR sensor, the recording medium storing the program for performing the following functions: Clustering function to group LiDAR data including multiple points of an object acquired by the LiDAR sensor; The function of generating multiple fragment frames for each channel using the results of clustering, including historical information; as well as Among the multiple segment frames, the function is to select the segment frame associated with the current time point of the target object being tracked. in, The selectable features include: The function of calculating the correlation index between the current representative point and the tracking representative point and the previous representative point of each of the multiple fragment frames at the current time point; Using the relevant index, the function of selecting candidate fragment boxes of the associated fragment boxes from among the plurality of fragment boxes; and The function of selecting the associated fragment frame at the current time point from the selected candidate fragment frames. The tracking representative point corresponds to the representative point of the tracking box of the target object at the current time point estimated using the historical information. The previously representative point corresponds to the representative point of the fragment frame that was selected as the associated fragment frame at a previous time point. The unit that stores historical information about the target object is called the channel. The fragment box is the result of converting the clustering result into a geometric box shape, and The information of the fragment frame includes at least one of the following: the width, length, position, and orientation of the fragment frame.

Images