Method of analyzing the shape of an object and device for tracking an object using a laser radar sensor

Abstract

The present invention relates to a method of analyzing an object shape and an apparatus for tracking an object using a laser radar sensor. The method of analyzing an object shape using a LiDAR sensor includes determining first to Mth shapes of first to Mth layers related to a target object using clustered LiDAR points, where M is a positive integer of 2 or more, and analyzing the determined first to Mth shapes according to a predetermined priority to determine a shape of the target object.

Description

[0001] This application claims priority to Korean Patent Application No. 10-2020-0171561, filed on December 9, 2020, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This invention relates to a method for analyzing the shape of an object and an apparatus for tracking objects using a LiDAR sensor. Background Technology

[0003] Information about the target vehicle can be obtained using LiDAR sensors. This acquired information can be used to assist the autonomous driving functions of vehicles equipped with LiDAR sensors (hereinafter referred to as the "master vehicle"). However, the reliability of the master vehicle may decrease when the information about the target vehicle obtained using LiDAR sensors is incorrect. Therefore, research is underway to address this issue. Summary of the Invention

[0004] Therefore, the present invention aims to provide a method for analyzing the shape of an object using a LiDAR sensor and an apparatus for tracking the object, which essentially eliminates one or more problems caused by the limitations and disadvantages of the prior art.

[0005] This invention provides a method for analyzing the shape of an object using a LiDAR sensor and a device for tracking objects, which can accurately analyze the shape of dynamic objects.

[0006] However, the objectives of this invention are not limited to those mentioned above. Other objectives not mentioned herein should be clearly understood by those skilled in the art from the following description.

[0007] The method for analyzing the shape of an object using a LiDAR sensor according to the implementation scheme may include: (a) determining the shape of all layers from the first layer to the Mth layer (where M is a positive integer of 2 or greater) associated with the target object using clustered LiDAR points; and (b) analyzing the determined shapes according to a predetermined priority to determine the shape of the target object.

[0008] For example, in step (a), the shape of the m-th layer (where 1 ≤ m ≤ M) from the first layer to the M-th layer can be determined. Step (a) may include (a1) searching among the LiDAR points included in the m-th layer for the breakpoint located furthest from the line segment connecting the first endpoint and the second endpoint. Step (a) may also include (a2) assigning shape markers to the m-th layer using at least one of the first line segment connecting the first endpoint and the breakpoint, the second line segment connecting the second endpoint and the breakpoint, the first LiDAR point located near the first line segment, or the second LiDAR point located near the second line segment.

[0009] For example, step (a2) may include analyzing the distribution of the first and second LiDAR points in layer m, and using the analysis results to assign interruption markers to layer m as shape markers. The interruption markers may indicate that the target object included in layer m is less likely to be a dynamic object.

[0010] For example, assigning an interruption marker to the m-th layer may include calculating a first average value of a first distance between a first line segment and a first LiDAR point, and using the first average value to calculate a first variance of the first distance. Assigning an interruption marker to the m-th layer may also include calculating a second average value of a second distance between a second line segment and a second LiDAR point, and using the second average value to calculate a second variance of the second distance. Assigning an interruption marker to the m-th layer may further include assigning an interruption marker to the m-th layer when each of the first variance and the second variance is greater than a variance threshold.

[0011] For example, step (a2) may further include, when either the first variance or the second variance is not greater than a variance threshold, considering the size of the shapebox of the m-th layer including the first line segment and the second line segment, temporarily assigning an L-shaped mark or an I-shaped mark to the m-th layer as a shape mark. Step (a2) may further include, for the L-shaped mark or I-shaped mark temporarily assigned to the m-th layer, using at least one of the first line segment, the second line segment, the first LiDAR point, or the second LiDAR point, finally assigning the L-shaped mark or I-shaped mark to the m-th layer.

[0012] For example, temporarily assigning an L-shaped mark or an I-shaped mark to the m-th layer may include using at least one of the length or width of the shape box to temporarily assign an L-shaped mark or an I-shaped mark to the m-th layer.

[0013] For example, temporarily assigning an L-shaped mark or an I-shaped mark to the m-th layer may include: temporarily assigning an I-shaped mark to the m-th layer when the width of the shape box falls within a first threshold width range; and temporarily assigning an L-shaped mark to the m-th layer when the width of the shape box falls within a second threshold width range. The first threshold width range may have a range from a first minimum value to a first maximum value, and the second threshold width range may have a range from a second minimum value to a second maximum value, wherein the second minimum value may be greater than or equal to the first maximum value.

[0014] For example, assigning an L-shaped marker to the m-th layer may include selecting the longer segment from the first and second segments as a reference segment, or selecting the shorter segment from the first and second segments as a non-reference segment. Assigning an L-shaped marker to the m-th layer may also include: assigning an L-shaped marker to the m-th layer when the length of the reference segment is greater than or equal to a threshold length; when the average and variance of the reference segment are less than the average and variance of the reference threshold, respectively; when the average and variance of the non-reference segment are less than the average and variance of the non-reference threshold, respectively; when a LiDAR point exists in each of the i regions (where i is a positive integer of 1 or greater) formed by dividing the direction intersecting the reference segment; when the distance between adjacent outer LiDAR points located in the regions is less than a threshold distance; and when the angle between the first and second segments is greater than a first angle and less than a second angle.

[0015] For example, ultimately assigning an I-shaped marker to the m-th layer may include selecting the longer segment from the first and second segments as the reference segment. Ultimately assigning an I-shaped marker to the m-th layer may also include assigning the I-shaped marker to the m-th layer when the mean and variance of the reference segment are less than the mean and variance of a reference threshold, respectively, and when the spacing between outer LiDAR points located in j regions (where j is a positive integer of 1 or greater) formed by division in the direction intersecting the reference segment is less than a threshold spacing distance.

[0016] For example, the method may further include checking whether the m-th layer is a top-related layer of the target object. When the m-th layer is a top-related layer of the target object, a non-reference threshold average and a non-reference threshold variance may be added, which may be used in determining whether to ultimately assign an L-shaped label to the (m+1)-th layer.

[0017] For example, checking whether the m-th layer is a top-related layer to the target object may include: checking whether a first ratio of the length of the shape box of the m-th layer to the length of the cluster box with respect to the target object is less than a first threshold ratio; when the first ratio is less than the first threshold ratio, searching for the peak point of the shape tag finally assigned to the m-th layer; and determining that the m-th layer is a top-related layer when a second ratio of the length from the peak point to the middle of the cluster box to half the length of the cluster box is less than a second threshold ratio.

[0018] For example, searching for peak points may include: when L-shaped markers are finally assigned to the m-th layer, identifying the LiDAR point located furthest from the shorter of the first and second line segments as the peak point; when I-shaped markers are finally assigned to the m-th layer, identifying the breakpoint as the peak point.

[0019] For example, determining the shape of a target object based on a predetermined priority may include determining that the shape of the target object cannot be recognized when there is a layer with an interrupt marker assigned among layers one through m. Determining the shape of a target object based on a predetermined priority may also include determining that the shape of the target object is L-shaped when there is no layer with an interrupt marker assigned among layers one through m and there is a layer with an L-shaped marker assigned. Determining the shape of a target object based on a predetermined priority may further include determining that the shape of the target object is I-shaped when there is no layer with either an interrupt marker or an L-shaped marker assigned among layers one through m and there is a layer with an I-shaped marker assigned. Determining the shape of a target object based on a predetermined priority may further include determining that the shape of the target object cannot be recognized when there is no layer with any of the interrupt marker, L-shaped marker, and I-shaped marker assigned among layers one through m.

[0020] An apparatus for tracking objects using a LiDAR sensor according to another embodiment may include: a LiDAR sensor, a clustering unit, and a shape analysis unit; the LiDAR sensor is configured to acquire a point cloud associated with a target object; the clustering unit is configured to cluster the point cloud; and the shape analysis unit is configured to analyze the shape of the target object using the LiDAR points clustered in the point cloud. The shape analysis unit may include: a layer shape determination unit and a target shape determination unit; the layer shape determination unit is configured to determine the shape of all layers from the first layer to the Mth layer (where M is a positive integer of 2 or greater) associated with the target object using the clustered LiDAR points; and the target shape determination unit is configured to analyze the determined shapes according to a predetermined priority to determine the shape of the target object.

[0021] For example, the layer shape determination unit can determine the shape of the m-th layer (where 1 ≤ m ≤ M) from the first layer to the M-th layer. The layer shape determination unit may include a determination preparation unit configured to search among the LiDAR points included in the m-th layer for the breakpoint located furthest from the line segment connecting the first endpoint and the second endpoint, and to generate a first line segment connecting the first endpoint and the breakpoint, and a second line segment connecting the second endpoint and the breakpoint. The layer shape determination unit may also include a mark assignment unit configured to assign shape marks to the m-th layer using at least one of the first line segment, the second line segment, a first LiDAR point near the first line segment, or a second LiDAR point near the second line segment.

[0022] For example, the layer shape determination unit may further include an object analysis unit configured to analyze the distribution of the first LiDAR point and the second LiDAR point in the m-th layer, and to assign interruption markers to the m-th layer as shape markers using the analysis results. The interruption markers may indicate that the target object included in the m-th layer is less likely to be a dynamic object.

[0023] For example, the object analysis unit may include a first variance calculation unit configured to calculate a first variance of the first distance between the first line segment and the first LiDAR point using a first average of the first distance. The object analysis unit may also include a second variance calculation unit configured to calculate a second variance of the second distance between the second line segment and the second LiDAR point using a second average of the second distance. The object analysis unit may further include a variance comparison unit configured to compare each of the first and second variances with a variance threshold, and to assign an interrupt flag to the m-th layer in response to the comparison result.

[0024] For example, the marker allocation unit may include a temporary marker allocation unit configured to, in response to the comparison result of the variance comparison unit, temporarily allocate an L-shaped marker or an I-shaped marker to the m-th layer as a shape marker, considering the size of the shapebox of the m-th layer including the first line segment and the second line segment. The marker allocation unit may also include a final marker allocation unit configured to, for the L-shaped marker or I-shaped marker temporarily allocated to the m-th layer, ultimately allocate the L-shaped marker or I-shaped marker to the m-th layer using at least one of the first line segment, the second line segment, the first LiDAR point, or the second LiDAR point.

[0025] For example, a temporary marker allocation unit can temporarily allocate an L-shaped marker or an I-shaped marker to the m-th layer using at least one of the length or width of the shape box.

[0026] For example, the temporary marker allocation unit may include a first width comparison unit configured to compare the width of the shapebox with a first threshold width range, and temporarily allocate an I-shaped marker to the m-th layer in response to the comparison result. The temporary marker allocation unit may also include a second width comparison unit configured to compare the width of the shapebox with a second threshold width range, and temporarily allocate an L-shaped marker to the m-th layer in response to the comparison result. The first threshold width range may have a range from a first minimum value to a first maximum value, and the second threshold width range may have a range from a second minimum value to a second maximum value. The second minimum value may be greater than or equal to the first maximum value.

[0027] For example, the final label allocation unit may include a reference segment selection unit configured to select the longer segment from the first segment and the second segment as the reference segment, and the shorter segment from the first segment and the second segment as the non-reference segment. The final label allocation unit may also include a first label allocation analysis unit configured to: allocate L-shaped labels to the m-th layer when: the length of the reference segment is greater than or equal to a threshold length; the average and variance of the reference segment are less than the average and variance of the reference threshold, respectively; the average and variance of the non-reference segment are less than the average and variance of the non-reference threshold, respectively; LiDAR points exist in each of the i regions (where i is a positive integer of 1 or greater) formed by dividing the direction intersecting the reference segment; the interval between adjacent outer LiDAR points located in the region is less than a threshold interval distance; and the angle between the first segment and the second segment is greater than a first angle and less than a second angle.

[0028] For example, the final label allocation unit may further include a second label allocation analysis unit configured to ultimately allocate I-shaped labels to the m-th layer when the mean and variance of the reference line segment are less than the reference threshold mean and reference threshold variance, respectively, and when the spacing between outer LiDAR points located in j regions (where j is a positive integer of 1 or greater) formed by partitioning in the direction intersecting the reference line segment is less than the threshold spacing distance.

[0029] For example, the layer shape determination unit may further include a top-level inspection unit configured to check whether the m-th layer is the top-related layer of the target object and output the inspection result. The first label assignment analysis unit may increase the non-reference threshold average and non-reference threshold variance in response to the inspection result of the top-level inspection unit. The non-reference threshold average and non-reference threshold variance may be used in determining whether to ultimately assign an L-shaped label to the (m+1)-th layer.

[0030] For example, the top-level inspection unit can check whether the first ratio of the length of the shape box of the m-th layer to the length of the cluster box about the target object is less than a first threshold ratio, can search for the peak point of the shape tag finally assigned to the m-th layer, and can check whether the second ratio of the length from the peak point to the middle of the cluster box to half the length of the cluster box is less than a second threshold ratio.

[0031] For example, in response to the comparison result of the first ratio and the first threshold ratio and the final assignment result of the shape marker by the final marker assignment unit, the top-level inspection unit can determine the LiDAR point located at the farthest point from the shorter line segment of the first line segment and the second line segment as the peak point, or it can determine the breakpoint as the peak point.

[0032] For example, the target shape determination unit may include a first mark checking unit configured to check whether there are layers with interruption marks assigned in the first to m-th layers. The target shape determination unit may also include a second mark checking unit configured to check whether there are layers with L-shaped marks assigned in response to the check result of the first mark checking unit. The target shape determination unit may further include a third mark checking unit configured to check whether there are layers with I-shaped marks assigned in response to the check result of the second mark checking unit. The target shape determination unit may also include a final shape output unit configured to determine the shape of the target object as an unrecognizable shape, an L-shape, or an I-shape in response to the check results of the first to third mark checking units. Attached Figure Description

[0033] The arrangement and implementation scheme are described in detail with reference to the following figures, wherein the same reference numerals denote the same elements, wherein:

[0034] Figure 1 This is a schematic block diagram of an object tracking device utilizing a LiDAR sensor according to an embodiment of the present invention;

[0035] Figure 2 This is a flowchart of a method for analyzing the shape of an object using a LiDAR sensor according to an embodiment of the present invention;

[0036] Figure 3 yes Figure 1 A block diagram of the implementation scheme of the shape analysis unit shown;

[0037] Figure 4 yes Figure 2 The flowchart of the implementation scheme for step 210 is shown;

[0038] Figure 5 This is an exemplary schematic diagram showing the LiDAR points included in the m-th layer;

[0039] Figure 6 yes Figure 4 The flowchart of the implementation scheme for step 316 is shown;

[0040] Figure 7 yes Figure 4 The flowchart of the implementation scheme for step 318 shown;

[0041] Figure 8(a) and 8(b) It helps to understand Figure 7 A schematic diagram of step 318A is shown;

[0042] Figure 9 yes Figure 4 The flowchart of the implementation scheme for step 320 is shown;

[0043] Figure 10 It helps to understand Figure 9 A schematic diagram of the implementation scheme shown;

[0044] Figure 11 yes Figure 4 A flowchart of another embodiment of step 320 is shown;

[0045] Figure 12 It helps to understand Figure 11 A schematic diagram of the implementation scheme shown;

[0046] Figure 13 yes Figure 4 The flowchart of the implementation scheme for step 322 shown;

[0047] Figure 14 It helps to understand Figure 13 A schematic diagram of step 602 is shown below;

[0048] Figure 15(a) and 15(b) It helps to understand Figure 13 A schematic diagram of step 604 is shown below;

[0049] Figure 16 It helps to understand Figure 13 A schematic diagram of step 606 shown;

[0050] Figure 17 yes Figure 2 The flowchart of the implementation scheme for step 220 is shown;

[0051] Figure 18 Various types of target vehicles based on the host vehicle were displayed;

[0052] Figure 19 This is a schematic diagram illustrating a shape analysis method and an object tracking device based on a comparative example;

[0053] Figures 20(a) to 20(e) This is a schematic diagram used to illustrate the extraction of an object's heading angle;

[0054] Figure 21 It is a schematic diagram used to illustrate the heading direction of the target vehicle;

[0055] Figures 22(a) and 22(b) are schematic diagrams illustrating the association performed by the object tracking device;

[0056] Figure 23 It is a schematic diagram used to illustrate dynamic objects. Detailed Implementation

[0057] The invention will now be described more fully with reference to the accompanying drawings, in which various exemplary embodiments are shown. However, the examples may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be more thorough and complete, and will more fully convey the scope of the invention to those skilled in the art.

[0058] It should be understood that when an element is referred to as being "on" or "below" another element, it can be directly on / below that element, or there can be one or more intermediate elements.

[0059] When an element is referred to as "on" or "below", "below the element" and "on the element" can be included based on that element.

[0060] Furthermore, relational terms such as "first," "second," "upper," "lower," and "below" are used only to distinguish one subject or element from another, and do not necessarily require or involve any actual or logical relationship or order between the subjects or elements. When components, devices, elements, etc., of the present invention are described as having a purpose or performing an operation or function, the components, devices, or elements should be considered as "configured" to satisfy that purpose or perform that operation or function. The present invention describes various components of the object tracking device as units, such as, but not limited to: a layer shape determination unit, a target shape determination unit, a determination preparation unit, an object analysis unit, a marker allocation unit, a top-level inspection unit, a first variance calculation unit, a second variance calculation unit, a variance comparison unit, a temporary marker allocation unit, a final marker allocation unit, first and second width comparison units, a reference line segment selection unit, a first marker allocation analysis unit, a second marker allocation analysis unit, first to third marker inspection units, and a final shape output unit. Each of these units may be implemented individually or may include a processor and memory (e.g., a non-volatile computer-readable medium) as part of the device.

[0061] The following describes, with reference to the accompanying drawings, a method 200 for analyzing the shape of an object using a LiDAR sensor and an apparatus 100 for tracking the object according to an embodiment. For ease of description, the method 200 for analyzing the shape of an object using a LiDAR sensor and the apparatus 100 for tracking the object are described using a Cartesian coordinate system (x-axis, y-axis, z-axis), but other coordinate systems may also be used. In a Cartesian coordinate system, the x-axis, y-axis, and z-axis are perpendicular to each other, but the embodiment is not limited to this. In other words, the x-axis, y-axis, and z-axis may be oblique to each other.

[0062] Figure 1 This is a schematic block diagram of an object tracking device 100 utilizing a LiDAR sensor according to the implementation scheme.

[0063] Figure 1 The object tracking device 100 shown may include a LiDAR sensor 110, a preprocessing unit 120, a clustering unit 130, and a shape analysis unit 140.

[0064] The LiDAR sensor 110 can acquire point clouds related to the target object and output the acquired point clouds as LiDAR data to the preprocessing unit 120.

[0065] The preprocessing unit 120 can preprocess LiDAR data. To this end, the preprocessing unit 120 can perform calibration to match the coordinates of the LiDAR sensor 110 with the coordinates of the vehicle equipped with the LiDAR sensor 110 (hereinafter referred to as the "main vehicle"). In other words, the preprocessing unit 120 can consider the position angle of the LiDAR sensor 110 mounted on the main vehicle and convert the LiDAR data into data suitable for a reference coordinate system. Furthermore, the preprocessing unit 120 can utilize the intensity or confidence information of the LiDAR data to perform filtering to remove points with low intensity or reflectivity. Additionally, the preprocessing unit 120 can remove data reflected from the main vehicle. In other words, since there are areas obstructed by the body of the main vehicle depending on the mounting position and field of view of the LiDAR sensor 110, the preprocessing unit 120 can use the reference coordinate system to remove data reflected from the body of the main vehicle.

[0066] Clustering unit 130 can group the point cloud into meaningful units according to predetermined criteria. The point cloud is LiDAR data composed of multiple points related to an object acquired by LiDAR sensor 110. In other words, clustering unit 130 can cluster the point cloud using the preprocessing results of preprocessing unit 120, and can output the clustered LiDAR points to shape analysis unit 140.

[0067] The shape analysis unit 140 can analyze the shape of the target object using LiDAR points from point cloud clustering, and can output the analysis results through the output terminal OUT1.

[0068] Figure 2 This is a flowchart of a method 200 for analyzing the shape of an object using a LiDAR sensor, according to the implementation plan.

[0069] Figure 1 The shape analysis unit 140 shown can perform... Figure 2 The shape analysis method 200 shown is not limited to this embodiment. In other words, according to another embodiment, Figure 2 The shape analysis method 200 shown can be derived from and Figure 1 The object tracking device 100 shown is executed by object tracking devices configured differently. In other words, Figure 2The method 200 shown is not limited to Figure 1 Any particular type of operation performed by the LiDAR sensor 110 in the illustrated device, the presence or absence of the preprocessing unit 120, any particular type of preprocessing performed by the preprocessing unit 120, or any particular type of clustering performed by the clustering unit 130.

[0070] Figure 3 yes Figure 1 A block diagram of an embodiment 140A of the shape analysis unit 140 shown.

[0071] Hereinafter, in order to better understand the present invention, the object shape analysis method 200 according to the embodiment is described as being composed of... Figure 3 The shape analysis unit 140A shown performs the analysis, but the implementation is not limited to this. In other words, according to another embodiment, the object shape analysis method 200 according to the embodiment can also be performed by... Figure 3 The shape analysis unit 140A shown is configured differently to perform shape analysis.

[0072] Figure 3 The shape analysis unit 140A shown may include a layer shape determination unit 142 and a target shape determination unit 144.

[0073] The layer shape determination unit 142 can receive clustered LiDAR points from the clustering unit 130 via the input terminal IN1, and can use the LiDAR points to determine the first to Mth shapes of the first to Mth layers related to the target object, and can output the determined shapes of the first to Mth layers to the target shape determination unit 144 (step 210). Here, "M" is a positive integer of 1 or greater. For example, "M" can be 6.

[0074] After step 210, the target shape determination unit 144 can determine the shape of the target object by analyzing the first shape to the Mth shape according to a predetermined priority, and can output the determined shape of the target object through the output terminal OUT1 (step 220).

[0075] The following description refers to the attached drawings. Figure 2 The object shape analysis method shown in 200 Figure 3 The layer shape determination unit 142 shown and Figure 3 The implementation scheme of the target shape determination unit 144 shown.

[0076] Figure 4 yes Figure 2 The flowchart of implementation scheme 210A for step 210 is shown.

[0077] Figure 3 The layer shape determination unit 142 shown can perform... Figure 4 The method 210A is shown. For this purpose, the layer shape determination unit 142 may include a determination preparation unit 152 and a mark allocation unit 156. Furthermore, the layer shape determination unit 142 may further include an object analysis unit 154. Furthermore, the layer shape determination unit 142 may further include a top-level inspection unit 158.

[0078] The shape of each of the M layers (i.e., the first layer to the Mth layer) associated with a target object can be determined as follows.

[0079] First, set “m” to 1 (step 310). Here, 1≤m≤M.

[0080] After step 310, among the LiDAR points included in the m-th layer, search for the breakpoint located furthest from the line segment (or baseline) connecting the first endpoint and the second endpoint (step 312).

[0081] Figure 5 This is an exemplary schematic diagram showing the LiDAR points included in the m-th layer.

[0082] For a better understanding of the present invention, please refer to Figure 5 describe Figure 4 The steps shown are 210A, but are not limited thereto.

[0083] LiDAR points associated with a target object can be divided into M layers (i.e., the first layer to the Mth layer) in the vertical direction (e.g., the z-axis direction).

[0084] After step 310, the LiDAR points included in the m-th layer (e.g., Figure 5 In the p1 to p10 shown, search for the breakpoint B (p4) that is furthest from the line segment EL that connects the first endpoint A (p1) and the second endpoint C (p10) (step 312).

[0085] Subsequently, a first line segment L1 connecting the first endpoint p1 and the breakpoint p4, and a second line segment L2 connecting the second endpoint p10 and the breakpoint p4 are generated (step 314).

[0086] Steps 310 to 314 above can be performed by Figure 3 The determination and preparation unit 152 shown is executed.

[0087] After step 314, the object analysis unit 154 can analyze the distribution pattern of the first LiDAR point and the second LiDAR point in the m-th layer, and can use the analysis results to determine whether to assign the interruption mark to the m-th layer as a shape mark (step 316).

[0088] Here, the first LiDAR point can be a LiDAR point (e.g., Figure 5 The LiDAR points (e.g., p2 and p3) located near the first line segment L1 in the p1 to p10 diagram shown. The second LiDAR point can be a LiDAR point (e.g., ...). Figure 5 The LiDAR points (e.g., p1 to p10) located near the second line segment L2 are shown in the diagram. Figure 5 (as shown in pages 5 to 9).

[0089] An interruption marker can be one that indicates a low probability that a target object displayed by the LiDAR points included in the m-th layer is a dynamic object. In other words, when the LiDAR points are widely dispersed in the m-th layer, there is a possibility that the target object is a static object rather than a dynamic object. Therefore, the variance of the LiDAR points can be used to check whether a target object is dynamic or static.

[0090] Figure 6 yes Figure 4 The flowchart of implementation scheme 316A for step 316 is shown.

[0091] Object analysis unit 154 can execute Figure 6 The step 316A is shown. For this purpose, the object analysis unit 154 may include, for example, a first variance calculation unit 162, a second variance calculation unit 164, and a variance comparison unit 166.

[0092] For example, refer to Figure 3 and Figure 6 After step 314, the first variance calculation unit 162 calculates the first average value A1 of the first distance between the first line segment L1 and the first LiDAR point, as shown in Equation 1 below (step 420).

[0093] [Equation 1]

[0094]

[0095] Here, "n" represents the total number of the first LiDAR points included in the m-th layer, and "xi" represents the first distance. (See reference) Figure 5 “xi” corresponds to the distance between each of the first LiDAR points p2 and p3 and the first line segment L1 in the x-axis direction.

[0096] After step 420, the first variance calculation unit 162 uses the first average value A1 of the first distance xi to calculate the first variance V1 of the first distance xi, as expressed by Equation 2 below, and outputs the calculated first variance V1 to the variance comparison unit 166 (step 422).

[0097] [Equation 2]

[0098]

[0099] After step 422, the second variance calculation unit 164 calculates the second average value A2 of the second distance yi between the second line segment L2 and the second LiDAR point, as shown in Equation 3 below (step 424).

[0100] [Equation 3]

[0101]

[0102] Here, "q" represents the total number of second LiDAR points included in the m-th layer, and "yi" represents the second distance. (Reference) Figure 5 “yi” corresponds to the distance between each of the second LiDAR points p5 to p9 and the second line segment L2 in the y-axis direction.

[0103] After step 424, the second variance calculation unit 164 uses the second average value A2 of the second distance yi to calculate the second variance V2 of the second distance yi, as shown in Equation 4 below, and outputs the calculated second variance V2 to the variance comparison unit 166 (step 426).

[0104] [Equation 4]

[0105]

[0106] After step 426, variance comparison unit 166 determines whether the first variance V1 is greater than the first variance threshold VE1 (step 428). If the first variance V1 is greater than the first variance threshold VE1, then variance comparison unit 166 determines whether the second variance V2 is greater than the second variance threshold VE2 (step 430). When it is determined that the second variance V2 is greater than the second variance threshold VE2, variance comparison unit 166 can assign an interrupt flag to the m-th layer, and the process can proceed to step 220 (step 432). According to the embodiment, the first variance threshold VE1 and the second variance threshold VE2 can be the same as or different from each other. Furthermore, each of the first variance threshold VE1 and the second variance threshold VE2 can be preset and stored for each layer, or they can be preset as constant values ​​regardless of the layer. For example, each of VE1 and VE2 can be 1.5.

[0107] However, when the first variance V1 is not greater than the first variance threshold VE1 or when the second variance V2 is not greater than the second variance threshold VE2, the m-th layer is more likely to be a dynamic object rather than a static object, thus the process proceeds to step 318. For example, a static object can be an inanimate object, such as a traffic light, a tree, a traffic sign, or a guardrail, while a dynamic object can be a moving object, such as a vehicle.

[0108] As described above, the variance comparison unit 166 compares the first variance V1 and the second variance V2 with the first variance threshold VE1 and the second variance threshold VE2, respectively, and assigns an interrupt flag to the m-th layer in response to the comparison result.

[0109] Alternatively, step 316 can be omitted in the object shape analysis method 200 using a LiDAR sensor according to the implementation scheme.

[0110] Meanwhile, when no interruption mark is assigned to the m-th layer, the mark assignment unit 156 can assign a shape mark to the m-th layer using at least one of the first line segment, the second line segment, the first LiDAR point, or the second LiDAR point (steps 318 and 320).

[0111] Therefore, such as Figure 3 As shown, the tag allocation unit 156 may include a temporary tag allocation unit 170 and a final tag allocation unit 180.

[0112] In response to the comparison result of the variance comparison unit 166, the temporary label allocation unit 170, considering the size of the shape box of the m-th layer including the first line segment L1 and the second line segment L2, temporarily assigns an L-shaped label or an I-shaped label to the m-th layer as a shape label. See below for reference. Figure 14 The shape box is described in detail. In other words, when it is identified that the first variance V1 is not greater than the first variance threshold VE1 and / or the second variance V2 is not greater than the second variance threshold VE2 as the comparison result of the variance comparison unit 166, and no interruption mark is assigned to the m-th layer, the temporary mark assignment unit 170 may consider the size of the shape box of the m-th layer including the first line segment and the second line segment, and temporarily assign an L-shaped mark or an I-shaped mark to the m-th layer as a shape mark (step 318).

[0113] Figure 7 yes Figure 4 The flowchart of implementation scheme 318A for step 318 is shown. Figures 8(a) and 8(b) are for understanding. Figure 7 The diagram shows step 318A.

[0114] For example, the temporary mark allocation unit 170 can temporarily allocate an L-shaped mark or an I-shaped mark to the m-th layer using at least one of the length or width of the shape box of the m-th layer (e.g., SB shown in FIG8(b)).

[0115] In other words, when it is identified that no interruption flag has been assigned to the m-th layer, it can be determined whether the width of the shape box of the m-th layer falls within the first threshold width range TWR1 or the second threshold width range TWR2 (step 440).

[0116] If the width of the shape box in layer m falls within the first threshold width range TWR1, then an I-shaped mark is temporarily assigned to layer m (step 442). However, if the width of the shape box falls within the second threshold width range, then an L-shaped mark is temporarily assigned to layer m (step 444).

[0117] When the first threshold width range TWR1 has a range from a first minimum value MIN1 to a first maximum value MAX1, and the second threshold width range TWR2 has a range from a second minimum value MIN2 to a second maximum value MAX2, the second minimum value MIN2 can be greater than or equal to the first maximum value MAX1. For example, the first threshold width range TWR1 can be 0m to 1m, and the second threshold width range TWR2 can be 1m to 8m. Thus, shape markers can be assigned to the m-th layer using the width of the shape box SB, but the implementation is not limited to this. In other words, according to another embodiment, shape markers can be temporarily assigned to the m-th layer using at least one of the width or length of the shape box SB.

[0118] According to the implementation scheme, device 100 can track objects with a length of 13 meters or less. When the length of shapebox SB shown in Figure 8(b) falls within the range of 1m to 13m, if the width of shapebox SB falls within the first threshold width range TWR1 of 0m to 1m, an I-shaped mark can be temporarily assigned to the m-th layer. If the width of shapebox SB falls within the range of 1m to 8m, an L-shaped mark can be temporarily assigned to the m-th layer.

[0119] In order to execute Figure 7 The method shown is as follows: Figure 3 As shown, the temporary marker allocation unit 170 may include a first width comparison unit 172 and a second width comparison unit 174.

[0120] In summary, the temporary mark allocation unit 170 can temporarily allocate an L-shaped mark or an I-shaped mark to the m-th layer using at least one of the length or width of the shape box SB (step 318A).

[0121] The first width comparison unit 172 can compare the width of the shape box SB with the first threshold width range TWR1, and can temporarily assign an I-shaped mark to the m-th layer in response to the comparison result. Additionally, the second width comparison unit 174 can compare the width of the shape box SB with the second threshold width range TWR2, and can temporarily assign an L-shaped mark to the m-th layer in response to the comparison result.

[0122] After step 318, the final marker allocation unit 180 may use at least one of the first line segment L1, the second line segment L2, the first LiDAR point, or the second LiDAR point to determine whether to finally allocate the L-shaped marker or I-shaped marker that has been temporarily allocated to the m-th layer to the m-th layer (step 320).

[0123] Figure 9 yes Figure 4 The flowchart of implementation scheme 320A for step 320 is shown. Figure 10 It helps to understand Figure 9 The diagram shown is a schematic representation of embodiment 320A. Figure 10 In this context, it is assumed that the first line segment L1 and the second line segment L2 correspond to the first line segment L1 and the second line segment L2 obtained in step 314, respectively.

[0124] Figure 11 yes Figure 4 The flowchart of another embodiment 320B of step 320 shown is as follows. Figure 12 It helps to understand Figure 11 The schematic diagram shown is of implementation scheme 320B.

[0125] When the L-shaped marker is temporarily assigned to the m-th layer in step 318, it can be done through... Figure 9 The method 320A shown ultimately assigns the shape mark to the m-th layer (step 320A). However, when the I-shaped mark is temporarily assigned to the m-th layer in step 318, it can be done through... Figure 11 The method 320B shown ultimately assigns shape marks to the m-th layer (step 320B).

[0126] In order to execute Figure 9 and Figure 11 The illustrated embodiments 320A and 320B show that the final marker allocation unit 180 may include, for example, a reference line segment selection unit 182, a first marker allocation analysis unit 184, and a second marker allocation analysis unit 186, such as... Figure 3 As shown.

[0127] When the comparison result based on the first width comparison unit 172 identifies that the L-shaped mark has been assigned to the m-th layer, the first mark assignment analysis unit 184 can execute... Figure 9 Steps 462 to 488 are shown. However, Figure 9 Step 462 shown can be performed by the reference segment selection unit 182 instead of by the first mark assignment analysis unit 184.

[0128] After step 318, the reference segment selection unit 182 selects the longer line segment from the first line segment L1 and the second line segment L2 provided by the preparation unit 152 as the reference line segment. The reference segment selection unit 182 then selects the shorter line segment from the first line segment L1 and the second line segment L2 as the non-reference line segment (step 460). For example, the reference... Figure 10 Since the second line segment L2 is longer than the first line segment L1, the first line segment L1 can be selected as the non-reference line segment, and the second line segment L2 can be selected as the reference line segment.

[0129] After step 460, check whether the length RL of the reference line segment (e.g., L2) is greater than or equal to the threshold length TL (step 462). From this point on, the threshold length can be set differently for each layer of M, or it can be set the same for all layers.

[0130] If the length RL of the reference segment is neither greater than nor equal to the threshold length TL, then the shape label is not assigned to the m-th layer (step 488). However, when the length RL of the reference segment is greater than or equal to the threshold length TL, the mean and variance of each of the reference and non-reference segments are calculated (step 464).

[0131] Here, the average value of the reference segment refers to the average distance between the reference segment and LiDAR points located near the reference segment. The variance of the reference segment refers to the variance of the distance between the reference segment and LiDAR points located near the reference segment. The average value of the non-reference segments refers to the average distance between the non-reference segments and LiDAR points located near the non-reference segments. The variance of the non-reference segments refers to the variance of the distance between the non-reference segments and LiDAR points located near the non-reference segments.

[0132] After step 464, check whether the average value AARL and variance AVRL of the reference line segment are less than the average value AARm and the variance AVRm of the reference threshold, respectively (step 466). Here, each of the average value AARm and the variance AVRm of the reference threshold can be preset and stored for each set of coordinates in the m-th layer, or they can be preset as constant values ​​and stored without considering the coordinates of the m-th layer.

[0133] If the average value of the reference line segment AARL is not less than the average value of the reference threshold AARm, or if the variance of the reference line segment AVRL is not less than the variance of the reference threshold AVRm, then the shape label is not assigned to the m-th layer (step 488).

[0134] However, if the average value AARL of the reference line segment is less than the average value AARm of the reference threshold, and the variance AVRL of the reference line segment is less than the variance AVRm of the reference threshold, then it is checked whether the average value AANRL and the variance AVNRL of the non-reference line segment are less than the average value AANRm and the variance AVNRm of the non-reference threshold, respectively (step 468). Here, each of the average value AANRm and the variance AVNRm of the non-reference threshold can be preset and stored for each set of coordinates in the m-th layer, or they can be preset as constant values ​​and stored without considering the coordinates of the m-th layer.

[0135] If the average value of the non-reference line segment AANRL is not less than the average value of the non-reference threshold AANRm, or if the variance of the non-reference line segment AVNRL is not less than the variance of the non-reference threshold AVNRm, then the shape label is not assigned to the m-th layer (step 488).

[0136] However, when the average value of the non-reference line segment AANRL is less than the average value of the non-reference threshold AANRm, and the variance of the non-reference line segment AVNRL is less than the variance of the non-reference threshold AVNRm, the region associated with the reference line segment is divided into i regions in the direction intersecting with the reference line segment (step 470). Here, "i" is a positive integer of 1 or greater. For example, "i" could be 4. For example, the reference... Figure 10 It can be seen that the region related to the reference line segment L2 is divided into four regions (i=4) AR1 to AR4 in the direction perpendicular to the reference line segment L2. To divide the region, three lines (i-1=3) can be arranged for orientation in the direction perpendicular to the reference line segment L2.

[0137] After step 470, check whether a LiDAR point exists in each of the i regions generated by the partitioning (step 480). For example, in Figure 10 In the diagram, LiDAR points ip1 to ip5 located to the left of the second line segment L2 are called "inner LiDAR points," and LiDAR points op3 to op7 located to the right of the second line segment L2 are called "outer LiDAR points." In this case, after step 470, it is checked whether there are inner or outer LiDAR points in each of the four regions AR1 to AR4 generated by the division (step 480). If there are no LiDAR points in any of the four regions generated by the division, the shape marker is not assigned to the m-th layer (step 488).

[0138] However, when LiDAR points exist in each region generated by the partitioning, it is checked whether the interval distance SD between adjacent outer LiDAR points located in the partitioned regions is less than the threshold interval distance d (step 482). Here, the threshold interval distance d can be preset. Figure 10In this process, the lines connecting adjacent outer LiDAR points can be line segments. For example, each line connecting outer LiDAR points op1 and op2, op3 and op4, op4 and op5, op5 and op6, and op6 and op7 can be a line segment. In step 482, it is determined whether the length SD of the line segment is less than the threshold interval distance d. If the length SD of the line segment is not less than the threshold interval distance d, then the shape mark is not assigned to the m-th layer (step 488).

[0139] However, when the length SD of a line segment is less than the threshold interval distance d, it is determined whether the angle θ12 between the first line segment L1 and the second line segment L2 is greater than the first angle θ1 and less than the second angle θ2 (step 484). Here, the first angle θ1 and the second angle θ2 can be preset for each layer, or they can be preset as constant values ​​regardless of the layer. If the angle θ12 between the first line segment L1 and the second line segment L2 is less than the first angle θ1 or greater than the second angle θ2, then the shape mark is not assigned to the m-th layer (step 488).

[0140] However, if the angle θ12 between the first line segment L1 and the second line segment L2 is greater than the first angle θ1 and less than the second angle θ2, then the L-shaped mark is finally assigned to the m-th layer (step 486).

[0141] When the comparison result based on the second width comparison unit 174 identifies that the I-shaped mark has been assigned to the m-th layer, the second mark assignment analysis unit 186 can perform... Figure 11 Steps 492 to 502 are shown.

[0142] First, refer to Figure 11 Select the longer line segment of the first line segment L1 and the second line segment L2 as the reference line segment (step 490). Since step 490 is the same as step 460, its description is omitted.

[0143] After step 490, determine the reference line segment ( Figure 12 The average value ABRL and variance BVRL of L2 shown are checked against whether they are less than the average value ABRm and variance BVRm of the reference threshold, respectively (steps 494 and 496). Here, the average value ABRm and variance BVRm of the reference threshold can be compared with... Figure 9 The reference threshold average value AARm and reference threshold variance AVRm shown are the same. The reference threshold average value ABRm and reference threshold variance BVRm can be preset and stored for each layer of M layers, or they can be preset as constant values ​​and stored regardless of the layers.

[0144] If the average value ABRL is not less than the reference threshold average value ABRm, or if the variance BVRL is not less than the reference threshold variance BVRm, then shape tags are not assigned to the m-th layer (step 502). However, if the average value ABRL is less than the reference threshold average value ABRm, and the variance BVRL is less than the reference threshold variance BVRm, then it is determined whether the spacing distance SD between the outer LiDAR points located in the j regions (the j regions are generated by the division in the direction intersecting the reference line segment) is less than the threshold spacing distance d (step 498). Here, "j" is a positive integer of 1 or greater. "j" can be the same as or different from "i". Since step 498 is the same as step 482, a repeated description of it is omitted.

[0145] If the interval distance SD is not less than the threshold interval distance d, then the shape mark is not assigned to the m-th layer (step 502). However, if the interval distance SD is less than the threshold interval distance d, then the I-shaped mark is finally assigned to the m-th layer (step 500).

[0146] At the same time, refer to again Figure 4 Step 210A may further include step 322, which is performed after step 320. In other words, Figure 3 The layer shape determination unit 142 shown may further include a top-level check unit 158 ​​that performs step 322. In some embodiments, step 322 and the top-level check unit 158 ​​may be omitted.

[0147] After step 320, the top-level inspection unit 158 ​​can check whether the m-th layer is the top-related layer of the target object (hereinafter referred to as the "top layer"), and can output the inspection result to the first mark assignment analysis unit 184 and the determination preparation unit 152 (step 322). If the m-th layer is the top layer of the target object, step 320 can be added (i.e., Figure 9 The non-reference threshold average AANRm and non-reference threshold variance AVNRm for the (m+1)th layer are shown in step 468 (step 324). Thus, by increasing the non-reference threshold average AANRm and non-reference threshold variance AVNRm, the conditions considered for assigning L-shaped markers to the (m+1)th layer can be relaxed. The aim is to reflect the structural characteristics of a target vehicle, where the front bumper is more rounded than the rear bumper, when determining whether to assign an L-shaped marker to the (m+1)th layer when the target object is a vehicle.

[0148] Step 324 can be performed by the first marker assignment analysis unit 184, the top-level inspection unit 158, or the determination preparation unit 152.

[0149] Figure 13 yes Figure 4 The flowchart of implementation scheme 322A for step 322 is shown; Figures 14 to 16 It helps to understand Figure 13 The diagram shows step 322A.

[0150] exist Figure 14 In this context, the cluster box CB can be a box that includes LiDAR points associated with layers 1 to M, and the shape box SB can be a box that includes LiDAR points associated with layer m.

[0151] According to the implementation plan, top-level inspection unit 158 ​​can perform... Figure 13 Steps 602 to 610 are shown.

[0152] First, refer to Figure 14 As shown in Equation 5 below, determine whether the first ratio R1 of the length XC of the shape box SB of the m-th layer to the length XCL of the cluster box CB about the target object is less than the first threshold ratio Rt (step 602).

[0153] [Equation 5]

[0154]

[0155] Here, "XC / XCL" represents the first ratio R1. The first threshold ratio Rt can be preset for each layer of M, or it can be preset to a constant value regardless of the M layers.

[0156] If the first ratio R1 is less than the first threshold ratio Rt, then a peak point is searched based on each shape marker finally assigned to the m-th layer (step 604). For example, the top-level inspection unit 158 ​​may, in response to the comparison result of the first ratio R1 and the first threshold ratio Rt and the final assignment result of the shape markers by the final marker assignment unit 184, determine the LiDAR point located at the furthest point between the first and second line segments as the peak point, or the breakpoint may be determined as the peak point.

[0157] Specifically, when assigning L-shaped markers to the m-th layer, referring to Figure 15(a), the LiDAR point pp1, located furthest from the shorter of the first line segment L1 and the second line segment L2 (L1 in Figure 15(a)), can be identified as the peak point. Alternatively, when assigning I-shaped markers to the m-th layer, as shown in Figure 15(b), the breakpoint (the point located in region B) pp2 can be identified as the peak point.

[0158] After step 604, it can be checked whether the second ratio of the length from the peak point to the middle of the cluster box to half the length of the cluster box is less than the second threshold ratio (step 606). For example, when the L-shaped marker is finally assigned to the m-th layer as shown in Figure 15(a) and the peak point is determined to be pp1, as... Figure 16As shown in Equation 6 below, we can check whether the second ratio R2 of the length d1 from the peak point pp1 to the middle of the cluster box CB to half the length d2 of the cluster box CB is less than the second threshold ratio R. A .

[0159] [Equation 6]

[0160]

[0161] Here, "d1 / d2" represents the second ratio R2. The second threshold ratio R can be preset for each layer of M layers. A Alternatively, the M layer can be disregarded and pre-set to a constant value.

[0162] If the second ratio R2 is less than the second threshold ratio R A If the first ratio R1 is not less than the first threshold ratio Rt, or if the second ratio R2 is not less than the second threshold ratio Rt, then the m-th layer is determined to be the top layer of the target object (step 608). A If so, then it is determined that the m-th layer is not the top layer of the target object (step 610).

[0163] Refer again Figure 4 After determining that the m-th layer is not the top layer of the target object, or after step 324, it is checked whether m is M (step 326). If m is not M, m is incremented by 1, and the process proceeds to step 312 (step 328). Accordingly, steps 312 to 324 are performed on the (m+1)-th layer. In other words, the shape mark is assigned to the (m+1)-th layer in the same way as the shape mark was assigned to the m-th layer. For example, steps 326 and 328 can be performed by the determination preparation unit 152, but the implementation is not limited to this.

[0164] Figure 17 yes Figure 2 The flowchart of implementation scheme 220A for step 220 is shown.

[0165] In order to execute Figure 17 The illustrated implementation scheme 220A, such as Figure 3 As shown, the target shape determination unit 144 may include first to third mark checking units 192, 194, 196 and a final shape output unit 198.

[0166] The first mark checking unit 192 checks whether there are any layers with interrupt marks assigned among the first layer to the m-th layer, and outputs the checking result to the final shape output unit 198 (step 630). When the first mark checking unit 192 identifies that there are layers with interrupt marks assigned among the first layer to the m-th layer, the final shape output unit 198 determines that the shape of the target object cannot be identified (or is unknown) (step 632).

[0167] When the first mark checking unit 192 identifies that no layer with an interrupt mark is assigned, the second mark checking unit 194 checks whether a layer with an L-shaped mark is assigned and outputs the check result to the final shape output unit 198 (step 634). When the first mark checking unit 192 and the second mark checking unit 194 identify that no layer with an interrupt mark is assigned among the first to m layers, but a layer with an L-shaped mark is assigned, the final shape output unit 198 determines that the target object has an "L" shape (step 636).

[0168] When the second mark checking unit 194 identifies that no layer has been assigned an L-shaped mark, the third mark checking unit 196 checks whether a layer has been assigned an I-shaped mark and outputs the check result to the final shape output unit 198 (step 638). When the first to m-th layers are identified as having no layer assigned either an interrupt mark or an L-shaped mark, but have a layer assigned an I-shaped mark, based on the check results of the first to third mark checking units 192, 194, and 196, the final shape output unit 198 determines that the target object has an "I" shape (step 640).

[0169] As described above, when the final shape output unit 198 determines the shape of the target object, it checks the interrupt marker, L-shaped marker, and I-shaped marker in this order.

[0170] However, when the first to m layers are identified as not having any of the interrupt markers, L-shaped markers, and I-shaped markers assigned based on the inspection results of the first to third marker inspection units 192, 194, and 196, the final shape output unit 198 determines that the shape of the target object cannot be identified (step 632).

[0171] Figure 18 Various types of target vehicles 710 to 716 based on the main vehicle 700 are shown.

[0172] refer to Figure 18 Target vehicle 716, whose side surface is scanned only from the main vehicle 700, has an I-shaped profile (or I-shaped outer profile). Target vehicles 710, 712, and 714, whose side surface and bumper are scanned from the main vehicle 700, have L-shaped profiles (or L-shaped outer profiles). Dynamic objects in urban areas or on highways mainly have L-shaped or I-shaped profiles. Therefore, in step 318, it is possible to temporarily determine whether an object has an I-shaped or L-shaped profile based on the dimensions of the shape box SB with the profile shape.

[0173] The outline (or shape) of a target vehicle, which is the target object, can be determined by utilizing the object tracking device 100 of the LiDAR sensor according to the above embodiment and the object shape analysis method 200. For example, refer to Figure 18 Target vehicles 710, 712, and 714 can be determined to have L-shaped profiles, while target vehicle 716 can be determined to have I-shaped profiles. In this case, the object tracking device 100 according to the embodiment can use the determined profiles of the target vehicles to identify the heading directions of target vehicles 710 to 716. For example, when target vehicles 710 to 716 have L-shaped and I-shaped profiles, the heading directions of target vehicles 710 to 716 can be parallel to the longer of the first and second line segments (e.g., Figure 18 (HD1, HD2, HD3, and HD4 are shown).

[0174] Hereinafter, comparative examples and embodiments of the present invention will be described with reference to the accompanying drawings.

[0175] Figure 19 This is a schematic diagram used to illustrate a shape analysis method and an object tracking device based on a comparative example.

[0176] In the comparative example, the breakpoint BP located furthest from the baseline 800 is searched. Then, the heading direction of the object is determined based on the longer of the first line segment L1 and the second line segment L2. Next, among the four corner points of the bounding box 802, the corner point 804 located closest to the breakpoint BP is identified as the corner point with an L-shaped feature. Then, the object is tracked using the first line segment L1, the second line segment L2, corner point 804, and the angle θ formed by the first line segment L1 and the second line segment L2 (e.g., the heading angle of the object is extracted). However, this comparative example of analyzing object shape using this method has limitations in determining the shape of static objects other than dynamic objects.

[0177] Figures 20(a) to 20(e) This is a schematic diagram used to illustrate the extraction of an object's heading angle.

[0178] Figure 20(a) is a two-dimensional projection of the object. Figures 20(b) to 20(e) The outline layers 0, 1, 2, and 3 of the object are shown respectively.

[0179] Figure 21 It is a schematic diagram used to illustrate the heading direction of the target vehicle.

[0180] In the case of extracting the heading angle of the target vehicle in the comparative example described above, as shown in Figure 20, because the target vehicle's bumper (BUM) has a curved shape rather than a rectangular shape, when extracting the heading angle using only the angle formed by the peak point, the heading angle may be highly dependent on the peak point, and thus the heading angle may be sensitive to and vary with the peak point. In other words, in the comparative example, it is possible to extract the heading angle from... Figure 21 The heading direction of the target vehicle 730 was incorrectly extracted from the four directions shown ①, ②, ③ and ④.

[0181] Conversely, according to an embodiment of the invention, the distribution of LiDAR points (e.g., the inner and outer LiDAR points mentioned above) located near the first line segment L1 and the second line segment L2 is numerically calculated, shape markers are assigned to each of the M layers generated by the division in the height direction (i.e., the z-axis direction) of the target vehicle, and the interconnection relationships between the layers are utilized. Accordingly, the characteristics of the dynamic object can be effectively identified, thereby accurately determining whether the target object has an L-shaped or I-shaped profile, or whether the shape of the target object is unrecognizable. As a result, the accurately determined shape of the target vehicle can also be used to accurately determine the heading direction of the target vehicle and provide highly reliable information about the appearance of the target vehicle (e.g., the width or length of the target vehicle). Specifically, the embodiment can minimize the error in determining the heading direction of the dynamic object by assigning shape markers to the dynamic object. Refer to below. Figure 23 Describe dynamic objects in more detail.

[0182] Figures 22(a) and 22(b) are schematic diagrams illustrating the association performed by the object tracking device 100.

[0183] When two target objects 740 and 742, which are separated from each other in the first frame t as shown in Figure 22(a), are clustered into one object in the second frame t+1 following the first frame t as shown in Figure 22(b), the two objects 740 and 742 may be combined because the comparison example does not provide a reference for distinguishing target objects 740 and 742. Conversely, according to the embodiment, since a shape box (or shape mark) is assigned to at least one of the two objects 740 and 742, information about the two objects 740 and 742 can be output separately, as shown in Figure 22(b), even if the two objects 740 and 742 are combined in the second frame t+1.

[0184] Figure 23 It is a schematic diagram used to illustrate dynamic objects.

[0185] In the comparative example, the distribution of LiDAR points cannot determine whether the target object is a dynamic object (e.g., a vehicle) 762 or a static object (e.g., a flower bed) 760. Instead, according to the implementation scheme, during the analysis of the target object's shape, by performing step 316 (or step 316A), the possibility that the target object is not a dynamic object can be identified. This, in turn, facilitates subsequent processes that determine whether an object whose shape has been ultimately determined is a dynamic or static object based on a score.

[0186] As is evident from the above description, the object shape analysis method and object tracking device utilizing a LiDAR sensor according to the embodiment can effectively identify the characteristics of dynamic objects, thereby accurately determining whether the target object has an L-shaped or I-shaped profile, or whether the shape of the target object is unrecognizable. Furthermore, the accurately determined shape of the target vehicle can be used to accurately determine the heading direction of the target vehicle. This minimizes the error in determining the heading direction of the target vehicle, provides highly reliable information about the appearance of the target vehicle (e.g., the width or length of the target vehicle), and facilitates subsequent processes that determine whether the object whose shape has been finally determined is a dynamic or static object based on a score. Additionally, when shape tags are assigned to at least one of two objects, information about each object can be output separately, even if the two objects are subsequently combined.

[0187] However, the effects that can be achieved by the present invention are not limited to those described above, and those skilled in the art should clearly understand from the above description other effects not mentioned herein.

[0188] The various embodiments described above can be combined with each other without departing from the scope of the invention, unless they are incompatible with each other.

[0189] Furthermore, for any element or process not described in detail in any of the various embodiments, reference may be made to the description of the element or process having the same reference numerals in another embodiment, unless otherwise stated.

[0190] Although the invention has been specifically shown and described with reference to specific embodiments thereof, these embodiments are provided for illustrative purposes only and are not intended to limit the invention. It will be apparent to those skilled in the art that various changes in form and detail may be made without departing from the essential characteristics of the embodiments described herein. For example, the various configurations described in the embodiments may be modified and applied. Furthermore, such modifications and differences in application should be interpreted as falling within the scope of the invention as defined by the appended claims.

Claims

1. A method for analyzing the shape of an object using a lidar sensor, the method comprising: Step (a): Use clustered lidar points to determine the shape of all layers from the first layer to the Mth layer associated with the target object, where M is a positive integer of 2 or greater; Step (b): Determine the shape of the target object based on the shape determined by the predetermined priority analysis. In step (a), the shape of the m-th layer from the first layer to the M-th layer is determined, where 1≤m≤M; Step (a) includes: Step (a1): Among the lidar points included in the m-th layer, search for the breakpoint located furthest from the line segment connecting the first endpoint and the second endpoint. Step (a2): Using at least one of the following: a first line segment connecting the first endpoint and the breakpoint, a second line segment connecting the second endpoint and the breakpoint, a first lidar point located near the first line segment, or a second lidar point located near the second line segment, a shape marker is assigned to the m-th layer. Step (a2) includes: Analyze the distribution of the first and second lidar points in the m-th layer; The analysis results are used to assign the interruption marker to the m-th layer as a shape marker; The interruption marker indicates that the target object included in the m-th layer is less likely to be a dynamic object; Assigning the interrupt flag to the m-th layer includes: Calculate the first average value of the first distance between the first line segment and the first lidar point; Calculate the first variance of the first distance using the first average value; Calculate the second average value of the second distance between the second line segment and the second lidar point; Calculate the second variance of the second distance using the second mean; When both the first variance and the second variance are greater than the variance threshold, an interruption flag is assigned to the m-th layer.

2. The method according to claim 1, wherein, Step (a2) further includes: When either the first variance or the second variance is not greater than the variance threshold, the size of the shape box of the m-th layer, which includes the first line segment and the second line segment, is considered, and an L-shaped mark or an I-shaped mark is temporarily assigned to the m-th layer as a shape mark. For the L-shaped or I-shaped marker temporarily assigned to the m-th layer, at least one of the first line segment, the second line segment, the first lidar point, or the second lidar point is used to finally assign the L-shaped or I-shaped marker to the m-th layer.

3. The method according to claim 2, wherein, Temporarily assigning L-type or I-type tags to the m-th layer includes: Using at least one of the length or width of the shape box, temporarily assign an L-shaped mark or an I-shaped mark to the m-th layer.

4. The method according to claim 3, wherein, Temporarily assigning L-type or I-type tags to the m-th layer includes: When the width of the shape box falls within the first threshold width range, the I-shaped mark is temporarily assigned to the m-th layer; When the width of the shape box falls within the second threshold width range, the L-shaped mark is temporarily assigned to the m-th layer; Wherein, the first threshold width range has a range from a first minimum value to a first maximum value; The second threshold width range has a range from a second minimum value to a second maximum value; The second minimum value is greater than or equal to the first maximum value.

5. The method according to claim 4, wherein, Finally, assigning L-shaped tags to the m-th layer includes: Choose the longer line segment from the first and second line segments as the reference line segment; Choose the shorter line segment from the first and second line segments as the non-reference line segment; When the length of the reference line segment is greater than or equal to the threshold length, when the average and variance of the reference line segment are less than the average and variance of the reference threshold respectively, when the average and variance of the non-reference line segment are less than the average and variance of the non-reference threshold respectively, when there is a lidar point in each of the i regions formed by dividing in the direction intersecting with the reference line segment, when the interval between adjacent outer lidar points in the region is less than the threshold interval distance, and when the angle between the first line segment and the second line segment is greater than the first angle and less than the second angle, the L-shaped mark is finally assigned to the m-th layer, where i is a positive integer of 1 or greater.

6. The method according to claim 4, wherein, Finally, assigning type I tags to the m-th layer includes: Choose the longer line segment from the first and second line segments as the reference line segment; When the mean and variance of the reference line segment are less than the mean and variance of the reference threshold, respectively, and when the interval between the outer lidar points located in the j regions formed by the division in the direction intersecting the reference line segment is less than the threshold interval distance, the I-shaped mark is finally assigned to the m-th layer, where j is a positive integer of 1 or greater.

7. The method of claim 5, further comprising: Check if the m-th layer is the top layer related to the target object; Wherein, when the m-th layer is the top-related layer of the target object, the average value of the non-reference threshold and the variance of the non-reference threshold are increased; The non-reference threshold average and non-reference threshold variance are used in determining whether to ultimately assign the L-shaped label to the (m+1)th layer.

8. The method according to claim 7, wherein, Checking whether the m-th layer is the top-related layer of the target object includes: Check whether the first ratio of the length of the shape box in the m-th layer to the length of the cluster box about the target object is less than a first threshold ratio; When the first ratio is less than the first threshold ratio, search for the peak point of the shape tag finally assigned to the m-th layer; When the second ratio of the length from the peak point to the middle of the cluster box to half the length of the cluster box is less than the second threshold ratio, the m-th layer is determined to be a top-related layer.

9. The method according to claim 8, wherein, Peak search times include: When the L-shaped marker is finally assigned to the m-th layer, the lidar point located at the farthest distance from the shorter line segment between the first and second line segments is determined as the peak point. When the Type I marker is finally assigned to the m-th layer, the breakpoint is determined as the peak point.

10. The method according to claim 4, wherein, Determining the shape of the target object based on a predetermined priority includes: When there are layers with interruption markers assigned in the first to the mth layers, the shape of the target object cannot be identified. When there are no layers with interruption markers assigned in layers 1 to m, and there are layers with L-shaped markers assigned, the shape of the target object is determined to be L-shaped. When there are no layers in layers 1 through m that have assigned either an interrupt marker or an L-shaped marker, and there are layers that have assigned an I-shaped marker, the shape of the target object is determined to be I-shaped. When there is no layer in layers 1 through m that has assigned any of the interrupt markers, L-shaped markers, and I-shaped markers, the shape of the target object cannot be determined.

11. An apparatus for tracking objects using a lidar sensor, the apparatus comprising: A lidar sensor configured to acquire a point cloud associated with a target object; Clustering unit, configured to cluster the point cloud; as well as A shape analysis unit configured to analyze the shape of a target object using lidar points clustered in a point cloud; The shape analysis unit includes: A layer shape determination unit is configured to determine the shape of all layers from the first to the Mth layer associated with the target object using clustered lidar points; where M is a positive integer of 2 or greater; and A target shape determination unit is configured to analyze and determine the shape of a target object based on a predetermined priority. The layer shape determination unit determines the shape of the m-th layer from the first layer to the M-th layer, where 1≤m≤M; The layer shape determination unit includes: A preparation unit is defined, configured to search among the lidar points included in the m-th layer for the breakpoint located furthest from the line segment connecting the first endpoint and the second endpoint, and to generate a first line segment connecting the first endpoint and the breakpoint, and a second line segment connecting the second endpoint and the breakpoint; and A marker allocation unit is configured to allocate shape markers to the m-th layer using at least one of a first line segment, a second line segment, a first lidar point located near the first line segment, or a second lidar point located near the second line segment. The layer shape determination unit further includes: The object analysis unit is configured to analyze the distribution of the first lidar point and the second lidar point in the m layer, and to assign the interruption marker to the m layer as a shape marker using the analysis results. The interruption marker indicates that the target object included in the m-th layer is less likely to be a dynamic object; The object analysis unit includes: The first variance calculation unit is configured to use the first average value of the first distance to calculate the first variance of the first distance between the first line segment and the first lidar point; The second variance calculation unit is configured to calculate the second variance of the second distance between the second line segment and the second lidar point using the second average value of the second distance; and A variance comparison unit is configured to compare each of a first variance and a second variance with a variance threshold, and to assign an interrupt flag to the m-th layer in response to the comparison result.

12. The apparatus for tracking objects using a lidar sensor according to claim 11, wherein, The tag allocation unit includes: A temporary marker allocation unit, configured to, in response to the comparison result of the variance comparison unit, consider the size of the shape box of the m-th layer including the first line segment and the second line segment, temporarily allocate an L-shaped marker or an I-shaped marker to the m-th layer as a shape marker; and The final marker allocation unit is configured to: for an L-shaped marker or I-shaped marker temporarily assigned to the m-th layer, use at least one of a first line segment, a second line segment, a first lidar point, or a second lidar point to finally assign the L-shaped marker or I-shaped marker to the m-th layer.

13. The apparatus for tracking objects using a lidar sensor according to claim 12, wherein, The temporary marker allocation unit temporarily assigns an L-shaped marker or an I-shaped marker to the m-th layer using at least one of the length or width of the shape box.

14. The apparatus for tracking objects using a lidar sensor according to claim 13, wherein, The temporary marker allocation unit includes: A first width comparison unit is configured to compare the width of a shape box with a first threshold width range, and temporarily assign an I-shaped marker to the m-th layer in response to the comparison result; and The second width comparison unit is configured to compare the width of the shape box with a second threshold width range, and temporarily assign an L-shaped mark to the m-th layer in response to the comparison result; Wherein, the first threshold width range has a range from a first minimum value to a first maximum value; The second threshold width range has a range from a second minimum value to a second maximum value; The second minimum value is greater than or equal to the first maximum value.

15. The apparatus for tracking objects using a lidar sensor according to claim 14, wherein, The final tag allocation unit includes: A reference segment selection unit is configured to select the longer segment from a first segment and a second segment as a reference segment, and to select the shorter segment from the first segment and the second segment as a non-reference segment; and The first marker allocation analysis unit is configured to: assign an L-shaped marker to the m-th layer when the length of the reference line segment is greater than or equal to the threshold length; when the average value and variance of the reference line segment are less than the average value and variance of the reference threshold, respectively; when the average value and variance of the non-reference line segment are less than the average value and variance of the non-reference threshold, respectively; when a lidar point exists in each of the i regions formed by dividing the direction intersecting with the reference line segment; when the interval distance between adjacent outer lidar points located in the region is less than the threshold interval distance; and when the angle between the first line segment and the second line segment is greater than the first angle and less than the second angle, where i is a positive integer of 1 or greater.

16. The apparatus for tracking objects using a lidar sensor according to claim 15, wherein, The final tag allocation unit further includes: The second marker allocation analysis unit is configured to: assign I-shaped markers to the m-th layer when the average and variance of the reference line segment are less than the average and variance of the reference threshold, respectively, and when the interval distance between the outer lidar points located in the j regions formed by the division in the direction intersecting with the reference line segment is less than the threshold interval distance, where j is a positive integer of 1 or greater.

17. The apparatus for tracking objects using a lidar sensor according to claim 15, wherein, The layer shape determination unit further includes: The top-level inspection unit is configured to check whether the m-th layer is the top-related layer of the target object and output the inspection result. The first tag assignment analysis unit increases the non-reference threshold average and non-reference threshold variance in response to the inspection result of the top-level inspection unit. The non-reference threshold average and non-reference threshold variance are used in determining whether to finally assign the L-shaped tag to the (m+1)th layer.

18. The apparatus for tracking objects using a lidar sensor according to claim 17, wherein, The top-level inspection unit: Check whether the first ratio of the length of the shape box in the m-th layer to the length of the cluster box about the target object is less than a first threshold ratio; Search for the peak point of the shape marker ultimately assigned to the m-th layer; Check whether the second ratio of the length from the peak point to the middle of the cluster box to half the length of the cluster box is less than the second threshold ratio.

19. The apparatus for tracking objects using a lidar sensor according to claim 18, wherein, The top-level inspection unit, in response to the comparison result of the first ratio and the first threshold ratio and the final allocation result of the shape mark by the final mark allocation unit, determines the lidar point located at the farthest point from the shorter line segment of the first line segment and the second line segment as the peak point, or determines the breakpoint as the peak point.

20. The apparatus for tracking objects using a lidar sensor according to claim 16, wherein, The target shape determination unit includes: The first marker checking unit is configured to check whether there are any layers from the first layer to the mth layer that have been assigned an interrupt marker; The second marker checking unit is configured to check whether there is a layer that has been assigned an L-shaped marker in response to the checking result of the first marker checking unit; The third marker checking unit is configured to check, in response to the checking result of the second marker checking unit, whether a layer with a type I marker has been assigned; and The final shape output unit is configured to determine the shape of the target object as an unrecognizable shape, an L-shape, or an I-shape in response to the inspection results of the first to third mark inspection units.

Images