Object recognition apparatus and method thereof

Abstract

The invention relates to an object recognition apparatus and a method thereof. An object recognition device includes a sensor (e.g., LIDAR) and a processor. The processor may obtain, via the sensor, at least one point representative of an external environment of the vehicle, and determine a plurality of waypoints representative of locations on the ground. The plurality of waypoints may correspond to a path along which the vehicle is expected to travel. The processor may further classify each of the plurality of waypoints into at least one of a plurality of groups based on a distance between the vehicle and a location of each of the plurality of waypoints on the ground, determine a representative point for each of the plurality of groups, a road profile including at least one of height information of a ground surface or contour information of the ground surface is determined, and a signal associated with autonomous driving control of the vehicle is output.

Description

[0001] Cross - reference to related applications

[0002] This application claims the benefit of priority to Korean Patent Application No. 10 - 2024 - 0001034, filed with the Korean Intellectual Property Office on January 3, 2024, the entire contents of which are incorporated herein by reference. Technical field

[0003] The present disclosure relates to an object recognition device and method, and more particularly, to a technique for identifying height information of the ground based on information obtained via a Light Detection and Ranging (LIDAR) device. Background art

[0004] In autonomous vehicles and vehicles equipped with driving assistance devices, techniques for detecting the surrounding environment are necessary for avoiding obstacles and identifying hazards.

[0005] A vehicle can obtain information about its surrounding environment through one or more sensors such as LIDAR, radar, and cameras.

[0006] If only relying on a camera to obtain information about the surrounding environment of a vehicle, the accuracy of the height information of the surrounding environment may be lower than a reference value. Specifically, when the accuracy of the height information of the ground on which the vehicle travels is also improved, ride comfort and driving stability can be improved. This is because vehicle control can be assisted by the height information of the ground.

[0007] Therefore, a technique for identifying the height information of the ground through LIDAR to improve the height information of the surrounding environment may be beneficial. Summary of the invention

[0008] The present disclosure has been made to solve the above - mentioned problems that occur in the prior art while maintaining the advantages achieved by the prior art.

[0009] One aspect of the present disclosure provides an object recognition device and method for identifying ground height information of a path along which a host vehicle is expected to travel via LIDAR.

[0010] One aspect of the present disclosure provides an object recognition device and method for improving the accuracy of height information by identifying the height information of the ground for a path along which a host vehicle is expected to travel via LIDAR.

[0011] One aspect of the present disclosure provides an object recognition device and method for improving driving stability by improving the accuracy of ground height information for a path along which a host vehicle is expected to travel if there are road surface defects.

[0012] One aspect of the present disclosure provides an object recognition device and method for enhancing the driver experience by improving the accuracy of ground height information for a path along which a host vehicle is expected to travel.

[0013] One aspect of the present disclosure provides an object recognition device and method for obtaining height information of the ground of a path along which a host vehicle is expected to travel, which is robust to noise.

[0014] Aspects of the present disclosure provide an object recognition device and method for identifying the confidence of points corresponding to the ground.

[0015] The technical problems to be solved by the present disclosure are not limited to the above problems, and those skilled in the art to which the present disclosure pertains will clearly understand any other technical problems not mentioned herein from the following description.

[0016] According to one or more exemplary embodiments of the present disclosure, a device may include: a sensor; and a processor. The processor may be configured to: obtain, via the sensor, at least one point representing the external environment of the vehicle; among the at least one point, determine a plurality of path points representing positions on the ground. The plurality of path points may correspond to a path along which the vehicle is expected to travel. The processor may be further configured to: classify each of the plurality of path points into at least one of a plurality of groups based on the distance between the vehicle and the position of each path point on the ground among the plurality of path points; determine a representative point for each of the plurality of groups based on the positions of each classified path point in the group corresponding to the representative point among the plurality of groups; determine a road profile including at least one of the height information of the ground or the contour information of the ground based on interpolation between the representative points of two adjacent groups among the plurality of groups; and output a signal associated with the autonomous driving control of the vehicle based on the determined road profile.

[0017] The processor may be configured to determine the plurality of path points by: determining the type of an object corresponding to each point among the at least one point representing the external environment of the vehicle; and determining the plurality of path points based on the type of the object and the position of the object.

[0018] The at least one point may be included in the input data of an artificial neural network (ANN). The processor may be configured to determine the type of the object by: determining the type of the object based on the output data of the ANN.

[0019] The processor may be further configured to determine the confidence of the representative point of each of the plurality of groups based on at least one of the following: the type of the object corresponding to the classified path point included in the group corresponding to the representative point, the number of classified path points included in the group corresponding to the representative point, or the height of the classified path points included in the group corresponding to the representative point.

[0020] A first representative point corresponding to a first group having a greater number of classified path points can have a greater confidence than a second representative point corresponding to a second group having a smaller number of classified path points than a specified number. The confidence of a third representative point corresponding to a third group can be greater than the confidence of a fourth representative point corresponding to a fourth group. A maximum height value of the classified path points in the third group is not greater than an average height value of the classified path points included in the third group by at least a specified amount. A maximum height value of the classified path points in the fourth group can be greater than an average height value of the classified path points included in the fourth group by at least a specified amount.

[0021] The processor can be configured to classify each of a plurality of path points by: classifying a first path point among the plurality of path points as the first group based on a first distance from a first position on the ground corresponding to the first path point to the vehicle satisfying a first distance range; and classifying a second path point among the plurality of path points as the second group based on a second distance from a second position on the ground corresponding to the second path point to the vehicle satisfying a second distance range different from the first distance range.

[0022] The processor can be further configured to: determine an expected trajectory of two front wheels relative to a driving direction based on a steering angle of the vehicle; and determine the path based on the expected trajectory.

[0023] The processor can be configured to determine a representative point of each of the plurality of groups by: determining the representative point based on at least one of: an average longitudinal position of one or more path points among the plurality of path points classified as the group corresponding to the representative point; and an average lateral position of one or more path points classified as the group corresponding to the representative point, or an average height of one or more path points classified as the group corresponding to the representative point.

[0024] The path can have a specified length.

[0025] The processor can be configured to determine a representative point of each of the plurality of groups by determining the representative point based on the number of classified path points included in the group corresponding to the representative point being greater than a threshold number.

[0026] According to one or more exemplary embodiments of the present disclosure, a method may include: obtaining, via a sensor, at least one point representing an external environment of a vehicle; determining, among the at least one point, a plurality of path points representing positions on the ground. The plurality of path points may correspond to a path along which the vehicle is expected to travel. The method may further include: dividing each of the plurality of path points into at least one of a plurality of groups based on a distance between the vehicle and a position of each of the plurality of path points on the ground; determining a representative point for each of the plurality of groups based on positions of each of the classified path points in the group corresponding to the representative point among the plurality of groups; determining a road profile including at least one of height information of the ground or contour information of the ground based on interpolation between representative points of two adjacent groups among the plurality of groups; and outputting a signal associated with autonomous driving control of the vehicle based on the determined road profile.

[0027] Determining the plurality of path points may include: determining a type of an object corresponding to each of the at least one point representing the external environment of the vehicle; and determining the plurality of path points based on the type of the object and the position of the object.

[0028] The at least one point may be included in input data of an artificial neural network (ANN). Determining the type of the object may include: determining the type of the object based on output data of the ANN.

[0029] The method may include: determining a confidence level of a representative point for each of the plurality of groups based on at least one of the following: a type of an object corresponding to a classified path point included in the group corresponding to the representative point, a number of classified path points included in the group corresponding to the representative point, or a height of a classified path point included in the group corresponding to the representative point.

[0030] A first representative point corresponding to a first group having a number of classified path points greater than a specified number may have a greater confidence level than a second representative point corresponding to a second group having a number of classified path points less than the specified number. A confidence level of a third representative point corresponding to a third group may be greater than a confidence level of a fourth representative point corresponding to a fourth group. A maximum height value of classified path points in the third group may not be greater than an average height value of classified path points included in the third group by at least a specified amount. A maximum height value of classified path points in the fourth group may be greater than an average height value of classified path points included in the fourth group by at least a specified amount.

[0031] Classifying each of the plurality of waypoints may include: classifying a first waypoint among the plurality of waypoints into a first group based on a first distance from a first position on the ground corresponding to the first waypoint to the vehicle satisfying a first distance range; and classifying a second waypoint among the plurality of waypoints into a second group based on a second distance from a second position on the ground corresponding to the second waypoint to the vehicle satisfying a second distance range different from the first distance range.

[0032] The method may further include: determining an expected trajectory of two front wheels relative to a driving direction based on a steering angle of the vehicle; and determining the path based on the expected trajectory.

[0033] Determining a representative point for each of the plurality of groups may include: determining the representative point based on at least one of: an average longitudinal position of one or more waypoints classified into the group corresponding to the representative point among the plurality of waypoints; and an average lateral position of one or more waypoints classified into the group corresponding to the representative point, or an average height of one or more waypoints classified into the group corresponding to the representative point.

[0034] The path may have a specified length.

[0035] Determining a representative point for each of the plurality of groups may include: determining the representative point based on the number of classified waypoints included in the group corresponding to the representative point being greater than a threshold number. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other objects, features, and advantages of the present disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings:

[0037] Figure 1 is a block diagram showing a configuration of an object recognition device according to an embodiment of the present disclosure;

[0038] Figure 2 shows an example of obtaining a road profile through a sensor (e.g., LIDAR) in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0039] Figure 3 shows a flowchart of an operation of an object recognition device for recognizing confidence and obtaining a road profile in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0040] Figure 4 shows another flowchart of an operation of an object recognition device for recognizing confidence and obtaining a road profile in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0041] Figure 5Shows an example of a path along which a host vehicle is expected to travel in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0042] Figure 6 Shows an example of a representative point recognized from a classification point in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0043] Figure 7 Shows a flowchart of an operation of an object recognition device for obtaining a road profile in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0044] Figure 8 Shows a flowchart of an operation of an object recognition device for assigning a confidence level to a representative point in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0045] Figure 9 Shows a graph for deriving a criterion for allocating a confidence level in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0046] Figure 10 Shows an example of a road profile according to an embodiment of the present disclosure; and

[0047] Figure 11 Shows a computing system related to an object recognition device or an object recognition method according to an embodiment of the present disclosure. Detailed Description of Specific Embodiments

[0048] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the exemplary drawings. Additionally, when attaching reference numerals to the components in the respective drawings, the same reference numerals are also assigned in cases where the same or equivalent components are shown in other drawings. Furthermore, when describing the embodiments of the present disclosure, detailed descriptions of well-known features or functions will be excluded so as not to unnecessarily obscure the gist of the present disclosure.

[0049] When describing the components according to the embodiments of the present disclosure, terms such as first, second, "A", "B", (a), (b), etc. may be used. These terms are only intended to distinguish one component from another, and these terms do not limit the nature, order, or sequence of the constituent components. Unless otherwise defined, all terms (including technical or scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure pertains. Terms defined in a commonly used dictionary should be interpreted as having a meaning equivalent to the context meaning in the relevant field, and should not be interpreted as having an ideal or overly formal meaning, unless explicitly defined as having an ideal or overly formal meaning in the present application.

[0050] In addition, in the present disclosure, the expressions "greater than" or "less than" may be used to indicate whether a specific condition is satisfied or achieved, but are only used for illustrative purposes and do not exclude "greater than or equal to" or "less than or equal to". A condition expressed as "greater than or equal to" may be replaced by "greater than", a condition expressed as "less than or equal to" may be replaced by "less than", and a condition expressed as "greater than or equal to and less than" may be replaced by "greater than or equal to and less than or equal to". In addition, "A" to "B" means at least one of the elements from A (including A) to B (including B).

[0051] Hereinafter, embodiments of the present disclosure will be described in detail with reference to Figures 1 to 11 the drawings.

[0052] Figure 1 FIG. is a block diagram showing the configuration of an object recognition device according to an embodiment of the present disclosure.

[0053] Referring to Figure 1 FIG., the object recognition device 101 may include a sensor (e.g., LIDAR) 103 and a processor 105.

[0054] The sensor (e.g., LIDAR) 103 and the processor 105 may be electronically and / or operably coupled to each other through an electronic component such as a communication bus.

[0055] According to an embodiment, hereinafter, operably combining hardware blocks means a direct connection or an indirect connection between hardware blocks established in a wired or wireless manner such that a first hardware of a hardware block is controlled by a second hardware of the hardware block. The type and / or quantity (e.g., number) of the hardware included in the object recognition device 101 is not limited to the type and / or quantity of the hardware shown in Figure 1 the drawings. For example, the object recognition device 101 may include only Figure 1 some of the hardware components shown in the drawings.

[0056] According to an embodiment, the processor 105 of the object recognition device 101 may recognize the external environment of the host vehicle based on the sensor (e.g., LIDAR) 103. For example, the processor 105 of the object recognition device 101 may obtain at least one point representing the external environment (e.g., ground, road, sidewalk, parking lot) through the sensor (e.g., LIDAR) 103.

[0057] According to an embodiment, the processor 105 of the object recognition device 101 may recognize the type of an object corresponding to each point included in at least one point representing the external environment of the host vehicle. For example, the objects may constitute the external environment. For example, the types of objects may include ground, sidewalk, road, and parking lot.

[0058] According to an embodiment, the processor 105 of the object recognition device 101 may determine the type of an object corresponding to each point included in the output data of the artificial neural network based on each point included in the input data of the artificial neural network and included in at least one point representing the external environment of the host vehicle.

[0059] According to an embodiment, the processor 105 of the object recognition device 101 may identify, based on the steering angle of the host vehicle, the path (e.g., expected trajectory) along which two front wheels (two front wheels when the vehicle moves forward or two rear wheels when the vehicle moves backward) are expected to travel (move) with respect to the traveling direction (e.g., moving direction), and identify the path along which the host vehicle is expected to travel based on the path along which the two wheels are expected to travel.

[0060] According to an embodiment, the processor 105 of the object recognition device 101 may identify, based on the type of the object and the position of the object, a path point that is a point representing the ground (e.g., terrain) and included in the path along which the host vehicle is expected to travel among at least one point. For example, the ground (e.g., the earth) may include a surface on which the host vehicle can drive, such as a sidewalk, a road, and a parking lot.

[0061] According to an embodiment, the processor 105 of the object recognition device 101 may classify the path points into at least one group according to the distance between the host vehicle and the ground corresponding to the path points.

[0062] According to an embodiment, if the distance between the host vehicle and the ground corresponding to a first path point included in the path points satisfies a first distance range, the processor 105 of the object recognition device 101 may classify the first path point into a first group.

[0063] According to an embodiment, the distance between the host vehicle and the ground corresponding to the first path point may include the distance from the ground corresponding to the first path point to the center of the bumper of the host vehicle, but embodiments of the present disclosure are not limited thereto. According to another embodiment, the point on the host vehicle used as a reference for the distance from the first path point to the host vehicle may include a point other than the center of the bumper.

[0064] According to an embodiment, the processor 105 of the object recognition device 101 may classify a second path point into a second group instead of the first group based on the fact that the distance between the host vehicle and the ground corresponding to the second path point included in the path points does not satisfy the first distance range and satisfies a second distance range different from the first distance range.

[0065] According to an embodiment, the distance between the host vehicle and the ground corresponding to the second waypoint may include the distance from the ground corresponding to the second waypoint to the center of the bumper of the host vehicle. However, embodiments of the present disclosure are not limited thereto. According to another embodiment, the point on the host vehicle that serves as a reference for the distance from the second waypoint to the host vehicle may include a point other than the center of the bumper.

[0066] According to an embodiment, the processor 105 of the object recognition device 101 may identify one representative point for each group based on the positions of the classified points, where the classified points are waypoints classified into each group included in at least one group.

[0067] For example, the longitudinal position of the representative point corresponding to a specific group may include the average of the longitudinal positions of the classified points included in the specific group. For example, the lateral position of the representative point corresponding to a specific group may include the average of the lateral positions of the classified points included in the specific group. For example, the height of the representative point corresponding to a specific group may include the average of the heights of the classified points included in the specific group.

[0068] According to an embodiment, if the number (e.g., count) of the classified points included in a specific group is greater than a threshold number, the processor 105 of the object recognition device 101 may identify the representative point corresponding to the specific group according to the positions of the classified points included in the specific group.

[0069] If the number of the classified points included in a specific group is not greater than the threshold number, the processor 105 of the object recognition device 101 may not identify the representative point corresponding to the specific group. This is because if the number of points is equal to or less than the threshold number, the points are determined to be unreliable.

[0070] According to an embodiment, the processor 105 of the object recognition device 101 may obtain a road profile based on interpolation performed between the representative points corresponding to two adjacent groups, where the two adjacent groups are at a certain distance from the ground corresponding to the representative points, and where the road profile represents at least one of ground height information of the path along which the host vehicle is expected to travel, or contour (e.g., shape) information of the ground of the path along which the host vehicle is expected to travel, or any combination thereof. For example, the road profile may represent height information of the path along which the host vehicle is expected to travel.

[0071] Figure 2 An example of obtaining a road profile by a sensor (e.g., LIDAR) in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0072] See Figure 2 , in the first scenario 201, the ground height of the first portion 203 on the path along which the host vehicle is expected to travel may be less than or equal to a specified height.

[0073] In the second scenario 211, the ground height of the second portion 213 on the path along which the host vehicle is expected to travel may be greater than the specified height.

[0074] According to an embodiment, in the first scenario 201, although road markings indicating a speed bump are shown, the first portion 203 may represent a flat ground that does not include a speed bump.

[0075] A speed bump is a structure installed to prevent a vehicle from accelerating and to improve traffic safety on a road or in a parking lot, and may be installed in a protruding form on the road surface. Road markings indicating a speed bump may be painted on the speed bump. Road markings indicating a speed bump may be mainly represented as white lines and yellow lines.

[0076] In the first scenario 201, since the speed bump markings and the actual structure of the ground do not match each other, it may be difficult for the processor of an existing object recognition device to calculate the ground structure of the first portion 203 through a camera. Since the processor of the object recognition device according to the embodiment determines the ground structure through a sensor (e.g., LIDAR), the processor of the object recognition device can recognize that the speed bump is not included in the first portion 203. The processor of the object recognition device according to the embodiment can recognize that the height of the first portion 203 is the same as the height of the road on which the host vehicle travels.

[0077] According to an embodiment, in the second scenario 211, the second portion 213 may not include road markings indicating a speed bump, but a ground including a speed bump may appear. In other words, the second portion 213 may include a speed bump with the paint having been lost.

[0078] In the second scenario 211, since the markings on the ground and the actual structure of the ground do not match each other, it may be difficult for the processor of an existing object recognition device to calculate the speed bump included in the second portion 213 through a camera. Since the processor of the object recognition device according to the embodiment determines the ground structure through a sensor (e.g., LIDAR), the processor of the object recognition device can recognize that the speed bump is included in the second portion 213. The processor of the object recognition device according to the embodiment can recognize that the height of the second portion 213 is higher than the height of the road on which the host vehicle travels.

[0079] The processor of the object recognition device according to the embodiment can improve the accuracy of the road profile and driving stability by relying on a ground structure or road surface defect different from the applied markings. Road surface defects may include raised obstacles (such as speed bumps) or dug-in obstacles (such as potholes).

[0080] Figure 3A flowchart showing the operation of an object recognition device for recognizing confidence and obtaining a road profile in an object recognition device or object recognition method according to an embodiment of the present disclosure.

[0081] Hereinafter, it is assumed that Figure 1 the processor 105 of the object recognition device 101 performs Figure 3 the process. Moreover, in Figure 3 the description, the operations described as being performed by the processor of the object recognition device may be understood to be controlled by the processor 105 of the object recognition device 101.

[0082] Referring to Figure 3 , in a first operation 301, the processor of the object recognition device according to an embodiment may identify the type of the object corresponding to the point and identify the path along which the host vehicle is expected to travel.

[0083] According to an embodiment, the type of the object corresponding to the point may include ground, sidewalk, road, and parking lot.

[0084] According to an embodiment, the processor of the object recognition device may, based on the steering angle of the host vehicle, with respect to the traveling direction (e.g., moving direction), identify the path (e.g., expected trajectory) along which the two front wheels (e.g., the two front wheels of a forward-moving vehicle or the two rear wheels of a backward-moving vehicle) are expected to travel (e.g., move). The processor of the object recognition device may identify the path along which the host vehicle is expected to travel based on the path along which the two wheels are expected to travel.

[0085] According to an embodiment, the processor of the object recognition device may, among a plurality of points, identify a path point that represents the ground and is included in the path along which the host vehicle is expected to travel.

[0086] In a second operation 303, the processor of the object recognition device according to an embodiment may identify a representative point of each group and the confidence of the representative point according to the information about the classified points included in each group.

[0087] According to an embodiment, the processor of the object recognition device may classify the path points into at least one group according to the distance from the ground corresponding to the path point to the host vehicle to identify the representative point and the confidence of the representative point.

[0088] According to an embodiment, the processor of the object recognition device may identify one representative point of each group based on the position of the classified points, where the classified points are the path points classified as being included in at least one group of each group.

[0089] For example, the processor of the object recognition device may classify the classified points into at least one group according to the distance from the ground corresponding to the classified points to the host vehicle.

[0090] According to an embodiment, the processor of the object recognition device may identify the confidence of the representative point corresponding to each group based on at least one of the type of the object corresponding to the classification points included in each group, the number of classification points included in each group, or the height of the classification points included in each group, or any combination thereof. The method for identifying the confidence will be described below with reference to Figure 9 a method for identifying the confidence will be described.

[0091] In the third operation 305, the processor of the object recognition device according to an embodiment may obtain a road profile by performing interpolation.

[0092] According to one embodiment, the processor of the object recognition device may obtain a road profile based on interpolation performed between representative points, the road profile indicating at least one of ground height information for a path along which the host vehicle is expected to travel, or profile (e.g., shape) information of the ground for a path along which the host vehicle is expected to travel, or any combination thereof, the two adjacent groups being separated by a distance from the ground corresponding to the representative points to the host vehicle.

[0093] Interpolation may refer to identifying the function value of a third variable located between a first variable value and a second variable value based on the function value of the first variable value and the function value of the second variable value.

[0094] According to an embodiment, the processor of the object recognition device may estimate the height of the ground between the position of the ground corresponding to the first representative point and the position of the ground corresponding to the second representative point by interpolation.

[0095] Figure 4 Another flowchart showing operations of an object recognition device for identifying confidence and obtaining a road profile in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0096] Hereinafter, it is assumed that Figure 1 the processor 105 of the object recognition device 101 performs Figure 4 the process. Further, in the Figure 4 description, operations described as being performed by the processor of the object recognition device may be understood as being controlled by the processor 105 of the object recognition device 101.

[0097] Referring to Figure 4 , in the first operation 401, the processor of the object recognition device according to an embodiment may identify the type of the object corresponding to the point.

[0098] In a second operation 403, a processor of an object recognition device according to an embodiment may identify a path along which a host vehicle is expected to travel based on paths along which two wheels are expected to travel. According to an embodiment, the first operation 401 may be performed before the second operation 403 or may be performed after the second operation 403 has been performed.

[0099] In a third operation 405, a processor of an object recognition device according to an embodiment may generate a grid map based on the path along which the host vehicle is expected to travel. According to an embodiment, the grid map may include a plurality of grids separated according to distances from the ground corresponding to points to the host vehicle. Each grid included in the plurality of grids may correspond to each group.

[0100] In a fourth operation 407, a processor of an object recognition device according to an embodiment may obtain representative points of each group based on the grid map.

[0101] In a fifth operation 409, a processor of an object recognition device according to an embodiment may identify a confidence level of the representative points.

[0102] In a sixth operation 411, a processor of an object recognition device according to an embodiment may obtain a road profile by performing interpolation.

[0103] According to an embodiment, the road profile may represent at least one of ground height information of the path along which the host vehicle is expected to travel, or ground profile (e.g., shape) information of the path along which the host vehicle is expected to travel, or any combination thereof.

[0104] Figure 5 An example of a path along which a host vehicle is expected to travel in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0105] See Figure 5 , a first screen 501 may include points representing an external environment. A second screen 503 may include points representing an external environment different from the external environment included in the first screen 501. The external environment may include a road, a vehicle, and a tree. Points representing the road may be included in points representing the ground.

[0106] A first sensor (e.g., LIDAR) screen 511 may include a plurality of points representing the ground and a road along which a host vehicle on a straight road is expected to travel. A second sensor (e.g., LIDAR) screen 513 may include points representing the ground and a path along which a vehicle on a curved path is expected to travel.

[0107] According to an embodiment, on the first sensor (e.g., LIDAR) screen 511, if the steering wheel of the host vehicle is not operated, a path identified based on the steering angle of the host vehicle may be displayed.

[0108] According to an embodiment, on the screen 513 of the second sensor (e.g., LIDAR), if the steering wheel of the host vehicle is operated to the left with respect to the host vehicle, a path identified based on the steering angle of the host vehicle may be displayed.

[0109] Figure 6 An example of a representative point identified from classification points in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0110] See Figure 6 , the first grid map 601 may include path points classified according to the distance between the host vehicle and the ground corresponding to the path points. The first grid map 601 may include a first preliminary grid 603, at least one grid 605, and a second preliminary grid 607. The second grid map 611 may include the same path points as the first grid map 601. The second grid map 611 may include representative points corresponding to the grids identified from the first grid map 601, respectively. The first curve graph 621 may display the representative points identified from the second grid map 611 according to height. The second curve graph 623 may include interpolation points generated by performing interpolation between the representative points displayed in the first curve graph 621. The interpolation points may represent ground height information of a path along which the host vehicle is expected to travel.

[0111] According to an embodiment, a processor of the object recognition device may generate a grid map identified according to a path along which the host vehicle is expected to travel. The processor of the object recognition device may map path points to the grid map, where the path points are points that represent the ground and are included in the path along which the host vehicle is expected to travel.

[0112] The first preliminary grid 603 and the second preliminary grid 607 may be reserved for performing interpolation. Due to the characteristics of the sensor (e.g., LIDAR), the grids included in at least one grid 605 may not include path points. In this case, the processor of the object recognition device may ensure the preliminary grids for performing interpolation. The sizes of the first preliminary grid 603 and the second preliminary grid 607 may be set to sizes that allow the path points included in at least one layer to be detected.

[0113] According to an embodiment, the horizontal axis (e.g., l) of the first grid map 601, the horizontal axis (e.g., l) of the second grid map 611, the horizontal axis (e.g., l) of the first curve graph 621, and the horizontal axis (e.g., l) of the second curve graph 623 may represent the distance between the host vehicle and the ground corresponding to the grid. The vertical axis (e.g., y) of the first grid map 601 and the vertical axis (e.g., y) of the second grid map 611 may represent the distance between the host vehicle and the representative point or the interpolation point. The vertical axes of the first curve graph 621 and the second curve graph 623 may represent the height of the ground corresponding to the representative point or the interpolation point.

[0114] According to an embodiment, in the first grid map 601, the horizontal length of at least one grid 605 may refer to the length of the path along which the host vehicle is expected to travel. The path along which the host vehicle is expected to travel may have a specified length.

[0115] According to an embodiment, in the second grid map 611, the processor of the object recognition device may identify one representative point (e.g., a solid point) of each group corresponding to each grid based on the position of the classified points, based on the number of classified points included in each group being greater than a threshold number. The reason is that if the number of classified points included in the grid is less than or equal to the threshold number, these points may be caused by noise.

[0116] The longitudinal position of the representative point corresponding to each group may be identified based on the average value of the longitudinal positions of the classified points included in each group. The lateral position of the representative point corresponding to each group may be identified based on the average value of the lateral positions of the classified points included in each group. The height of the representative point may be identified based on the average value of the heights of the classified points classified into each group.

[0117] According to an embodiment, in the first curve graph 621, the processor of the object recognition device may display the height of the point according to the distance from the ground corresponding to the representative point to the host vehicle. Referring to the first curve graph 621, it can be identified that as the distance from the host vehicle increases, the ground height increases.

[0118] In the second curve graph 623, the processor of the object recognition device may estimate the height information about the ground that does not correspond to the representative point by performing interpolation between the representative points of each group. Interpolation may be performed because classified points may not exist in the grid, because points may be lost due to the characteristics of the sensor (e.g., LIDAR), and the lateral resolution may be lower than the longitudinal resolution.

[0119] Figure 7 A flowchart showing the operation of an object recognition device for obtaining a road profile in an object recognition device or an object recognition method according to an embodiment of the present disclosure.

[0120] In the following, it is assumed that Figure 1 the processor 105 of the object recognition device 101 performs Figure 7 the process. Moreover, in Figure 7 the description, the operations described as being performed by the processor of the object recognition device can be understood as being controlled by the processor 105 of the object recognition device 101.

[0121] Refer to Figure 7 , in the first operation 701, the processor of the object recognition device according to the embodiment can obtain at least one point representing the external environment of the host vehicle through a sensor (e.g., LIDAR).

[0122] In the second operation 703, the processor of the object recognition device according to the embodiment can identify a path point among the at least one point, and the path point is a point representing the ground and included in the path along which the host vehicle is expected to travel.

[0123] In the third operation 705, the processor of the object recognition device according to the embodiment can classify the path points into at least one group according to the distance between the host vehicle and the ground corresponding to the path points.

[0124] In the fourth operation 707, the processor of the object recognition device according to the embodiment can identify a representative point of each group based on the positions of the classified points, and the classified points are the path points of each group classified as being included in the at least one group.

[0125] In the fifth operation 709, the processor of the object recognition device according to the embodiment can perform interpolation between the representative points corresponding to two adjacent groups, and the two adjacent groups are separated according to the distance between the host vehicle and the ground corresponding to each representative point.

[0126] In the sixth operation 711, the processor of the object recognition device according to the embodiment can obtain a road profile representing at least one of the height information of the ground, the contour (e.g., shape) information of the ground, or any combination thereof. The height information of the ground can represent the height information of the ground along the path along which the host vehicle is expected to travel. The contour (e.g., shape) information of the ground can represent the contour (e.g., shape) information of the ground along the path along which the host vehicle is expected to travel.

[0127] Figure 8 A flowchart showing the operations of the object recognition device for assigning confidence to the representative points in the object recognition device or object recognition method according to an embodiment of the present disclosure.

[0128] In the following, it is assumed that Figure 1 the processor 105 of the object recognition device 101 performs Figure 8 the process. In addition, in Figure 8In the description, the operations described as being performed by the processor of the object recognition device can be understood as being controlled by the processor 105 of the object recognition device 101.

[0129] See Figure 8 , the processor of the object recognition device according to an embodiment can identify the confidence of the representative point corresponding to each group based on at least one of the type of the object corresponding to the classification points included in each group, the number of classification points included in each group, or the height of the classification points included in each group or any combination thereof.

[0130] The processor of the object recognition device according to an embodiment can obtain a road profile based on the classification points obtained via a sensor (e.g., LIDAR), and assign a confidence to the classification points included in each group based on the road profile. If the confidence is classified into a specified number of levels (e.g., five levels), the processor of the object recognition device according to an embodiment can assign the confidence to the classified points. For example, the confidence when there are no road surface defects or obstacles can be greater than the confidence when there are road surface defects or obstacles on the ground. For example, the reliability (e.g., about 15 points) when the height distribution of the classification points of the group is uniform and all the classification points of the group represent the ground is greater than the reliability (e.g., about 0 points) when the height distribution of the classification points of the group is non-uniform and the group includes classification points representing stationary objects (e.g., buildings).

[0131] In the first operation 801, the processor of the object recognition device according to an embodiment can determine whether to identify only the points representing the ground. If only the points representing the ground are identified, the processor of the object recognition device can perform the second operation 803. If only the points representing the ground are not identified, the processor of the object recognition device can perform the third operation 805.

[0132] According to an embodiment, the points representing the ground can include the points representing the road, the points representing the sidewalk, the points representing the parking lot, and the points corresponding to a part of the ground.

[0133] In the third operation 805, the processor of the object recognition device can determine whether to identify the points representing the ground. If the points representing the ground are identified, the processor of the object recognition device can perform the fourth operation 807. If the points representing the ground are not identified, the processor of the object recognition device can perform the fifth operation 809.

[0134] According to an embodiment, when the number of classification points included in a specific group is greater than a specified number, the confidence level (e.g., confidence score) of the representative point corresponding to the specific group can be recognized as greater than when the number of classification points included in the specific group is less than or equal to the specified number. In other words, the higher the confidence level, the more (e.g., the higher the number) the classification points are included in the group.

[0135] In the fifth operation 809, the processor of the object recognition device according to an embodiment can allocate a confidence level of zero.

[0136] In the fourth operation 807, the processor of the object recognition device according to an embodiment can recognize whether the ratio of the number of points representing the ground to the total number of points is greater than a specified ratio. If the ratio of the number of points representing the ground to the total number of points is greater than the specified ratio, the processor of the object recognition device can perform the sixth operation 817. If the ratio of the number of points representing the ground to the total number of points is less than or equal to the specified ratio, the processor of the object recognition device can perform the seventh operation 819.

[0137] According to an embodiment, when the value obtained by subtracting the average value of the heights represented by the classification points included in a specific group from the maximum value of the heights represented by the classification points included in the specific group is greater than a specified difference, the processor of the object recognition device can recognize that the confidence level of the representative point corresponding to the specific group is less than the confidence level of the representative point corresponding to the specific group when the value obtained by subtracting the average value from the maximum value is less than or equal to the specified difference. In other words, the lower the confidence level of the representative point, the greater the difference between the maximum value of the height and the average value of the height.

[0138] In the seventh operation 819, the processor of the object recognition device according to an embodiment can allocate a confidence level of 3.

[0139] In the sixth operation 817, the processor of the object recognition device according to an embodiment can recognize whether the value obtained by subtracting the average value of the heights of the classification points in a specific group from the maximum value of the heights of the classification points in the specific group is greater than a specified difference. If the value obtained by subtracting the average value of the heights of the classification points in the specific group from the maximum value of the heights of the classification points in the specific group is less than or equal to the specified difference, the processor of the object recognition device can perform the eighth operation 821. If the value obtained by subtracting the average value of the heights of the classification points in the specific group from the maximum value of the heights of the classification points in the specific group is greater than the specified difference, the processor of the object recognition device can perform the seventh operation 819.

[0140] In the eighth operation 821, the processor of the object recognition device according to an embodiment can allocate a confidence level of 7.

[0141] In a second operation 803, a processor of an object recognition device according to an embodiment may identify whether the number of points is greater than a specified number. If the number of points is greater than the specified number, the processor of the object recognition device may perform a ninth operation 811. If the number of points is less than or equal to the specified number, the processor of the object recognition device may perform a tenth operation 813.

[0142] In the tenth operation 813, a processor of an object recognition device according to an embodiment may allocate a confidence level 11.

[0143] In the ninth operation 811, a processor of an object recognition device according to an embodiment may identify whether the ratio of the number of points representing a road to the total number of points is greater than a specified ratio. If the ratio of the number of points representing a road to the total number of points is greater than the specified ratio, the processor of the object recognition device may perform an eleventh operation 815. If the ratio of the number of points representing a road to the total number of points is less than or equal to the specified ratio, the processor of the object recognition device may perform the tenth operation 813.

[0144] In the eleventh operation 815, a processor of an object recognition device according to an embodiment may allocate a confidence level 15.

[0145] Figure 9 A graph showing criteria for allocating a confidence level in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0146] See Figure 9 , a first graph 901 may display the number of groups based on a value obtained by subtracting an average value of heights represented by classification points included in a group from a maximum value of heights represented by classification points included in the group. The classification points shown in the first graph 901 may be obtained corresponding to flat ground where the height is included in a specified range.

[0147] A second graph 911 may display the number of groups according to the number of classification points representing the ground included in a group. The classification points representing the second graph 911 may show points obtained corresponding to curved ground where the height is outside the specified range.

[0148] According to an embodiment, in the first graph 901, in the case of a group of classification points correspondingly included according to flat ground where the height is included in a specified range, the number of groups having a value obtained by subtracting an average value of heights in the group from a maximum value of heights in the group may rapidly decrease after a specified difference (for example, about 0.05 meters).

[0149] Therefore, the processor of the object recognition device can identify whether the classification points included in the group correspond to flat ground based on the value obtained by subtracting the average value of the heights from the maximum value of the heights in the group and the specified difference. For example, reference can be made to Figure 8 the sixth operation 817 of

[0150] In other words, when the value obtained by subtracting the average value of the heights represented by the classification points included in the specific group from the maximum value of the heights represented by the classification points included in the specific group is greater than the specified difference (e.g., about 0.05 meters), the processor of the object recognition device according to the embodiment can identify that the confidence level (e.g., about 3) corresponding to the representative point of the specific group is less than the confidence level (e.g., about 7) corresponding to the representative point of the specific group when the value obtained by subtracting the average value from the maximum value is less than or equal to the specified difference.

[0151] According to the embodiment, in the second curve graph 911, according to the number of classification points included in the group and representing the ground, the number of groups can rapidly decrease after the specified number (e.g., 8). The classification points representing the second curve graph 911 can be obtained according to the curved ground.

[0152] The processor of the object recognition device can identify whether the points included in the group correspond to curved ground based on the number of classification points representing the ground and included in the group and the specified number. For example, reference can be made to Figure 8 the second operation 803 of

[0153] In other words, when the number of classification points included in the specific group is greater than the specified number (e.g., about 8), the confidence level (e.g., about 15) corresponding to the representative point of the specific group can be identified as greater than the confidence level (e.g., about 11) corresponding to the representative point of the specific group when the number of classification points included in the specific group is less than or equal to the specified number.

[0154] Figure 10 An example of a road profile according to an embodiment of the present disclosure is shown.

[0155] Reference is made to Figure 10 , if a speed bump is included in the path along which the host vehicle is expected to travel, the first screen 1001 can display the screen identified via a sensor (e.g., LIDAR). If a speed bump is included in the path along which the host vehicle is expected to travel, the first image 1003 can display the screen identified via a camera. The path along which the host vehicle is expected to travel can include a first road profile 1005 of the predicted path for the left wheel and a second road profile 1007 of the predicted path for the right wheel.

[0156] According to an embodiment, a processor of the object recognition device may identify a portion corresponding to a speed bump based on the first path profile 1005 and the second path profile 1007.

[0157] If a ramp is included in a path along which the host vehicle is expected to travel, the second screen 1011 may display a screen identified via a sensor (e.g., LIDAR). If a ramp is included in a path along which the host vehicle is expected to travel, the second image 1013 may display a screen identified via a camera. The path along which the host vehicle is expected to travel may include a third road profile 1015 of a predicted path for the left wheel and a fourth path profile 1017 of a predicted path for the right wheel.

[0158] According to an embodiment, a processor of the object recognition device may identify a portion corresponding to a ramp based on the third road profile 1015 and the fourth road profile 1017.

[0159] Figure 11 A computing system related to an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0160] See Figure 11 , the computing system 1100 may include at least one processor 1110, a memory 1130, a user interface input device 1140, a user interface output device 1150, a storage device 1160, and a network interface 1170 that are connected to each other via a bus 1120.

[0161] The processor 1110 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1130 and / or the storage device 1160. The memory 1130 and the storage device 1160 may include various types of volatile or non-volatile storage media. For example, the memory 1130 may include a ROM (read-only memory) 1131 and a RAM (random access memory) 1132.

[0162] Accordingly, operations of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware or a software module executed by the processor 1110 or a combination thereof. The software module may reside on a storage medium (i.e., the memory 1130 and / or the storage device 1160) such as a RAM, a flash memory, a ROM, an EPROM, an EEPROM, a register, a hard disk, a removable disk, and a CD-ROM.

[0163] An exemplary storage medium may be coupled to the processor 1110, and the processor 1110 may read information from the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with the processor 1110. The processor and the storage medium may be present in an application specific integrated circuit (ASIC). The ASIC may reside within a user terminal. In another case, the processor and the storage medium may reside in the user terminal as separate components.

[0164] The above description is only an illustration of the technical concept of the present disclosure, and those skilled in the art to which the present disclosure pertains may make various modifications and changes without departing from the essential features of the present disclosure.

[0165] Therefore, the embodiments disclosed in the present disclosure are not intended to limit the technical concept of the present disclosure, but are used to describe the present disclosure, and the scope of the technical concept of the present disclosure is not limited by the embodiments. The protection scope of the present disclosure shall be interpreted by the appended claims, and all technical concepts within the scope equivalent thereto shall be interpreted as being included within the scope of the present disclosure.

[0166] The present technology may identify ground height information of a path along which a host vehicle is expected to travel via a sensor (e.g., LIDAR).

[0167] In addition, the present technology may improve the accuracy of height information by identifying ground height information of a path along which a host vehicle is expected to travel via a sensor (e.g., LIDAR).

[0168] In addition, even if there are road surface defects, the present technology may improve driving stability by improving the accuracy of ground height information of a path along which a host vehicle is expected to travel.

[0169] In addition, the present technology may improve the driver experience by improving the accuracy of ground height information of a path along which a host vehicle is expected to travel.

[0170] In addition, the present technology may obtain ground height information that is robust to noise.

[0171] In addition, the present technology may identify the confidence level of points corresponding to the ground.

[0172] In addition, various effects directly or indirectly understood through the present disclosure may be provided.

[0173] In the above, although the present disclosure has been described with reference to exemplary embodiments and the drawings, the present disclosure is not limited thereto, but various modifications and changes may be made by those skilled in the art to which the present disclosure pertains without departing from the spirit and scope of the present disclosure claimed in the appended claims.

Claims

1. An object recognition device, comprising: a sensor; and a processor configured to: obtain, via the sensor, at least one point representing an external environment of a vehicle; determine, among the at least one point, a plurality of path points representing positions on the ground, wherein the plurality of path points correspond to a path along which the vehicle is expected to travel; classify each of the plurality of path points into at least one of a plurality of groups based on a distance between the vehicle and a position of each of the plurality of path points on the ground; determine a representative point for each of the plurality of groups based on positions of each of the classified path points in the group corresponding to the representative point among the plurality of groups; determine a road profile including at least one of height information of the ground or contour information of the ground based on interpolation between representative points of two adjacent groups among the plurality of groups; and output a signal associated with autonomous driving control of the vehicle based on the determined road profile.

2. The object recognition device according to claim 1, wherein, The processor is configured to determine the plurality of path points by: determining a type of an object corresponding to each of the at least one point representing the external environment of the vehicle; and determining the plurality of path points based on the type of the object and the position of the object.

3. The object recognition device according to claim 2, wherein, The at least one point is included in input data of an artificial neural network ANN, and wherein the processor is configured to determine the type of the object by: determining the type of the object based on output data of the ANN.

4. The object recognition device according to claim 1, wherein, The processor is further configured to determine a confidence level of the representative point of each of the plurality of groups based on at least one of: a type of an object corresponding to a classified path point included in the group corresponding to the representative point, a number of the classified path points included in the group corresponding to the representative point, or a height of the classified path points included in the group corresponding to the representative point.

5. The object recognition device according to claim 1, wherein, A first representative point corresponding to a first group having a greater number of classified path points than a second representative point corresponding to a second group having a smaller number of classified path points has a greater confidence level, and wherein a third representative point corresponding to a third group has a greater confidence level than a fourth representative point corresponding to a fourth group, wherein a maximum height value of the classified path points in the third group is not greater than an average height value of the classified path points included in the third group by at least a specified amount, and wherein a maximum height value of the classified path points in the fourth group is greater than the average height value of the classified path points included in the fourth group by the at least specified amount.

6. The object recognition device according to claim 1, wherein, The processor is configured to classify each of the plurality of path points by: classifying the first path point among the plurality of path points into a first group based on a first distance from a first position corresponding to the first path point on the ground to the vehicle satisfying a first distance range; and Based on a second distance from a second position corresponding to the second path point on the ground to the vehicle satisfying a second distance range different from the first distance range, classify the second path point among the plurality of path points into a second group.

7. The object recognition device according to claim 1, wherein, The processor is further configured to: Based on the steering angle of the vehicle, determine an expected trajectory of two front wheels relative to the driving direction; and Based on the expected trajectory, determine the path.

8. The object recognition device according to claim 1, wherein The processor is configured to determine a representative point for each group among the plurality of groups by: Determining the representative point based on at least one of the following: An average longitudinal position of one or more path points classified as a group corresponding to the representative point among the plurality of path points; and An average lateral position of the one or more path points classified as a group corresponding to the representative point, or An average height of the one or more path points classified as a group corresponding to the representative point.

9. The object recognition device according to claim 1, wherein, The path has a specified length.

10. The object recognition device according to claim 1, wherein, The processor is configured to determine a representative point for each group among the plurality of groups by determining the representative point based on the number of classified path points included in the group corresponding to the representative point being greater than a threshold number.

11. An object recognition method, comprising: Obtain, via a sensor, at least one point representing the external environment of a vehicle; Among the at least one point, determine a plurality of path points representing positions on the ground, where the plurality of path points correspond to a path along which the vehicle is expected to travel; Based on a distance between the vehicle and the position of each path point among the plurality of path points on the ground, classify each path point among the plurality of path points into at least one group among a plurality of groups; Based on the positions of each classified path point in the group corresponding to the representative point among the plurality of groups, determine the representative point for each group among the plurality of groups; Based on interpolation between representative points of two adjacent groups among the plurality of groups, determine a road profile including at least one of height information of the ground or contour information of the ground; and Based on the determined road profile, output a signal associated with autonomous driving control of the vehicle.

12. The object recognition method according to claim 11, wherein, Determining the plurality of path points includes: Determine the type of an object corresponding to each point among the at least one point representing the external environment of the vehicle; and Based on the type of the object and the position of the object, determine the plurality of path points.

13. The object recognition method according to claim 12, wherein, The at least one point is included in input data of an artificial neural network ANN, and wherein determining the type of the object includes: Based on output data of the ANN, determine the type of the object.

14. The object recognition method according to claim 11, further comprising: Based on at least one of the following, determine a confidence level of a representative point for each group among the plurality of groups: the type of an object corresponding to a classified path point included in the group corresponding to the representative point, the number of the classified path points included in the group corresponding to the representative point, or the height of the classified path points included in the group corresponding to the representative point.

15. The object recognition method according to claim 11, wherein, The first representative point corresponding to the first group having a greater number of classification path points than a specified number has a greater confidence than the second representative point corresponding to the second group having a smaller number of classification path points than the specified number, and wherein the third representative point corresponding to the third group has a greater confidence than the fourth representative point corresponding to the fourth group, wherein a maximum height value of the classification path points in the third group is not greater than an average height value of the classification path points included in the third group by at least a specified amount, and wherein the maximum height value of the classification path points in the fourth group is greater than the average height value of the classification path points included in the fourth group by the at least specified amount.

16. The object recognition method according to claim 11, wherein, Classifying each of the plurality of path points includes: classifying a first path point among the plurality of path points into a first group based on a first distance from a first position on the ground corresponding to the first path point to the vehicle satisfying a first distance range; and classifying a second path point among the plurality of path points into a second group based on a second distance from a second position on the ground corresponding to the second path point to the vehicle satisfying a second distance range different from the first distance range.

17. The object recognition method according to claim 11, further comprising: determining an expected trajectory of two front wheels relative to a driving direction based on a steering angle of the vehicle; and determining the path based on the expected trajectory.

18. The object recognition method according to claim 11, wherein, Determining a representative point for each of the plurality of groups includes: determining the representative point based on at least one of: an average longitudinal position of one or more path points among the plurality of path points classified as the group corresponding to the representative point; and an average lateral position of the one or more path points classified as the group corresponding to the representative point, or an average height of the one or more path points classified as the group corresponding to the representative point.

19. The object recognition method according to claim 11, wherein, The path has a specified length.

20. The object recognition method according to claim 11, wherein, Determining the representative point for each of the plurality of groups includes: determining the representative point based on the number of classification path points included in the group corresponding to the representative point being greater than a threshold number.

Images