Apparatus for recognizing object and method thereof

Abstract

An embodiment object recognition apparatus includes a LIDAR, a camera, a radar, one or more processors, and a storage device storing programs executed by the one or more processors, the program includes instructions to identify a fusion trajectory corresponding to an external object acquired by a camera and a radar, identify a target fusion trajectory, identify a LIDAR trajectory corresponding to the external object represented by the target fusion trajectory, identify a target LIDAR trajectory, identify a second LIDAR point satisfying a positional condition, and identify a target LIDAR trajectory corresponding to the external object represented by the target fusion trajectory. Identifying, among the second LIDAR points, at least one third LIDAR point that satisfies a distance condition, identifying the distance condition based on a distance from a point corresponding to the host vehicle to the second LIDAR point, and associating and storing a target fusion trajectory with the third LIDAR point based on identifying the third LIDAR point.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of Korean Patent Application No. 10-2023-0143150, filed on October 24, 2023, which is hereby incorporated by reference into this application. Technical Field

[0003] The present disclosure relates to an object recognition device and method. Background Art

[0004] Autonomous vehicles or vehicles with activated driver assistance features require technology to detect the surrounding environment and distinguish obstacles in order to adjust their driving path and avoid obstacles with minimal driver intervention.

[0005] The vehicle can obtain data indicating the position of objects around the vehicle through LIDAR. The distance from the LIDAR to the object can be obtained by the interval between the time when the laser is emitted by the LIDAR and the time when the laser reflected by the object is received. The vehicle can recognize the position of a point outside the object in the space where the vehicle is located based on the angle of the emitted laser and the distance to the object.

[0006] Autonomous vehicles or vehicles with activated driver assistance devices can recognize the positions of external objects and distinguish obstacles based on trajectories obtained by multiple sensors. Summary of the invention

[0007] The present disclosure relates to object recognition devices and methods. Certain embodiments relate to techniques for improving the performance of object recognition devices based on fused trajectories recognized by multiple sensors.

[0008] The embodiments of the present disclosure can solve the problems occurring in the prior art while maintaining the advantages achieved by the prior art unchanged.

[0009] Embodiments of the present disclosure provide an object recognition device and method, which are capable of recognizing LIDAR points corresponding to a target fusion trajectory recognized by at least one sensor.

[0010] Embodiments of the present disclosure provide an object recognition apparatus and method capable of improving the position accuracy of an external object corresponding to a target fused trajectory recognized by at least one sensor.

[0011] Embodiments of the present disclosure provide an object recognition device and method for improving the classification accuracy of a target fusion trajectory recognized by at least one sensor.

[0012] The technical problems that can be solved by the embodiments of the present disclosure are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0013] According to an embodiment of the present disclosure, an object recognition device includes a LIDAR, a camera, a radar, and one or more processors and a storage device storing a program to be executed by the one or more processors.

[0014] According to an embodiment, the program may include instructions for: identifying at least one fused track corresponding to at least one external object acquired by the camera and the radar; identifying a target fused track in the at least one fused track based on a category to which the at least one fused track is classified, a longitudinal component of a distance from a point corresponding to the host vehicle to the at least one fused track, or any combination thereof; identifying at least one LIDAR track corresponding to the external object represented by the target fused track in the at least one external object acquired by the LIDAR; identifying at least one target LIDAR track in the at least one LIDAR track based on a distance from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track, wherein the point corresponding to the target fused track represents the object identified by the camera. a point corresponding to the camera trajectory, representing at least one external object and identifying a target fused trajectory; identifying a second LIDAR point satisfying a position condition, an angle condition, or any combination thereof, from among the first LIDAR points of the at least one target LIDAR trajectory, wherein the position condition is a position condition determined based on a position difference between the point corresponding to the target fused trajectory and the first LIDAR point, a position of the radar, and a position of the camera, and wherein the angle condition is an angle condition identified based on a half-straight line; identifying at least one third LIDAR point satisfying a distance condition from among the second LIDAR points, wherein the distance condition is identified based on a distance from the point corresponding to the host vehicle to the second LIDAR point; and associating and storing the target fused trajectory with the at least one third LIDAR point based on identifying the at least one third LIDAR point.

[0015] According to an embodiment, the program further includes instructions for identifying a target fused track in the at least one fused track based on at least one fused track identified via the camera and the radar, the at least one fused track being classified into a specified category, and identifying that the longitudinal component is less than a specified distance.

[0016] According to an embodiment, the program further includes instructions for identifying at least one target LIDAR track in the at least one LIDAR track based on a half-line passing through a track box representing the at least one LIDAR track identified by the LIDAR or based on identifying that at least one of the vertical distances from the half-line to four vertices of the track box is less than or equal to a specified distance.

[0017] According to an embodiment, the position condition may include: a first condition that the first value satisfies a first specified range, the first value is identified based on a difference between the longitudinal position of the radar and the longitudinal position of the first LIDAR point and a difference between the lateral position of the camera and the lateral position of the first LIDAR point; and a second condition of a second specified range that the second value is different from the first specified range, the second value is identified based on a difference between a position of a center point of one of four line segments of a trajectory box representing the target fused trajectory and the position of the first LIDAR point.

[0018] According to an embodiment, the angle condition may include the following condition: in a state where the first LIDAR points are arranged in descending order of angles between each of the half-straight lines from the point corresponding to the host vehicle to the first LIDAR point and the half-straight line, the LIDAR points are included in a specified order among the first LIDAR points.

[0019] According to an embodiment, the distance condition may include a condition that the LIDAR point is closest to a point corresponding to the host vehicle among the second LIDAR points.

[0020] According to an embodiment, the distance condition may include a first condition or a second condition, the first condition being that at least one third LIDAR point among the second LIDAR points corresponds to the minimum angle among the angles between each of the half-straight lines from the point corresponding to the main vehicle to the second LIDAR point and the half-straight line, and the second condition being that at least one third LIDAR point among the second LIDAR points is closest to the target fusion trajectory.

[0021] According to an embodiment, the point corresponding to the host vehicle may include a center point of a line segment representing a front bumper module of the host vehicle. The point corresponding to the target fusion track may include a center point of a track box representing the camera track.

[0022] According to an embodiment, the processor may identify, through the camera, points corresponding to the target fusion trajectory.

[0023] According to an embodiment, the program further includes instructions for identifying a category into which at least one fused trajectory is classified based on a portion of an image acquired by the camera.

[0024] According to an embodiment, the category into which at least one fused track may be classified represents a pedestrian. The target fused track has a longitudinal component of the distance from the point corresponding to the host vehicle to the at least one fused track, the longitudinal component being less than a specified distance in the at least one fused track identified as a pedestrian by the camera. The at least one target LIDAR track may include at least one candidate track that can correspond to the pedestrian represented by the target fused track. The at least one third LIDAR point may include a LIDAR point corresponding to the pedestrian represented by the target fused track.

[0025] According to an embodiment, the program may further include instructions for associating and storing the target fused track with the at least one third LIDAR point by changing the point of the target fused track to the point of the at least one third LIDAR point based on identifying the at least one third LIDAR point.

[0026] According to an embodiment of the present disclosure, an object recognition method may include: identifying at least one fused track corresponding to at least one external object acquired by a camera and a radar; identifying a target fused track in the at least one fused track based on a category to which the at least one fused track is classified, a longitudinal component of a distance from a point corresponding to a host vehicle to the at least one fused track, or any combination thereof; identifying at least one LIDAR track corresponding to the external object represented by the target fused track in the at least one external object acquired by the LIDAR; identifying at least one target LIDAR track in the at least one LIDAR track based on a distance from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track, wherein the point corresponding to the target fused track represents a target object recognized by the camera. a point corresponding to a host vehicle trajectory, representing at least one external object and identifying a target fused trajectory; identifying a second LIDAR point satisfying a position condition, an angle condition or any combination thereof among the first LIDAR points of the at least one target LIDAR trajectory, wherein the position condition is a position condition determined based on a position difference between the point corresponding to the target fused trajectory and the first LIDAR point, a position of the radar and a position of the camera, and the angle condition is an angle condition identified based on the half-line; at least one third LIDAR point satisfying a distance condition among the second LIDAR points, the distance condition being identified based on a distance from a point corresponding to the host vehicle to the second LIDAR point; and associating and storing the target fused trajectory with the at least one third LIDAR point based on identifying the at least one third LIDAR point.

[0027] According to an embodiment, identifying the target fused track in at least one fused track based on the category into which the at least one fused track is classified, the longitudinal component of the distance from the point corresponding to the host vehicle to the at least one fused track, or any combination thereof may include identifying the target fused track in at least one fused track based on at least one fused track identified via a camera and a radar, the at least one fused track being classified into a specified category, and the longitudinal component being identified as being less than a specified distance.

[0028] According to an embodiment, identifying at least one target LIDAR track in at least one LIDAR track based on a distance from a half-straight line from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track may include: identifying at least one target LIDAR track in at least one LIDAR track based on the half-straight line passing through a track box representing the at least one LIDAR track identified by the LIDAR or based on identifying that at least one of the vertical distances from the half-straight line to four vertices of the track box is less than or equal to a specified distance.

[0029] According to an embodiment, the position condition may include: a first value satisfies a first condition of a first specified range, the first value is identified based on a difference between a longitudinal position of the radar and a longitudinal position of the first LIDAR point and a difference between a lateral position of the camera and a lateral position of the first LIDAR point; and a second value satisfies a second condition of a second specified range different from the first specified range, the second value is identified based on a difference between a position of a center point of one of four line segments of a trajectory box representing the target fused trajectory and a position of the first LIDAR point.

[0030] According to an embodiment, the angle condition may include the following condition: in a state where the first LIDAR points are arranged in descending order of angles between each of the half-straight lines from the point corresponding to the host vehicle to the first LIDAR point and the half-straight line, among the first LIDAR points, the LIDAR points are included in the specified order.

[0031] According to an embodiment, the distance condition may include a condition that the LIDAR point is closest to a point corresponding to the host vehicle among the second LIDAR points.

[0032] According to an embodiment, the distance condition may include a first condition or a second condition, the first condition being that at least one third LIDAR point among the second LIDAR points corresponds to the minimum angle among the angles between each of the half-straight lines from the point corresponding to the main vehicle to the second LIDAR point, and the second condition being that at least one third LIDAR point among the second LIDAR points is closest to the target fusion trajectory.

[0033] According to an embodiment, the point corresponding to the host vehicle may include a center point of a line segment representing a front bumper module of the host vehicle, and the point corresponding to the target fused trajectory may include a center point of a trajectory box representing a camera trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The above and other purposes, features and advantages of the embodiments of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings, in which:

[0035] Figure 1 A block diagram showing a configuration of an object recognition device according to an embodiment of the present disclosure;

[0036] Figure 2 A flowchart showing an operation for identifying a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0037] Figure 3 A flowchart showing an operation for identifying a LIDAR point corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0038] Figure 4 An example of identifying a target LIDAR track including LIDAR points corresponding to a target fusion track in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown;

[0039] Figure 5 A flowchart showing an operation for associating and storing a target fusion trajectory with a LIDAR point in an object recognition device or an object recognition method according to an embodiment of the present disclosure;

[0040] Figure 6 An example of improving the position accuracy of an external object corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown;

[0041] Figure 7 An example of improving the position accuracy of an external object corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown;

[0042] Figure 8 An example showing a change in the scale of radar points corresponding to a target fusion trajectory related to an object recognition device or an object recognition method according to an embodiment of the present disclosure; and

[0043] Fig. 9 A computing system related to an object recognition device and an object recognition method according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0044] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to exemplary figures. When adding reference numerals to the components of each figure, it should be noted that the same or equivalent components are represented by the same numerals, even when they are shown on other figures. In addition, when describing the embodiments of the present disclosure, detailed descriptions of well-known features or functions will be omitted so as not to unnecessarily obscure the main points of the present disclosure.

[0045] When describing the components according to the embodiments of the present disclosure, terms such as first, second, "A", "B", (a), (b), etc. may be used. These terms are intended only to distinguish one component from another, and these terms do not limit the nature, order, or sequence of the constituent components. Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as those commonly understood by those skilled in the art to which the present disclosure belongs. Such terms as defined in commonly used dictionaries will be interpreted as having a meaning equivalent to the contextual meaning in the relevant technical field, and should not be interpreted as having an ideal or overly formal meaning unless explicitly defined as having a meaning in this application.

[0046] In the following, reference will be made to Figures 1 to 9 Embodiments of the present disclosure are described in detail.

[0047] Figure 1 is a block diagram showing a configuration of an object recognition device according to an embodiment of the present disclosure.

[0048] refer to Figure 1 , the object recognition device 101 according to an embodiment of the present disclosure may be implemented in a vehicle. In this case, the object recognition device 101 may be integrally formed with an internal control unit of the vehicle, or may be implemented as a separate device and connected to the control unit of the vehicle through a separate connection device.

[0049] refer to Figure 1 , the object recognition device 101 may include a camera 103 , a radar 105 , a LIDAR 107 and a processor 109 .

[0050] According to an embodiment, the processor 109 of the object recognition device 101 may recognize a track corresponding to an external object based on the camera 103, the radar 105, or the LIDAR 107. The processor 109 of the object recognition device 101 may recognize a track corresponding to an external object. A track may refer to an object in a tracking operation of recognizing whether an external object recognized in a specific frame is the same object as an external object included in at least one frame previously acquired before the specific frame. An identifier may be assigned to the track. A track may refer to a sensor-based output unit for tracking the recognized external object over time.

[0051] According to an embodiment, the processor 109 of the object recognition device 101 may recognize a camera track corresponding to an external object based on a portion of an image acquired through the camera 103. The processor 109 of the object recognition device 101 may recognize a radar track corresponding to an external object based on at least one radar point acquired through the radar 105. The processor 109 of the object recognition device 101 may recognize a LIDAR track corresponding to an external object based on at least one LIDAR point acquired through the LIDAR 107.

[0052] According to an embodiment, the processor 109 of the object recognition device 101 may identify a fused track based on at least two tracks corresponding to the same external object among the camera track, the radar track, and the LIDAR track. For example, the processor 109 of the object recognition device 101 may identify a fused track corresponding to an external object based on the camera track and the radar track.

[0053] It should be noted that the position accuracy of the fused trajectory when the LIDAR point corresponding to the fused trajectory and identified by the LIDAR 107 is not identified may be lower than the position accuracy of the fused trajectory when the LIDAR point corresponding to the fused trajectory and identified by the LIDAR 107 is identified. Therefore, the processor 109 of the object recognition device 101 may improve the position accuracy of the fused trajectory by identifying the LIDAR point corresponding to the fused trajectory. Because the calculation of the processor 109 is required to identify the LIDAR point corresponding to the fused trajectory, the processor 109 of the object recognition device 101 may identify the LIDAR point corresponding to the fused trajectory only for the fused trajectory that meets the specified condition to improve the calculation efficiency. The fused trajectory that meets the specified condition may be referred to as a target fused trajectory.

[0054] According to an embodiment, the specified condition may include a condition that the fused track is recognized by the camera 103 and the radar 105. The reason for this is that when the fused track is recognized by the LIDAR 107, even if the LIDAR point corresponding to the fused track is recognized, the position accuracy of the external object corresponding to the fused track is not significantly improved. Therefore, for computational efficiency, the processor 109 of the object recognition device 101 may recognize the fused track as a fused target track based on the fused track recognized by the camera 103 and the radar 105.

[0055] According to an embodiment, the specified condition may include a condition that the fused trajectory is classified into a specified category (e.g., a pedestrian). The processor 109 of the object recognition device 101 may identify the category into which at least one fused trajectory is classified based on a portion of the image acquired by the camera 103. The reason is that when a pedestrian is positioned to overlap with another external object in the field of view of the host vehicle, the processor 109 of the object recognition device 101 may identify the LIDAR point corresponding to the pedestrian identified by the LIDAR 107 as a LIDAR point corresponding to the external object. Compared with the fused trajectory identified as a non-pedestrian category, the fused trajectory identified as a pedestrian may often be identified as overlapping with another external object. In addition, in autonomous driving or driver-assisted driving, the importance of identifying pedestrians may be higher than the importance of identifying objects classified into other categories.

[0056] According to an embodiment, the specified condition may include a condition that a longitudinal component of a distance from a point of the fused trajectory corresponding to the host vehicle (e.g., a center point of a line segment representing a front bumper module of the host vehicle) to the fused trajectory is less than a specified distance (e.g., about 50 m). To improve computational efficiency, the processor 109 of the object recognition device 101 may not recognize a LIDAR point corresponding to a fused trajectory located outside the specified distance from the host vehicle.

[0057] According to an embodiment, the processor 109 of the object recognition device 101 may recognize a target LIDAR track among at least one LIDAR track recognized by the LIDAR 107 and corresponding to an external object based on a position of the external object corresponding to the target fused track.

[0058] The processor 109 of the object recognition device 101 may identify at least one target LIDAR track in at least one LIDAR track based on a distance from a half-straight line drawn from a point corresponding to the main vehicle (e.g., a center point of a line segment representing a front bumper module of the main vehicle) to a point corresponding to the target fused track (e.g., a center point of a track box representing a camera track) to at least one LIDAR track. The point corresponding to the target fused track may represent a point corresponding to a camera track recognized by a camera, representing at least one external object, and identifying the target fused track. According to an embodiment, the point corresponding to the target fused track may include a center point of a track box representing a camera track.

[0059] For example, when the half-line passes through a track box representing at least one LIDAR track recognized by the LIDAR 107 , the processor 109 of the object recognition device 101 may recognize the at least one LIDAR track corresponding to the track box as at least one target LIDAR track.

[0060] For example, when at least one of the vertical distances from the half-line to the four vertices of the trajectory box is less than or equal to a specified distance (e.g., approximately 1 m), the processor 109 of the object recognition device 101 may recognize at least one LIDAR trajectory corresponding to the trajectory box as at least one target LIDAR trajectory.

[0061] According to an embodiment, the processor 109 of the object recognition device 101 may recognize a point corresponding to the target fused track through the camera 103. According to an embodiment, the at least one target LIDAR track may include at least one candidate track that may correspond to an external object represented by the target fused track (e.g., an object classified as a pedestrian).

[0062] According to an embodiment, among the first LIDAR points constituting at least one target LIDAR trajectory, the processor 109 of the object recognition device 101 may identify a second LIDAR point that satisfies at least one of a position condition determined based on the difference between the position between the first LIDAR point and the point corresponding to the target fusion trajectory and the position of the radar 105 and the position of the camera 103, or an angle condition determined based on a half-straight line, or any combination thereof.

[0063] For example, the position condition may include a condition that a first value identified based on the difference between the longitudinal position of the radar 105 and the longitudinal position of the first LIDAR point and the difference between the lateral position of the camera 103 and the lateral position of the first LIDAR point satisfies a specified range (e.g., less than about 5 m), and a condition that a second value identified based on the difference between the position of the first LIDAR point and the position of the center point of one of the four line segments constituting the trajectory box representing the target fused trajectory satisfies a specified range different from the specified range (e.g., less than about 1.25 m).

[0064] The first value may be identified by the following equation 1. “l” may represent the longitudinal position of the radar 105. 1 " may represent the longitudinal position of the first LIDAR point. "c" may represent the lateral position of the camera 103. 2 ” may represent the lateral position of the first LIDAR point.

[0065] Equation 1:

[0066]

[0067] For example, the angle condition may include a condition that when the first LIDAR points are arranged in descending order of angles between half-straight lines from a point corresponding to the host vehicle to the first LIDAR points and each of the half-straight lines, the LIDAR points in the first LIDAR points are included in the specified order.

[0068] According to an embodiment, the processor 109 of the object recognition device 101 may identify at least one third LIDAR point satisfying a distance condition identified based on a distance from a point corresponding to the host vehicle (e.g., a center point of a line segment representing a front bumper module of the host vehicle) to the second LIDAR point among the second LIDAR points. The at least one third LIDAR point may include a LIDAR point corresponding to a pedestrian, which is a category of the target fused trajectory.

[0069] For example, the distance condition may include a condition that, among the second LIDAR points, the LIDAR point is closest in distance to a point corresponding to the host vehicle.

[0070] For another example, the distance condition may include a condition that at least one third LIDAR point among the second LIDAR points corresponds to a minimum angle among angles between each of the half-straight lines from the point corresponding to the host vehicle to the second LIDAR point and the half-straight line, and a condition that at least one third LIDAR point among the second LIDAR points is closest to the target fused trajectory.

[0071] According to an embodiment, the processor 109 of the object recognition device 101 may associate and store the target fused trajectory with the at least one third LIDAR point based on identifying the at least one third LIDAR point.

[0072] In automated driving or driving based on driver assistance devices, pedestrian recognition may be required to ensure safety.

[0073] When the position accuracy of the fused trajectory corresponding to the external object classified as a pedestrian is less than a reference value, the processor of the object recognition device may over-control the host vehicle to perform sudden braking or lose control of the host vehicle. In addition, when the category of the fused trajectory corresponding to the external object other than the pedestrian is erroneously identified as a pedestrian, the processor of the object recognition device may over-control the host vehicle to perform emergency braking or may lose control of the host vehicle.

[0074] When the fused track corresponding to the external object is classified by the camera, the processor of the object recognition device may mistakenly identify the category of the fused track. When the position of the fused track corresponding to the external object is recognized by the camera, the processor of the object recognition device may mistakenly identify the position of the fused track. Therefore, the processor of the object recognition device may fuse the fused track with the information recognized by the LIDAR to reduce the frequency of mistakenly identifying the category of the fused track and improve the position accuracy of the fused track.

[0075] The object recognition device according to the embodiment may associate the target fused track with the LIDAR point instead of associating the target fused track with the LIDAR track. Although the LIDAR track identifies a single external object, the LIDAR track may be divided into a plurality of tracks or filtered due to low reliability. When the target fused track and the LIDAR track are associated with each other, the processor of the object recognition device may associate the target fused track with the LIDAR point instead of associating the target fused track with the LIDAR track, thereby improving the position accuracy of the fused track.

[0076] Figure 2 A flowchart of an operation for identifying a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0077] In the following, it is assumed Figure 1 The object recognition device 101 performs Figure 2In addition, Figure 2 In the description, the operations described as being performed by the object recognition device may be understood to be controlled by the processor 109 of the object recognition device 101 .

[0078] refer to Figure 2 In the first operation 201, the processor of the object recognition device according to the embodiment may recognize two or more tracks through two or more sensors. For example, the processor of the object recognition device may recognize a camera track, which is a track obtained by a camera. The processor of the object recognition device may recognize a radar track, which is a track obtained by a radar. The processor of the object recognition device may recognize a LIDAR track, which is a track obtained by a LIDAR.

[0079] In the second operation 211, the processor of the object recognition device according to the embodiment may generate a fused trajectory through trajectory fusion. The second operation 211 may include a third operation 213, a fourth operation 215, a fifth operation 217, and a sixth operation 219. The module performing the second operation 211 may include a module performing the third operation 213, a module performing the fourth operation 215, a module performing the fifth operation 217, and a module performing the sixth operation 219.

[0080] For example, the processor of the object recognition device may generate a fused trajectory by fusing at least two of the camera trajectory, the radar trajectory, and the LIDAR trajectory. It should be noted that the fused trajectory may represent the same external object. In other words, the processor of the object recognition device may generate a fused trajectory by fusing at least two of the camera trajectory representing a specific object, the radar trajectory representing an object identical to the specific object, and the LIDAR trajectory representing an object identical to the specific object.

[0081] In the third operation 213, the processor of the object recognition device according to the embodiment may perform a preprocessing process. According to the embodiment, the processor of the object recognition device may specify a valid track among the tracks recognized by the sensor. For example, the processor of the object recognition device may identify a track whose signal strength representing the track is equal to or greater than the reference strength as a valid track.

[0082] In the fourth operation 215, the processor of the object recognition device according to the embodiment may perform a prediction process. The processor of the object recognition device may predict and recognize the relative position of the track relative to the host vehicle that changes according to the operation of the host vehicle. The processor of the object recognition device may recognize the predicted position of the track by adding the compensation position value to the recognized position of the track or subtracting the compensation position value from the recognized position of the track.

[0083] In the fifth operation 217, the processor of the object recognition device according to the embodiment may perform an association process. The processor of the object recognition device may identify the reference trajectory and the reference trajectory among the trajectories recognized by the sensor and representing the same external object. The processor of the object recognition device may fuse information about the trajectories representing the same external object and recognize them as one fused trajectory.

[0084] In the sixth operation 219, the processor of the object recognition device according to the embodiment may perform a track adjustment process. The processor of the object recognition device may manage the states of the recognized fused track, merged track, deleted track, and adjusted track position.

[0085] The operation of associating the fused trajectory with the LIDAR point disclosed in the embodiment of the present disclosure may be included in the third operation 213, the fifth operation 217, or the sixth operation 219. Figure 3 The operation of associating the fused trajectory with the LIDAR point disclosed in the embodiment of the present disclosure is described.

[0086] When the operation of associating the fused trajectory with the LIDAR point disclosed in the embodiment of the present disclosure is included in the third operation, the trajectory representing the external object can be effectively identified even if the signal strength of the trajectory is weak, resulting in a reduction in the frequency of deleting the trajectory representing the external object. The processor of the object recognition device can improve the performance of the object recognition device by considering the trajectory representing the external object when generating the fused trajectory.

[0087] Figure 3 A flowchart is shown of an operation for identifying LIDAR points corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure.

[0088] In the following, it is assumed Figure 1 The object recognition device 101 performs Figure 3 In addition, Figure 3 In the description, the operations described as being performed by the object recognition device may be understood to be controlled by the processor 109 of the object recognition device 101 .

[0089] refer to Figure 3 In a first operation 301, a processor of an object recognition device according to an embodiment may recognize a target fusion trajectory. The first operation 301 may include a second operation 303, a third operation 305, and a fourth operation 307.

[0090] In a second operation 303 , the processor of the object recognition device according to an embodiment may recognize at least one fused track through the camera and the radar.

[0091] Since the reliability of the fused track identified by the camera and the radar is higher than the reliability of the fused track identified by one of the camera and the radar when the category of the fused track is falsely identified, it may be difficult for the processor of the object recognition device to determine whether the fused track is falsely identified. The reliability of the fused track may refer to a weight given by the object recognition device to identify whether the fused track is a false track.

[0092] Furthermore, when the position of the fused trajectory is erroneously identified, the validity of the fused trajectory identified by the camera and the radar is higher than the validity of the fused trajectory identified by one of the camera and the radar. The validity of the fused trajectory may refer to the probability that the fused trajectory is an erroneous trajectory.

[0093] In the third operation 305, the processor of the object recognition device according to the embodiment may recognize the category of the fused trajectory. The processor of the object recognition device may recognize the category (eg, pedestrian) into which at least one fused trajectory is classified based on a portion of the image acquired by the camera.

[0094] In the fourth operation 307, the processor of the object recognition device according to the embodiment may identify the longitudinal component from the point corresponding to the host vehicle to the fused trajectory. The processor of the object recognition device may include a condition that the longitudinal component of the distance from the point of the fused trajectory corresponding to the host vehicle (e.g., the center point of the line segment representing the front bumper module of the host vehicle) to the fused trajectory is less than a specified distance (e.g., about 50m). In order to improve computational efficiency, the processor of the object recognition device may not identify LIDAR points corresponding to the fused trajectory located outside the specified distance from the host vehicle.

[0095] In the fifth operation 311, the processor of the object recognition device according to an embodiment may identify at least one target LIDAR track. The fifth operation 311 may include a sixth operation 313, a seventh operation 315, and an eighth operation 317. According to an embodiment, the at least one target LIDAR track may include at least one candidate track that may correspond to an external object represented by the target fused track (e.g., an object classified as a pedestrian).

[0096] In a sixth operation 313, the processor of the object recognition device according to an embodiment may recognize at least one LIDAR track through LIDAR. The LIDAR track may include a LIDAR point.

[0097] In the seventh operation 315, the processor of the object recognition device according to the embodiment may recognize a half-line based on the point corresponding to the target fusion trajectory. The recognized half-line may point from the point corresponding to the host vehicle (e.g., the center point of the line segment representing the front bumper module of the host vehicle) to the point corresponding to the target fusion trajectory (e.g., the center point of the trajectory box representing the camera trajectory). The point corresponding to the target fusion trajectory may represent a point corresponding to the camera trajectory recognized by the camera, representing at least one external object, and recognizing the target fusion trajectory.

[0098] In an eighth operation 317, the processor of the object recognition device according to an embodiment may identify a distance from the half-line to at least one LIDAR track. The processor of the object recognition device may identify at least one target LIDAR track among the at least one LIDAR track based on the distance from the half-line corresponding to the point of the host vehicle (e.g., the center point of the line segment representing the front bumper module of the host vehicle) to the point corresponding to the target fused track (e.g., the center point of the track box representing the camera track) to the at least one LIDAR track.

[0099] For example, when the half-line passes through a track box representing at least one LIDAR track recognized by the LIDAR, the processor of the object recognition device may recognize the at least one LIDAR track corresponding to the track box as at least one target LIDAR track.

[0100] For example, when at least one of the vertical distances from the half-line to the four vertices of the trajectory box is less than or equal to a specified distance (e.g., approximately 1 m), the processor of the object recognition device may recognize at least one LIDAR track corresponding to the trajectory box as at least one target LIDAR track.

[0101] In a ninth operation 321, the processor of the object recognition device according to an embodiment may recognize a third LIDAR point. The ninth operation 321 may include a tenth operation 323, an eleventh operation 325, and a twelfth operation 327. The third LIDAR point may represent a pedestrian, which is a category of the target fused trajectory.

[0102] In a tenth operation 323, the processor of the object recognition device according to an embodiment may recognize at least one first LIDAR point. The first LIDAR point may be included in at least one target LIDAR track.

[0103] In the eleventh operation 325, the processor of the object recognition device according to an embodiment may identify at least one second LIDAR point. According to an embodiment, among the first LIDAR points constituting at least one target LIDAR track, the processor of the object recognition device may identify a second LIDAR point that satisfies at least one of a position condition determined based on a difference between a position between the first LIDAR point and a point corresponding to the target fused track and a position of the radar and a position of the camera, or an angle condition determined based on a half straight line, or any combination thereof. A specified number of second LIDAR points (e.g., two) may be identified.

[0104] For example, the position condition may include a condition that a first value identified based on a difference between the longitudinal position of the radar and the longitudinal position of the first LIDAR point and a difference between the lateral position of the camera and the lateral position of the first LIDAR point satisfies a specified range (e.g., less than approximately 5 m), and a condition that a second value identified based on a difference between the position of the first LIDAR point and the position of a center point of one of four line segments constituting a trajectory box representing the target fused trajectory satisfies a specified range different from the specified range (e.g., less than approximately 1.25 m).

[0105] For example, the angle condition may include a condition that when the first LIDAR points are arranged in descending order of angles between each of the half-straight lines from the point corresponding to the host vehicle to the first LIDAR points and the half-straight line, the LIDAR points in the first LIDAR points are included in the specified order.

[0106] In the twelfth operation 327, the processor of the object recognition device according to an embodiment may identify at least one third LIDAR point. According to an embodiment, the processor of the object recognition device may identify at least one third LIDAR point satisfying a distance condition identified based on a distance from a point corresponding to the host vehicle (e.g., a center point of a line segment representing a front bumper module of the host vehicle) to the second LIDAR point among the second LIDAR points. The at least one third LIDAR point may include a LIDAR point corresponding to a pedestrian, which is a category of the target fused trajectory.

[0107] For example, the distance condition may include a condition that, among the second LIDAR points, the LIDAR point is closest in distance to a point corresponding to the host vehicle.

[0108] For another example, the distance condition may include a condition that at least one third LIDAR point among the second LIDAR points corresponds to a minimum angle among angles between each of the half-straight lines from the point corresponding to the host vehicle to the second LIDAR point, or a condition that the LIDAR point is closest to the target fused trajectory among the second LIDAR points.

[0109] In a thirteenth operation 329, the processor of the object recognition device according to an embodiment may associate and store the target fused trajectory with the third LIDAR point. The processor of the object recognition device may store information about the third LIDAR point and the target fused trajectory.

[0110] Figure 4 An example of identifying a target LIDAR track including LIDAR points corresponding to a target fused track in an object recognition apparatus or an object recognition method according to an embodiment of the present disclosure is shown.

[0111] refer to Figure 4 , in the case 401, the half straight line 415 may point from the point 403 corresponding to the host vehicle to the point 405 corresponding to the target fused track. The first LIDAR track 407, the second LIDAR track 409, the third LIDAR track 411, and the fourth LIDAR track 413 may be obtained through LIDAR. The processor of the object recognition device may identify at least one candidate track that may correspond to the external object indicated by the target fused track among the first LIDAR track 407, the second LIDAR track 409, the third LIDAR track 411, and the fourth LIDAR track 413. The at least one candidate track may be referred to as a target LIDAR track.

[0112] According to an embodiment, the point 405 corresponding to the target fused trajectory may mean a point corresponding to a camera trajectory recognized by a camera, representing at least one external object, and recognizing the target fused trajectory.

[0113] According to one embodiment, the processor of the object recognition device may identify at least one target LIDAR track among the at least one LIDAR track based on identifying that the half straight line 415 passes through a track box representing at least one LIDAR track recognized by the LIDAR (e.g., the first LIDAR track 407, the second LIDAR track 409, the third LIDAR track 411, and the fourth LIDAR track 413), or at least one of the vertical distances from the half straight line 415 to four vertices of the track box is less than or equal to a specified distance.

[0114] The processor of the object recognition device may identify the first LIDAR track 407 as a target LIDAR track based on identifying that the half-line 415 passes through the track box represented by the first LIDAR track 407, or at least one of the vertical distances from the half-line 415 to the four vertices of the track box represented by the first LIDAR track 407 is less than or equal to a specified distance.

[0115] The processor of the object recognition device may identify the second LIDAR track 409 as a target LIDAR track based on identifying that at least one of the vertical distances from the half-line 415 to the four vertices of the track box represented by the second LIDAR track 409 is less than or equal to a specified distance.

[0116] The processor of the object recognition device may identify the third LIDAR trajectory 411 as a target LIDAR trajectory based on identifying that the half-line 415 passes through the trajectory box represented by the third LIDAR trajectory 411 and at least one of the vertical distances from the half-line 415 to the four vertices of the trajectory box represented by the third LIDAR trajectory 411 is less than or equal to a specified distance.

[0117] The processor of the object recognition device may identify the fourth LIDAR track 413 as a target LIDAR track based on identifying that a distance from the half-line 415 to one of four vertices of the track box represented by the fourth LIDAR track 413 is less than a specified distance.

[0118] According to an embodiment, the processor of the object recognition device may identify an equation representing the half line 415 using Equation 2. "a" may represent a longitudinal position of the target fused track. "b" may represent a lateral position of the target fused track.

[0119] Equation 2:

[0120] a·xb·y+c=0

[0121] According to an embodiment, the azimuth angle accuracy recognized by the camera may be higher than the longitudinal position accuracy recognized by the camera. Therefore, the processor of the object recognition device may recognize an equation representing the half line 415 based on the angle recognized by the camera.

[0122] Figure 5 A flowchart showing an operation of associating and storing a target fusion trajectory with a LIDAR point in an object recognition device or an object recognition method according to an embodiment of the present disclosure.

[0123] In the following, it is assumed Figure 1 The object recognition device 101 performs Figure 5 In addition, Figure 5 In the description, the operations described as being performed by the object recognition device may be understood to be controlled by the processor 109 of the object recognition device 101 .

[0124] refer to Figure 5 In a first operation 501, a processor of an object recognition device according to an embodiment may recognize at least one fused track through a camera and a radar. The at least one fused track may correspond to at least one external object.

[0125] In a second operation 503, the processor of the object recognition device according to an embodiment may recognize a target fused trajectory among the at least one fused trajectory based on a category into which the at least one fused trajectory is classified or a longitudinal component of a distance from a point corresponding to the host vehicle to the at least one fused trajectory, or any combination thereof.

[0126] In a third operation 505, the processor of the object recognition device according to an embodiment may recognize at least one LIDAR track through LIDAR. The LIDAR track may correspond to the external object represented by the target fused track.

[0127] In a fourth operation 507, the processor of the object recognition device according to an embodiment may identify at least one target LIDAR track in at least one LIDAR track. The processor of the object recognition device may identify at least one target LIDAR track in at least one LIDAR track based on a distance from a half-straight line from a point corresponding to the main vehicle toward a point corresponding to the target fused track to the at least one LIDAR track. In other words, the processor of the object recognition device may determine a LIDAR track that satisfies a condition as a target LIDAR track in at least one LIDAR track. The processor of the object recognition device may identify whether the LIDAR track satisfies a condition based on a distance from a half-straight line from a point corresponding to the main vehicle toward a point corresponding to the target fused track to the at least one LIDAR track. When the processor of the object recognition device according to an embodiment fails to identify a target LIDAR track that satisfies the condition in the LIDAR track, the processor of the object recognition device may end the operation for associating and storing the target fused track with the LIDAR point.

[0128] In a fifth operation 509, the processor of the object recognition device according to the embodiment may identify a second LIDAR point that satisfies at least one of a position condition or an angle condition or any combination thereof among the first LIDAR points constituting the at least one target LIDAR track. The position condition may be determined based on a difference between a position of a point corresponding to the target fused track and the first LIDAR point, a position of the radar, and a position of the camera. The angle condition may be identified based on a half-straight line.

[0129] In the sixth operation 511, the processor of the object recognition device according to the embodiment may identify at least one third LIDAR point satisfying the distance condition among the second LIDAR points. The distance condition may be identified based on the distance from the point corresponding to the host vehicle to the second LIDAR point.

[0130] In a seventh operation 513, the processor of the object recognition device according to an embodiment may associate and store the target fused trajectory with at least one third LIDAR point. The processor of the object recognition device may associate and store the target fused trajectory with at least one third LIDAR point based on identifying at least one third LIDAR point.

[0131] The processor of the object recognition device according to an embodiment may associate and store the target fused trajectory with the at least one third LIDAR point by changing the point of the target fused trajectory to the point of the at least one third LIDAR point based on recognizing the at least one third LIDAR point.

[0132] Figure 6 An example of improving the position accuracy of an external object corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0133] refer to Figure 6 , on the camera screen 601, the external object corresponding to the vehicle may overlap with the external object corresponding to the pedestrian.

[0134] The first output screen 611 may include a first fused track 613 representing a vehicle, a first target fused track 615 representing a pedestrian, a first LIDAR track 617 representing a vehicle, and a first half straight line 619. The first half straight line 619 may point from a point corresponding to the host vehicle to a point corresponding to the first target fused track 615.

[0135] The second output screen 621 includes a second fused track 623 representing a vehicle, a second target fused track 625 representing a pedestrian, a second LIDAR track 627 representing a vehicle, and a second half straight line 629. The second half straight line 629 may point from a point corresponding to the host vehicle to a point corresponding to the second target fused track 625.

[0136] According to an embodiment, the point corresponding to the first target fused track 615 may mean a point corresponding to the first camera track recognized by the camera, representing at least one external object, and recognizing the first target fused track.

[0137] According to an embodiment, the point corresponding to the second target fused track 625 may mean a point corresponding to the second camera track recognized by the camera, representing at least one external object, and recognizing the second target fused track.

[0138] According to an embodiment, the first output screen 611 may represent a screen output by an existing object recognition device. The LIDAR point representing the pedestrian may be included in the first LIDAR track 617. Therefore, the processor of the object recognition device may not output a LIDAR track different from the first LIDAR track 617 representing the pedestrian through LIDAR. Because the processor of the object recognition device cannot recognize the LIDAR information on the first target fused track 615, the position accuracy of the first target fused track 615 may be lower than that of the target fused track containing the LIDAR information.

[0139] The second output screen 621 may represent a screen output by the object recognition device according to an embodiment. The LIDAR point representing the pedestrian may be included in the second LIDAR track 627. The processor of the object recognition device may identify a target LIDAR track (e.g., the second LIDAR track 627) that may correspond to the second target fused track 625 in the LIDAR track based on the position of the second half straight line 629.

[0140] The processor of the object recognition device may identify at least one second LIDAR point that satisfies the position condition or angle condition from among at least one first LIDAR point included in the target LIDAR track. The processor of the object recognition device may identify a third LIDAR point that satisfies the distance condition from among at least one second LIDAR point. The processor of the object recognition device may associate and store the second target fused track 625 with the third LIDAR point.

[0141] Figure 7 An example of improving the position accuracy of an external object corresponding to a target fusion trajectory in an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0142] refer to Figure 7 , on the camera screen 701, the external object corresponding to the vehicle may overlap with the external object corresponding to the bushes.

[0143] The first output screen 711 may include a camera track 715 representing bushes, a first target fused track 719 identified based on radar points 717 representing bushes, and a first LIDAR track 713 representing a structure installed at a road boundary behind the bushes.

[0144] The second output screen 721 may include a second target fused track 727 corresponding to the bushes, a second LIDAR track 725 representing a structure installed at a road boundary behind the bushes, and a second half straight line 723. The half straight line may point from a point corresponding to the host vehicle to a point corresponding to the second target fused track 727. According to an embodiment, the point corresponding to the second target fused track 727 may represent a point corresponding to a camera track recognized by a camera, representing at least one external object, and recognizing the second target fused track 727.

[0145] According to an embodiment, the first output screen 711 may represent a screen output by an existing object recognition device. Due to the shaking of the bushes, the processor of the object recognition device may obtain a camera track 715 representing the bushes and a first target fused track 719 that is erroneously recognized based on a radar point 717 representing the bushes. Because the first target fused track 719 is a track recognized by two or more sensors, the reliability of the first target fused track 719 may be higher than the reliability of a track recognized by a single sensor. Therefore, the processor of the object recognition device may output the first target fused track 719. According to the output of the first target fused track 719, the host vehicle may decelerate or brake to avoid the erroneous object.

[0146] The second output screen 721 may represent a screen output by the object recognition device according to an embodiment. The LIDAR points representing the bushes may be included in the second LIDAR track 725. The processor of the object recognition device may identify a target LIDAR track (e.g., the second LIDAR track 725) that may correspond to the second target fused track 727 in the LIDAR track based on the position of the second half straight line 723.

[0147] The processor of the object recognition device may identify at least one second LIDAR point satisfying a position condition or an angle condition from at least one first LIDAR point included in the target LIDAR track (e.g., the second LIDAR track 725). The processor of the object recognition device may identify a third LIDAR point satisfying a distance condition from at least one second LIDAR point. The processor of the object recognition device may associate and store the second target fused track 727 with the third LIDAR point.

[0148] The processor of the object identification device may associate second target fused track 727 with the LIDAR points, thereby improving the positional accuracy of second target fused track 727 representing the bushes compared to the positional accuracy of first target fused track 719 .

[0149] Figure 8 An example of a change in the scale of radar points corresponding to a target fusion trajectory related to an object recognition device or an object recognition method according to an embodiment of the present disclosure is shown.

[0150] refer to Figure 8 , the screen 801 may include LIDAR points 803 acquired through LIDAR, camera tracks 805 acquired through the camera, radar points 807 acquired through radar, target fusion tracks 809 and radar detection points 811.

[0151] According to an embodiment, the processor of the object recognition device may identify at least one fused track based on the camera track 805 and the radar point 807. The processor of the object recognition device may identify a target fused track 809 satisfying a specified condition among the at least one fused track to improve computational efficiency.

[0152] According to an embodiment, the processor of the object recognition device may recognize the target LIDAR track among at least one LIDAR track recognized by the LIDAR and corresponding to the external object based on the position of the external object corresponding to the target fused track 809. The processor of the object recognition device may recognize the LIDAR point 803 representing the same external object as the external object represented by the target fused track 809 classified into the category corresponding to the pedestrian.

[0153] According to one embodiment, the processor of the object recognition device may identify at least one target LIDAR track among the at least one LIDAR track based on a distance from a semi-straight line from a point corresponding to the host vehicle (e.g., a center point of a line segment representing a front bumper module of the host vehicle) to a point corresponding to the target fused track 809 (e.g., a center point of a track box representing a camera track recognized by a camera representing at least one external object and identifying the target fused track 809) to the at least one LIDAR track.

[0154] According to an embodiment, the processor of the object recognition device may identify a LIDAR point that satisfies a position condition, an angle condition, or a distance condition among the LIDAR points included in the target LIDAR trajectory. The processor of the object recognition device may be used in an associated operation (e.g., Figure 2 In a fifth operation 217), the identified LIDAR points are associated with the target fusion trajectory.

[0155] It should be noted that when the category of the fused track is a vehicle, the processor of the object recognition device may identify the fused track based on the radar detection points and the camera track and the radar points to associate the fused track with the LIDAR points, and the processor of the object recognition device may perform the associating operation (e.g., Figure 2 In the fifth operation 217 of the method, the target fusion track is associated with its corresponding LIDAR point. When the target fusion track is associated with the corresponding LIDAR point, the radar detection point can be less affected by the external environment such as exhaust gas or water splash, thereby improving the performance of the object recognition device in identifying the position of the external object.

[0156] Fig. 9 A computing system related to an object recognition device and an object recognition method according to an embodiment of the present disclosure is shown.

[0157] refer to Fig. 9The computing system 900 may include at least one processor 910 , a memory 930 , a user interface input device 940 , a user interface output device 950 , a storage device (ie, memory) 960 , and a network interface 970 , which are connected to each other via a bus 920 .

[0158] The processor 910 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 930 and / or the storage device 960. The memory 930 and the storage device 960 may include various types of volatile or non-volatile storage media. For example, the memory 930 may include a ROM (Read Only Memory) 931 and a RAM (Random Access Memory) 932.

[0159] Therefore, the operations of the methods or algorithms described in conjunction with the embodiments disclosed herein may be directly embodied in a hardware or software module, or a combination thereof, executed by the processor 910. The software module may reside on a storage medium (i.e., the memory 930 and / or the storage device 960) such as RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, and CD-ROM.

[0160] An exemplary storage medium may be coupled to the processor 910, and the processor 910 may read information from the storage medium and may record information in the storage medium. Alternatively, the storage medium may be integrated with the processor 910. The processor and the storage medium may reside in an application specific integrated circuit (ASIC). The ASIC may reside in a user terminal. In another case, the processor and the storage medium may reside in a user terminal as separate components.

[0161] The above description merely illustrates the technical concept of the embodiments of the present disclosure, and those skilled in the art to which the present disclosure belongs may make various modifications and changes without departing from the basic features of the embodiments of the present disclosure.

[0162] Therefore, the embodiments disclosed in the present disclosure are not intended to limit the technical concepts of the embodiments of the present disclosure, but to describe exemplary embodiments of the present disclosure, and the scope of the technical concepts of the embodiments of the present disclosure is not limited by the embodiments. The protection scope of the present disclosure should be interpreted by the following claims, and all technical concepts within the scope equivalent thereto should be interpreted as included within the scope of the present disclosure.

[0163] Embodiments of the present technology may identify LIDAR points corresponding to a fused track of an object identified by at least one sensor.

[0164] In addition, embodiments of the present technology may improve the position accuracy of an external object corresponding to a target fused trajectory identified by at least one sensor.

[0165] Additionally, embodiments of the present technology may reduce the frequency of erroneous braking of an autonomous vehicle or a vehicle with an activated driver assistance device by improving the position accuracy of an external object corresponding to a target fused trajectory identified by at least one sensor.

[0166] In addition, embodiments of the present technology may reduce the misidentification frequency of the category of the target fused trajectory by improving the position accuracy of the external object corresponding to the target fused trajectory recognized by at least one sensor.

[0167] Furthermore, various effects directly or indirectly understood through the present disclosure can be provided.

[0168] Although the embodiments of the present disclosure have been described above with reference to the exemplary embodiments and the accompanying drawings, the embodiments of the present disclosure are not limited thereto, but various modifications and changes may be made to the embodiments of the present disclosure by those skilled in the art to which the present disclosure belongs without departing from the spirit and scope of the present disclosure as claimed in the following claims.

Claims

1. An object recognition device, the device comprising: LIDAR; camera; radar; one or more processors; and A storage device storing a program to be executed by the one or more processors, the program comprising instructions for: identifying at least one fused track corresponding to at least one external object acquired by the camera and the radar; identifying a target fused track among the at least one fused track based on the class into which the at least one fused track is classified, a longitudinal component of a distance from a point corresponding to a host vehicle to the at least one fused track, or any combination thereof; identifying at least one LIDAR track corresponding to the external object represented by the target fused track among the at least one external object acquired by the LIDAR; identifying at least one target LIDAR track in the at least one LIDAR track based on a distance from a half-straight line from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track, wherein the point corresponding to the target fused track represents a point corresponding to a camera track identified by the camera, represents the at least one external object and identifies the target fused track; Identify a second LIDAR point satisfying a position condition, an angle condition, or any combination thereof, from among the first LIDAR points of the at least one target LIDAR track, wherein the position condition is a position condition determined based on a position difference between a point corresponding to the target fused track and the first LIDAR point, a position of the radar, and a position of the camera, and the angle condition is an angle condition identified based on the half-line; identifying at least one third LIDAR point among the second LIDAR points that satisfies a distance condition, the distance condition being identified based on a distance from a point corresponding to the host vehicle to the second LIDAR point; as well as The target fused trajectory is associated and stored with the at least one third LIDAR point based on identifying the at least one third LIDAR point.

2. The apparatus of claim 1 , wherein the program further comprises instructions for identifying the target fused track among the at least one fused track based on the at least one fused track identified via the camera and the radar, the at least one fused track being classified into a specified category, and identifying that the longitudinal component is less than a specified distance.

3. The apparatus of claim 1 , wherein the program further comprises instructions for identifying the at least one target LIDAR track in the at least one LIDAR track based on a half-line passing through a track box or based on identifying that at least one of perpendicular distances from the half-line to four vertices of the track box is less than or equal to a specified distance, the track box representing the at least one LIDAR track identified by the LIDAR.

4. The apparatus of claim 1, wherein the location condition comprises: a first value satisfies a first condition of a first specified range, the first value being identified based on a difference between a longitudinal position of the radar and a longitudinal position of the first LIDAR point and a difference between a lateral position of the camera and a lateral position of the first LIDAR point; and A second value satisfies a second condition of a second specified range different from the first specified range, and the second value is identified based on a difference between a position of a center point of one of four line segments of a trajectory box representing the target fused trajectory and a position of the first LIDAR point.

5. The apparatus of claim 1 , wherein the angle condition comprises a condition of including LIDAR points within a specified order among the first LIDAR points in a state where the first LIDAR points are arranged in descending order of angles between each of half straight lines from a point corresponding to the host vehicle to the first LIDAR points and the half straight lines. 6 . The apparatus of claim 1 , wherein the distance condition comprises a condition that a LIDAR point is closest to a point corresponding to the host vehicle among the second LIDAR points.

7. The apparatus according to claim 1, wherein the distance condition comprises a first condition or a second condition, the first condition being that the at least one third LIDAR point among the second LIDAR points corresponds to a minimum angle among angles between each of the half-straight lines from the point corresponding to the host vehicle to the second LIDAR point and the half-straight line, and the second condition being that the at least one third LIDAR point among the second LIDAR points is closest to the target fused trajectory.

8. The apparatus of claim 1, wherein: The point corresponding to the host vehicle includes a center point of a line segment representing a front bumper module of the host vehicle; and The point corresponding to the target fusion trajectory includes a center point of a trajectory box representing the camera trajectory.

9. The apparatus of claim 1, wherein the program further comprises: Instructions for identifying, with the camera, a point corresponding to the target fused trajectory.

10. The apparatus of claim 1, wherein the program further comprises: Instructions for identifying a category into which the at least one fused trajectory is classified based on a portion of an image acquired by the camera.

11. The apparatus of claim 1, wherein: The category into which the at least one fused trajectory is classified represents a pedestrian; the target fused track having a longitudinal component of a distance from a point corresponding to the host vehicle to the at least one fused track, the longitudinal component being less than a specified distance in the at least one fused track identified as the pedestrian by the camera; The at least one target LIDAR track includes at least one candidate track that can correspond to the pedestrian represented by the target fused track; and The at least one third LIDAR point includes a LIDAR point corresponding to the pedestrian represented by the target fused trajectory.

12. The apparatus of claim 1 , wherein the program further comprises instructions for associating and storing the target fused track with the at least one third LIDAR point by changing a point of the target fused track to a point of the at least one third LIDAR point based on identifying the at least one third LIDAR point.

13. A method for object recognition, the method comprising: identifying at least one fused track corresponding to at least one external object acquired by the camera and the radar; identifying a target fused track among the at least one fused track based on the class into which the at least one fused track is classified, a longitudinal component of a distance from a point corresponding to a host vehicle to the at least one fused track, or any combination thereof; identifying at least one LIDAR track corresponding to the external object represented by the target fused track among the at least one external object acquired by LIDAR; identifying at least one target LIDAR track in the at least one LIDAR track based on a distance from a half-straight line from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track, wherein the point corresponding to the target fused track represents a point corresponding to a camera track identified by the camera, represents the at least one external object and identifies the target fused track; Identify a second LIDAR point satisfying a position condition, an angle condition, or any combination thereof, from among the first LIDAR points of the at least one target LIDAR track, wherein the position condition is a position condition determined based on a position difference between a point corresponding to the target fused track and the first LIDAR point, a position of the radar, and a position of the camera, and the angle condition is an angle condition identified based on the half-line; identifying at least one third LIDAR point among the second LIDAR points that satisfies a distance condition, the distance condition being identified based on a distance from a point corresponding to the host vehicle to the second LIDAR point; as well as The target fused trajectory is associated and stored with the at least one third LIDAR point based on identifying the at least one third LIDAR point.

14. The method according to claim 13, wherein: Identifying the target fused track in the at least one fused track based on the category into which the at least one fused track is classified, the longitudinal component of the distance from the point corresponding to the host vehicle to the at least one fused track, or any combination thereof includes: The target fused track is identified in the at least one fused track based on the at least one fused track identified via the camera and the radar, the at least one fused track is classified into a specified category, and the longitudinal component is identified as being less than a specified distance.

15. The method according to claim 13, wherein: Identifying at least one target LIDAR track in the at least one LIDAR track based on a distance from a half-line from a point corresponding to the host vehicle to a point corresponding to the target fused track to the at least one LIDAR track includes: The at least one target LIDAR track is identified in the at least one LIDAR track based on the half-line passing through a track box or based on identifying that at least one of the perpendicular distances from the half-line to four vertices of the track box is less than or equal to a specified distance, the track box representing the at least one LIDAR track identified by the LIDAR.

16. The method of claim 13, wherein the location condition comprises: a first value satisfies a first condition of a first specified range, the first value being identified based on a difference between a longitudinal position of the radar and a longitudinal position of the first LIDAR point and a difference between a lateral position of the camera and a lateral position of the first LIDAR point; and A second value satisfies a second condition of a second specified range different from the first specified range, and the second value is identified based on a difference between a position of a center point of one of four line segments of a trajectory box representing the target fused trajectory and a position of the first LIDAR point.

17. The method according to claim 13, wherein the angle condition comprises a condition of including LIDAR points within a specified order among the first LIDAR points in a state where the first LIDAR points are arranged in descending order of angles between each of half-straight lines from a point corresponding to the host vehicle to the first LIDAR points and the half-straight lines.

18. The method of claim 13, wherein the distance condition comprises a condition that a LIDAR point is closest to a point corresponding to the host vehicle among the second LIDAR points.

19. The method according to claim 13, wherein the distance condition comprises a first condition or a second condition, the first condition being that the at least one third LIDAR point among the second LIDAR points corresponds to a minimum angle among angles between each of the half-straight lines from the point corresponding to the host vehicle to the second LIDAR point and the half-straight line, and the second condition being that the at least one third LIDAR point among the second LIDAR points is closest to the target fused trajectory.

20. The method of claim 13, wherein: The point corresponding to the host vehicle includes a center point of a line segment representing a front bumper module of the host vehicle; and The point corresponding to the target fusion trajectory includes a center point of a trajectory box representing the camera trajectory.

Images