Apparatus for controlling vehicle and method thereof

Abstract

The invention discloses an apparatus for controlling a vehicle and a method thereof. A vehicle control apparatus including a sensor and a processor is presented. The processor is configured to generate a tracking box from a virtual box acquired at an initial point in time, determine an associated virtual box at a subsequent point in time, and selectively merge the virtual boxes based on factors such as distance, road edge location, or virtual box type, thereby generating a signal indicative of the merged virtual box.

Description

[0001] Related Applications

[0002] This application claims the benefit of priority from Korean Patent Application No. 10-2023-0120569 filed in the Korean Intellectual Property Office on September 11, 2023, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a vehicle control device and a method thereof, and more particularly, to a technology for using a sensor such as light detection and ranging (LiDAR). Background Art

[0004] Various studies are being conducted to recognize external objects by using various sensors to assist the host vehicle in traveling.

[0005] Specifically, when the host vehicle is traveling in a driving assistance device activation mode or an autonomous driving mode, external objects may be recognized by using a sensor (eg, LiDAR).

[0006] In the merging process of the virtual box generated using LiDAR, the virtual box may be incorrectly merged. If the merged box obtained by incorrectly merging the virtual boxes is mapped to an external object, the type, state, or size of the external object may be incorrectly identified or determined. If the type, state, or size of the external object is incorrectly identified or determined, the driving route of the host vehicle may be significantly or incorrectly changed. Summary of the invention

[0007] According to the present disclosure, a device for controlling a vehicle may include a sensor and a processor, wherein the processor is configured to generate a tracking frame based on a virtual frame corresponding to an object at a first time point obtained by the sensor, by converting the virtual frame into a virtual frame at a second time point after the first time point, determine a virtual frame associated with the tracking frame in the multiple virtual frames at the second time point based on obtaining multiple virtual frames at the second time point formed by obtaining multiple points at the second time point by the sensor, and selectively merge all or part of the determined virtual frames based on at least one of the distance between the multiple virtual frames at the second time point, whether the multiple virtual frames at the second time point are outside the edge of a road, or the types of the multiple virtual frames at the second time point, and output a signal indicating the selectively merged virtual frame.

[0008] The processor is configured to determine a virtual frame associated with the tracking frame among the multiple virtual frames at the second time point based further on that the tracking frame overlaps each of the multiple virtual frames at the second time point at a specified ratio or greater, or that distances between multiple first points included in the tracking frame and multiple second points included in the multiple virtual frames at the second time point are within a correlation distance.

[0009] The processor is configured to determine that the object is not obscured, and determine the stationary state or the moving state of the object based on the type of the object being determined as a first type including pedestrians or a third type different from the second type including special vehicles, and the number of the multiple virtual frames at the second time point is less than a specified number.

[0010] The processor is configured to determine that the object is in the stationary state, determine that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length, determine a first straight line perpendicular to a line segment forming the first virtual frame from a first center point of a first virtual frame in the multiple virtual frames at the second time point based on that the object is in the stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, and selectively skip merging at least a portion of the determined virtual frames based on that a distance between a second center point of a second virtual frame in the multiple virtual frames at the second time point and the first straight line is greater than or equal to the second reference length.

[0011] The processor is configured to: determine that a road on which a vehicle including the device is traveling is a straight road; determine that an object is in a stationary state among multiple states including a stationary state and a moving state, and determine all or part of the determined virtual frame within a first specified distance in a direction perpendicular to the front of the vehicle.

[0012] The processor is configured to determine a first area including the road and the road edge, determine a second area different from the first area including the road and the road edge, and selectively skip merging all or part of the determined virtual frames detected in the second area among all or part of the determined virtual frames detected within the first specified distance.

[0013] The processor is configured to receive an entry signal indicating that a vehicle including the device enters an enclosed space, determine that the object is in a stationary state among a plurality of states including a stationary state and a moving state, determine that the determined virtual frame is located within a second specified distance in a direction perpendicular to the object, determine that a length of each of the determined virtual frames is greater than or equal to a reference length, and selectively skip merging all or part of the determined virtual frames based on determining that a number of points contained in the determined virtual frame is greater than or equal to a first reference number.

[0014] The processor is configured to determine that the object is in a stationary state among a plurality of states including a stationary state and a moving state, determine that a size of a tracking frame is greater than or equal to a reference size, determine that a number of points contained in the tracking frame is greater than or equal to a second reference number, and selectively skip merging all or a portion of the determined virtual frame based on the type of the determined virtual frame being determined to be a third type different from a first type associated with a pedestrian and different from a second type associated with a vehicle.

[0015] The processor is configured to determine a type of the determined virtual frame based on a density of points included in the determined virtual frame.

[0016] The processor is configured to determine that the determined type of the virtual frame is the third type based on the density of the points being greater than or equal to a reference density.

[0017] According to the present disclosure, a method for controlling a vehicle may include: generating a tracking frame by converting a virtual frame corresponding to an object at a first time point after the first time point through a device and based on obtaining a virtual frame corresponding to an object at a first time point through a sensor, determining a virtual frame associated with the tracking frame among the multiple virtual frames at the second time point based on obtaining multiple virtual frames at the second time point formed by obtaining multiple points at the second time point through the sensor, and selectively merging all or part of the determined virtual frames based on at least one of the distance between the multiple virtual frames at the second time point, whether the multiple virtual frames at the second time point are outside the edge of a road, or the types of the multiple virtual frames at the second time point, and outputting a signal indicating the selectively merged virtual frame.

[0018] The method, wherein determining the virtual frame associated with the tracking frame among the multiple virtual frames at the second time point is further based on the tracking frame overlapping each of the multiple virtual frames at the second time point at a specified ratio or greater, or the distances between multiple first points included in the tracking frame and multiple second points included in the multiple virtual frames at the second time point are within a related distance.

[0019] The method may further include determining that the object is not obscured, and determining a stationary state or a moving state of the object based on the type of the object being determined to be a first type including pedestrians or a third type different from the second type including special vehicles, and the number of the multiple virtual frames at the second time point is less than a specified number.

[0020] The method may further include determining that the object is in the stationary state, determining that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length, determining a first straight line perpendicular to a line segment forming the first virtual frame from a first center point of a first virtual frame in the multiple virtual frames at the second time point based on that the object is in the stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, and selectively skipping merging at least a portion of the determined virtual frames based on that a distance between a second center point of a second virtual frame in the multiple virtual frames at the second time point and the first straight line is greater than or equal to the second reference length.

[0021] The method may also include: determining that the road on which the vehicle including the device is traveling is a straight road; determining that the object is in a stationary state among multiple states including a stationary state and a moving state, and determining all or part of the determined virtual frame within a first specified distance in a direction perpendicular to the front of the vehicle.

[0022] The method may also include determining a first area including the road and the road edge, determining a second area different from the first area including the road and the road edge, and selectively skipping merging all or part of the determined virtual frames detected in the second area among all or part of the determined virtual frames detected within the first specified distance.

[0023] The method may also include receiving an entry signal indicating that a vehicle has entered an enclosed space, determining that an object is in a stationary state among a plurality of states including a stationary state and a moving state, determining that the determined virtual frame is located within a second specified distance in a direction perpendicular to the object, determining that a length of each virtual frame in the determined virtual frame is greater than or equal to a reference length, and selectively skipping merging all or part of the determined virtual frame based on determining that a number of points contained in the determined virtual frame is greater than or equal to a first reference number.

[0024] The method may further include determining that the object is in a stationary state among a plurality of states including a stationary state and a moving state, determining that a size of a tracking frame is greater than or equal to a reference size, determining that a number of points contained in the tracking frame is greater than or equal to a second reference number, and selectively skipping merging all or a portion of the determined virtual frame based on the type of the determined virtual frame being determined to be a third type different from a first type associated with a pedestrian and different from a second type associated with a vehicle.

[0025] The method may further include determining the type of the determined virtual frame based on a density of points included in the determined virtual frame. The method may further include determining that the type of the determined virtual frame is a third type based on the density of points being greater than or equal to a reference density. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings:

[0027] Figure 1 An embodiment of a block diagram of a vehicle control device according to an embodiment of the present disclosure is shown;

[0028] Figure 2 An embodiment of generating a merge frame by using a virtual frame in the embodiment of the present disclosure is shown;

[0029] Figure 3 An embodiment of identifying a virtual frame outside a road edge in an embodiment of the present disclosure is shown;

[0030] Figure 4 An embodiment of identifying the distance between virtual frames in an embodiment of the present disclosure is shown;

[0031] Figure 5 An embodiment of identifying a virtual frame outside a road edge in an embodiment of the present disclosure is shown;

[0032] Figure 6 An embodiment of a flow chart showing a vehicle control method according to another embodiment of the present disclosure; and

[0033] Figure 7 An embodiment of a computing system associated with a vehicle control device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0034] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. When adding reference numerals to components of the respective drawings, it should be noted that the same components have the same reference numerals even though they are shown in another drawing. In addition, when describing the embodiments of the present disclosure, if a well-known function or configuration may make the subject matter of the present disclosure unnecessarily obscure, the detailed description associated with the well-known function or configuration will be omitted.

[0035] When describing the elements of the embodiments of the present disclosure, the terms first, second, A, B, (a), (b) etc. can be used in this article. These terms are only used to distinguish one element from another element, but do not limit the corresponding elements, and are independent of the properties, order or priority of the corresponding elements. In addition, unless otherwise defined, all terms including technical terms and scientific terms used herein will be interpreted as the customary terms of the field to which this disclosure belongs. It should be understood that the terms used in this article should be interpreted as having the same meaning as their meaning in the context of the present disclosure and the related art, and should not be interpreted in an idealized or overly formal sense, unless clearly defined in this article.

[0036] In the following, reference will be made to Figures 1 to 7 Various embodiments of the present disclosure are described in detail.

[0037] Figure 1 An embodiment of a block diagram of a vehicle control device according to an embodiment of the present disclosure is shown.

[0038] Reference Figure 1 The vehicle control device 100 according to one embodiment of the present disclosure may be implemented inside or outside the vehicle, and some components included in the vehicle control device 100 may be implemented inside or outside the vehicle. In this case, the vehicle control device 100 may be integrated with the internal control unit of the vehicle, and may be implemented by a separate device so as to be connected to the control unit of the vehicle by means of a separate connection device. For example, the vehicle control device 100 may further include Figure 1 Components not shown.

[0039] Reference Figure 1 The vehicle control device 100 according to the embodiment may include a processor 110 and a sensor (eg, LiDAR 120). The processor 110 or the sensor 120 may be electrically connected to each other or operably connected to each other through electronic components including a communication bus.

[0040] Hereinafter, the fact that hardware is operably coupled may include the fact that a direct or indirect connection between hardware is established by wire or wirelessly so that the second hardware is controlled by the first one of the hardware. Although different blocks are shown, the embodiment is not limited thereto.

[0041] Figure 1Some of the hardware in the vehicle control device 100 may be included in a single integrated circuit including a system on a chip (SoC). The type or number of hardware included in the vehicle control device 100 is not limited to Figure 1 For example, the vehicle control device 100 may include only Figure 1 Some of the hardware is shown in FIG.

[0042] The vehicle control device 100 according to the embodiment may include hardware for processing data based on one or more instructions. The hardware for processing data may include a processor 110 .

[0043] For example, the hardware for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The processor 110 may have a structure of a single-core processor, or may have a structure of a multi-core processor including a dual-core, quad-core, hexa-core, or octa-core.

[0044] The LiDAR 120 included in the vehicle control apparatus 100 according to the embodiment may obtain a data set from identifying objects around the vehicle control apparatus 100. For example, the LiDAR 120 may identify at least one of the position of the surrounding object, the moving direction of the surrounding object, the speed of the surrounding object, or any combination thereof based on a pulse laser signal emitted from the LiDAR 120, reflected by the surrounding object, and returned.

[0045] The processor 110 included in the vehicle control apparatus 100 according to the embodiment may obtain a virtual frame at a first time point corresponding to a designated object at the first time point through the LiDAR 120. For example, the processor 110 may obtain the virtual frame at the first time point by using a plurality of points obtained at the first time point through the LiDAR 120.

[0046] The processor 110 may convert the virtual frame at the first time point into a virtual frame at a second time point after the first time point to generate a tracking frame.

[0047] For example, at least one of the first time point, the second time point, or any combination thereof may include a time point for obtaining a data set by LiDAR 120. For example, the second time point may be referred to as a current time point (e.g., time point "t"). For example, the second time point may include a time point at which a current frame is obtained. For example, the first time point may include a time point immediately before the current frame is obtained. For example, the first time point may include a time point "t-1". It indicates that the time point is at least one of the first time point, the second time point, or any combination thereof. However, at least one of the first time point, the second time point, or any combination thereof may mean a frame number. For example, if the second time point means the tth frame, the first time point may mean the (t-1)th frame.

[0048] In an embodiment, the processor 110 may obtain a plurality of points at the second time point through the LiDAR 120. For example, the processor 110 may obtain a virtual frame at the second time point formed by the plurality of points obtained at the second time point through the LiDAR 120. For example, the virtual frame at the second time point may be referred to as a "meta box". For example, the virtual frame at the second time point may include a frame formed based on points obtained as light emitted from the LiDAR 120 is reflected from an external object.

[0049] In an embodiment, the processor 110 may identify a virtual frame associated with the tracking frame in the virtual frame at the second time point based on obtaining the virtual frame at the second time point.

[0050] For example, the processor 110 may identify a ratio at which the tracking frame overlaps each virtual frame at the second time point. For example, the processor 110 may identify that the tracking frame overlaps each virtual frame in the virtual frame obtained at the second time point at a specified ratio (e.g., about 25%) or more.

[0051] For example, the processor 110 may identify the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point. For example, the processor 110 may determine whether the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point is within the relevant distance. For example, the relevant distance may be referred to as the "Mahalanobis distance".

[0052] In an embodiment, the processor 110 may identify a virtual frame associated with the tracking frame in the virtual frame at the second time point based on the fact that the tracking frame overlaps with each virtual frame in the virtual frame obtained at the second time point at a specified ratio or greater, or the distance between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point is identified as being within the relevant distance.

[0053] For example, the processor 110 may identify a virtual frame associated with the tracking frame by identifying that the tracking frame overlaps each virtual frame in the virtual frame obtained at the second time point at a specified ratio or greater, or that the distances between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point are within a relevant distance.

[0054] In an embodiment, the processor 110 may merge or may not merge all or part of the associated virtual frames based on at least one of the following: the distance between the virtual frames at the second time point, whether the virtual frame at the second time point is outside the edge of the road, the type of the virtual frame at the second time point, or any combination thereof. Alternatively or in addition, the processor 110 may selectively merge all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether the virtual frame at the second time point is outside the edge of the road, the type of the virtual frame at the second time point, or any combination thereof. Hereinafter, the operation of not merging virtual frames may include the operation of excluding all or part of the virtual frames from the merge candidates.

[0055] In an embodiment, the processor 110 may project a plurality of points obtained by a sensor (e.g., LiDAR) onto a plane formed by at least two axes of the x-axis, the y-axis, and the z-axis. The processor 110 may obtain (or generate) a virtual frame based on the plurality of points projected onto the plane formed by at least two axes of the x-axis, the y-axis, and the z-axis.

[0056] In an embodiment, the processor 110 may identify a virtual frame on a plane formed by at least two axes of the x-axis, the y-axis, and the z-axis. Hereinafter, at least one of the virtual frame, the tracking frame, the merged frame, or any combination thereof may include a frame generated based on a plurality of points projected onto the plane formed by the x-axis, the y-axis, and the z-axis.

[0057] In an embodiment, based on the distance between the virtual frames at the second time point, the processor 110 may or may not merge all or part of the associated virtual frames.

[0058] For example, the processor 110 may determine whether a specified object corresponding to the tracking frame is blocked. The processor 110 may identify the type of the specified object. For example, the processor 110 may identify the type of the specified object based on the tracking frame associated with the specified object. For example, the tracking frame may include information at a previous time point. For example, the information at the previous time point may include at least one of a forward direction of the tracking frame, a type of an object corresponding to the tracking frame, a speed of the object corresponding to the tracking frame, a position of the object corresponding to the tracking frame, or any combination thereof.

[0059] In an embodiment, the processor 110 may identify that the specified object is not occluded.

[0060] In an embodiment, the processor 110 may identify that the type of the specified object is a first specified type including a pedestrian, or may identify that the type of the specified object is different from a second specified type including a special vehicle. For example, the special vehicle may include at least one of a 1-ton truck, a truck connected to a container frame, a heavy truck, or any combination thereof.

[0061] In an embodiment, the processor may determine whether the number of virtual frames at the second time point associated with the tracking frame is less than a specified number (eg, 3).

[0062] In an embodiment, the processor 110 can identify the stationary state or the moving state of the specified object based on the following facts: the specified object is not obscured, and the type of the specified object is identified as a first specified type including pedestrians, or the type of the specified object is identified as different from a second specified type including special vehicles, and the number of virtual frames at a second time point associated with the tracking frame is less than a specified number.

[0063] In an embodiment, the processor 110 may recognize that the designated object is in a stationary state.

[0064] In an embodiment, the processor 110 may identify at least one of a width of the tracking frame, a length of the tracking frame, or any combination thereof. The processor 110 may identify at least one of the width, the length, or any combination thereof of the tracking frame based on coordinates of vertices included in the tracking frame on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis.

[0065] For example, the width of the tracking frame may include a length corresponding to the y axis on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis. For example, the length of the tracking frame may include a length corresponding to the x axis on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis.

[0066] In an embodiment, the processor 110 may determine whether the designated object is in a stationary state and whether at least one of a width or a length of the tracking frame is greater than or equal to a first reference length (eg, approximately 30 m).

[0067] In an embodiment, the processor 110 may determine whether the designated object is in a stationary state and whether at least one of the width or the length of the tracking frame is greater than or equal to the first reference length. The processor 110 may identify a first straight line perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame in the virtual frame at the second time point based on the fact that the designated object is in a stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length.

[0068] The processor 110 may determine whether a distance between a second center point of a second virtual frame in a virtual frame at a second time point and the first straight line is greater than or equal to a second reference length (eg, approximately 20 m).

[0069] The processor 110 may not merge the virtual frame associated with the tracking frame based on the fact that the distance between the second center point of the second virtual frame in the virtual frame at the second time point and the first straight line is greater than or equal to the second reference length.

[0070] In an embodiment, the processor 110 may determine whether the virtual frame at the second time point is outside the edge of the road. For example, the edge of the road may include at least one of a guardrail, a tunnel wall, a marker for identifying an area different from an area where the vehicle can drive, or any combination thereof.

[0071] In an embodiment, the processor 110 may identify a road on which the host vehicle including the vehicle control apparatus 100 is traveling. For example, the processor 110 may identify that the road on which the host vehicle is traveling is a straight road.

[0072] The processor 110 may recognize that the road on which the host vehicle is traveling is a straight road and that the designated object is in a stationary state among the stationary state and the moving state.

[0073] The processor 110 may identify all or part of the virtual frame associated with the tracking frame within a first specified distance (e.g., about 50 m) in a direction perpendicular to the front of the host vehicle (e.g., the positive x-axis). Because the reliability of the virtual frame identified in the area exceeding the first specified distance is lower than the reliability of the virtual frame identified within the first specified distance, the processor 110 may identify all or part of the virtual frame associated with the tracking frame within the first specified distance.

[0074] The processor 110 may identify a road on which the host vehicle is traveling and an edge of the road. For example, the processor 110 may identify a first area including a road on which the host vehicle is traveling and an edge of the road.

[0075] The processor 110 may identify a second area different from the first area including the road and the road edge. For example, the second area may include an area outside the road edge.

[0076] The processor 110 may not merge all or part of the associated virtual frames identified in the second area among all or part of the associated virtual frames identified within the first specified distance. For example, the processor 110 may not merge the virtual frames identified outside the edge of the road. For example, the processor 110 may exclude the virtual frames identified outside the edge of the road from the merge candidates.

[0077] In an embodiment, the processor 110 may receive an entry signal indicating that the host vehicle including the vehicle control device 100 enters a tunnel. For example, the processor 110 may receive the entry signal indicating that the host vehicle is entering a tunnel from at least one of a hardware component, a software component, or any combination thereof included in the host vehicle.

[0078] The processor 110 may identify the designated object as being in a stationary state among the stationary state and the moving state based on receiving an entry signal indicating that the host vehicle enters the tunnel. For example, the fact that the designated object is in a stationary state means that the designated object is a stationary object.

[0079] The processor 110 may determine whether the virtual frame associated with the tracking frame is located within a second specified distance (eg, 2 m) in a direction perpendicular to the specified object. For example, the processor 110 may determine whether the associated virtual frame is located within the second specified distance from the tracking frame.

[0080] The processor 110 may identify the number of points included in the associated virtual frame. For example, the processor 110 may determine whether the number of points included in the associated virtual frame is greater than or equal to a first reference number (eg, approximately 1000).

[0081] Based on recognizing that the number of points included in the associated virtual frame is greater than the first reference number, the processor 110 may not merge all or part of the associated virtual frame.

[0082] According to an embodiment, the processor 110 may merge all or part of the associated virtual frames based on the fact that the number of points included in the associated virtual frames is less than the first reference number. For example, the processor 110 may generate a merged frame based on merging all or part of the associated virtual frames. For example, the merged frame may include a frame corresponding to the external object at the current time point (e.g., the second time point).

[0083] The processor 110 may output the merged box based on obtaining the merged box. Hereinafter, the operation of outputting the merged box or outputting the virtual box may include the operation of mapping the merged box to an external object or mapping the virtual box to an external object.

[0084] In an embodiment, the processor 110 may identify that the designated object is in a stationary state among a stationary state and a moving state. The processor 110 may identify the size of the tracking frame based on the designated object being in a stationary state.

[0085] For example, the processor 110 may identify at least one of a width of the tracking frame, a length of the tracking frame, or any combination thereof. The processor 110 may identify the size of the tracking frame based on the width and the length of the tracking frame.

[0086] The processor 110 may determine whether the size of the tracking frame is greater than or equal to a reference size (eg, approximately 20 m2 ). For example, the processor 110 may identify the number of points included in the tracking frame based on the size of the tracking frame being greater than or equal to the reference size.

[0087] For example, the processor 110 may determine whether the number of points included in the associated virtual frame is greater than or equal to a first reference number (eg, approximately 1000).

[0088] In an embodiment, the processor 110 may identify the type of the associated virtual frame. For example, the processor 110 may determine whether the type of the associated virtual frame is a third specified type including a bush object based on the density of points included in the associated virtual frame. For example, the bush object may include an object having an irregular shape including a bush.

[0089] The processor 110 may not merge all or part of the associated virtual frames based on the type of the associated virtual frames being identified as the third designated type.

[0090] In an embodiment, the processor 110 may determine whether the density of the points is greater than or equal to a reference density. The processor 110 may identify that the type of the associated virtual frame is the third specified type based on the density of the points being greater than or equal to the reference density.

[0091] In an embodiment, based on the fact that the specified object is in a stationary state, the size of the tracking frame is greater than or equal to the reference size, the number of points included in the tracking frame is greater than or equal to the second reference number, and the type of the associated virtual frame is a third specified type, the processor 110 may not merge all or part of the associated virtual frame.

[0092] The processor 110 may output the associated virtual frame based on not merging all or part of the associated virtual frame. For example, the processor 110 may map each virtual frame in the associated virtual frame to an external object. For example, the processor 110 may output the associated virtual frame based on mapping each virtual frame in the associated virtual frame to an external object.

[0093] Figure 2 An embodiment of generating a merged frame by using a virtual frame according to an embodiment of the present disclosure is shown.

[0094] Figure 2 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 2 The operations can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0095] refer to Figure 2In the first embodiment 201, the processor included in the vehicle control device according to the embodiment can obtain a virtual frame 211 at a first time point corresponding to a specified object at the first time point through a sensor (e.g., LiDAR). For example, the processor can obtain the virtual frame 211 by using a plurality of points obtained at the time point "t-1".

[0096] Reference Figure 2 In the second embodiment 203, in an embodiment, the processor may convert the virtual frame 211 at the first time point into a virtual frame at a second time point after the first time point to generate a tracking frame 213. For example, the second time point may be referred to as a current frame.

[0097] refer to Figure 2 In the third embodiment 205, in an embodiment, the processor can obtain multiple virtual frames (e.g., virtual frames 221, 223, 225 and 227) at a second time point through LiDAR, and these virtual frames are formed by multiple points obtained at the second time point.

[0098] The processor may identify a virtual frame associated with the tracking frame 213 among the virtual frames 221 , 223 , 225 , and 227 at the second time point based on obtaining the virtual frames 221 , 223 , 225 , and 227 at the second time point formed by the plurality of points obtained at the second time point.

[0099] For example, the processor may determine whether the tracking box 213 is associated with the virtual boxes 221, 223, 225, and 227 at the second time point based on at least one of the following: a tracking point, an output point, a center point, a ratio of overlap of the tracking box 213 with multiple virtual boxes (e.g., virtual boxes 221, 223, 225, and 227) at the second time point, or any combination thereof.

[0100] For example, the processor may determine whether the tracking box 213 is associated with multiple virtual boxes (e.g., virtual boxes 221, 223, 225, and 227) at the second time point based on a correlation or a threshold distance between the tracking box 213 and the multiple virtual boxes (e.g., virtual boxes 221, 223, 225, and 227) at the second time point.

[0101] For example, the processor may determine whether the tracking frame 213 is associated with a plurality of virtual frames (e.g., virtual frames 221, 223, 225, and 227) at the second time point based on the correlation distance obtained according to the Mahalanobis distance. For example, if the Mahalanobis distance is less than or equal to the reference distance (e.g., about 0.5 m), the processor may determine that the tracking frame 213 is associated with a plurality of virtual frames (e.g., virtual frames 221, 223, 225, and 227) at the second time point.

[0102] For example, the processor may determine that the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on the fact that the tracking frame 213 overlaps the virtual frames 221, 223, 225, and 227 at the second time point by a reference ratio (e.g., approximately 25%) or more.

[0103] refer to Figure 2 In the fourth embodiment 207 , in an embodiment, the processor may generate a merged frame 230 based on the associated virtual frames 221 , 223 , 225 , and 227 .

[0104] For example, the processor may merge the virtual boxes 221, 223, 225, and 227 associated with the tracking box 213. For example, the processor may merge all or part of the virtual boxes 221, 223, 225, and 227 associated with the tracking box 213.

[0105] The processor may generate a merged box 230 based on merging all or part of the associated virtual boxes 221 , 223 , 225 , and 227 .

[0106] Hereinafter, an embodiment of a vehicle control apparatus for generating the merging box 230 will be described later.

[0107] Figure 3 An embodiment of identifying a virtual frame outside a road edge in an embodiment of the present disclosure is shown.

[0108] Figure 3 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 3 The operations can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0109] refer to Figure 3 , the vehicle control apparatus according to an embodiment may be included in the host vehicle 300. The processor included in the vehicle control apparatus according to an embodiment may identify a road on which the host vehicle is traveling. According to an embodiment, the processor may identify a road on which the host vehicle can travel. The processor may identify a first area 321 including a road and a road edge 311 (e.g., a boundary line, a guardrail).

[0110] The processor may identify a second area 323 that is different from the first area 321. For example, the second area 323 may be an area on which the host vehicle cannot travel, or may include a road opposite to a road on which the host vehicle can travel. For example, assuming that the road on which the host vehicle can travel is a lane heading to a specific destination (e.g., Seoul, New York, etc.), the road opposite to the road on which the host vehicle can travel may be a lane heading away from the specific destination (e.g., Seoul, New York, etc.).

[0111] In an embodiment, the processor may identify a virtual frame 331 outside the road edge 311. For example, when the host vehicle 300 travels in a straight line and moves in the longitudinal direction, the processor may identify the virtual frame 331 outside the road edge 311. For example, the longitudinal direction may include an x-axis direction of a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis (e.g., a travel direction of the host vehicle 300, a longitudinal direction).

[0112] In an embodiment, the processor may identify an associated virtual frame 331 within a first specified distance (eg, 50m or 100m) in a direction perpendicular to the front of the host vehicle 300. For example, the direction perpendicular to the front of the host vehicle 300 may include a y-axis direction.

[0113] The processor may not merge all or part of the virtual frames 331 identified outside the road edge 311. For example, the processor may output each of the virtual frames 331 identified outside the road edge 311 as a separate object based on not merging all or part of the virtual frames 331 identified outside the road edge 311.

[0114] As described above, the processor included in the vehicle control apparatus according to the embodiment may not merge all or part of the virtual frame 331 identified outside the road edge 311. The processor may not merge all or part of the virtual frame 331 identified outside the road edge 311, thereby reducing calculation time.

[0115] In an embodiment, the processor 110 may receive an entry signal indicating that the host vehicle 300 has entered an enclosed space (eg, an underpass, such as a tunnel).

[0116] For example, processor 110 may identify a tunnel wall based on receiving a signal indicating that host vehicle 300 is entering a tunnel. Figure 3 The virtual frame 331 outside the road edge 311 identified in FIG. 3 may include a tunnel wall.

[0117] The processor may not merge the virtual boxes 331 corresponding to the tunnel walls, thereby reducing calculation time.

[0118] Figure 4 An embodiment of identifying the distance between virtual frames in an embodiment of the present disclosure is shown.

[0119] Figure 4 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 4 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0120] Figure 4 An embodiment is shown in which all or part of a virtual box associated with a tracking box is merged based on a distance between virtual boxes at a second point in time.

[0121] Reference Figure 4 , the processor included in the vehicle control device according to the embodiment may identify one or more virtual frames (e.g., virtual frames 411 and 413) associated with the tracking frame 410 in the virtual frame at the second time point. For example, if the designated object corresponding to the tracking frame 410 is in a stationary state, the processor may perform the following operations.

[0122] In an embodiment, the processor 110 may determine whether the designated object is in a stationary state or in a moving state based on at least one of the following: whether the designated object corresponding to the tracking frame 410 is obscured (e.g., blocked, hidden, or obscured and not visible), the type of the designated object, the number of virtual frames at the second time point, or any combination thereof.

[0123] The processor can identify the stationary state or moving state of the specified object based on the following facts: the specified object is not obscured or blocked and the type of the specified object is identified as a first specified type including pedestrians, or the type of the specified object is identified as a third type different from a second specified type including special vehicles (e.g., emergency vehicles, fire trucks, ambulances, police cars, etc.) and the number of virtual frames at the second time point is less than the specified number.

[0124] The processor may recognize that the designated object is stationary, and may recognize that at least one of the width 419 of the tracking frame 410 , the length of the tracking frame 410 , or any combination thereof is greater than or equal to a first reference length (eg, approximately 30 m).

[0125] For example, if the designated object corresponding to the tracking frame 410 is in a stationary state, and the width 419 of the tracking frame 410 is greater than or equal to a first reference length (e.g., 30 m), and if the distance between the associated virtual frames is greater than or equal to a second reference length (e.g., 20 m), the processor may not merge the associated virtual frames.

[0126] For example, the processor may identify a first straight line 415 that is perpendicular to a line segment forming the first virtual frame 411 from a center point of the first virtual frame 411 based on the fact that the first virtual frame 411 is associated with the second virtual frame 413 .

[0127] The processor may identify a distance 417 between the first straight line 415 and the center point of the second virtual frame 413. The processor may not merge or skip merging the first virtual frame 411 and the second virtual frame 413 based on identifying or determining that the distance between the first straight line 415 and the center point of the second virtual frame 413 is greater than or equal to a reference distance (e.g., about 20 m).

[0128] The operation of the processor not merging or skipping merging the first virtual frame 411 and the second virtual frame 413 may include an operation in which the processor excludes at least one of the first virtual frame 411 , the second virtual frame 413 , or any combination thereof from a merging candidate.

[0129] Figure 5 An embodiment of identifying a virtual frame outside a road edge (eg, a boundary line, a guardrail, a fence, or a road wall, etc.) in an embodiment of the present disclosure is shown.

[0130] Figure 5 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 5 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0131] refer to Figure 5 , the vehicle control device according to the embodiment may be included in the host vehicle 500. The processor included in the vehicle control device may identify the road on which the host vehicle 500 is traveling and the road edge 511 (eg, a boundary line, guardrail, fence, or road wall, etc.).

[0132] The processor may identify a first area 521 including a road and a road edge 511. The processor may identify a second area 523 that is different from the first area 521. For example, the second area 523 may include areas outside the road edge 511 (e.g., unpaved roads, sidewalks, bike paths, beaches, mountain trails, off-road trails, restricted areas, construction sites, bodies of water, stairs, elevated walkways, bridges, pedestrian streets, parks, etc.). For example, the second area 523 may include an area where the host vehicle 500 cannot travel.

[0133] In an embodiment, the processor may identify a virtual frame 531 outside the road edge 511. For example, the processor may identify a virtual frame 531 in the second area 523.

[0134] If the specified object corresponding to the tracking frame is stationary and the size of the tracking frame is greater than or equal to the reference size (e.g., approximately 20 m 2 ), and the number of points included in the tracking box is greater than or equal to a reference number (e.g., approximately 1000), the processor may perform the following operations.

[0135] In an embodiment, the processor may identify the type of the virtual frame 531 identified as being outside the road edge 511. For example, the processor may identify the type of the virtual frame 531 based on the density of points included in the virtual frame 531 identified as being outside the road edge 511.

[0136] For example, the processor may determine whether the density of the points included in the virtual frame 531 is greater than or equal to the reference density. The processor may identify that the type of the virtual frame 531 is the third specified type based on the fact that the density of the points included in the virtual frame 531 is greater than or equal to the reference density. For example, the processor may identify that the type of the virtual frame 531 is the third specified type including a shrub object based on the fact that the density of the points included in the virtual frame 531 is greater than or equal to the reference density. For example, the shrub object may include an object having an irregular shape (e.g., a shrub, a woody stem, a branch covered with needles, etc.).

[0137] In an embodiment, the processor may not merge virtual frames 531 identified as the third designated type. For example, the processor may output each virtual frame 531 based on not merging or selectively skipping merging virtual frames 531 identified as the third designated type.

[0138] Figure 6 An embodiment of a flowchart of a vehicle control method according to another embodiment of the present disclosure is shown.

[0139] In the following, reference will be made to Figure 6 A vehicle control method according to another embodiment of the present disclosure is described in detail. Figure 6 is a flowchart for describing a vehicle control method according to another embodiment of the present disclosure.

[0140] In the following, it is assumed Figure 1 The vehicle control device 100 executes Figure 6 Additionally or alternatively, Figure 6 In the description, it can be understood that the operations described as being performed by the device are controlled by the processor 110 of the vehicle control device 100.

[0141] Figure 6 At least one of the operations may be performed by Figure 1 The vehicle control device 100 executes. Figure 6 Each operation in can be performed sequentially, but does not necessarily have to be performed sequentially. For example, the order of the operations can be changed, and at least two operations can be performed in parallel.

[0142] refer to Figure 6In operation S601, the vehicle control method according to the embodiment may include the following operations: based on a virtual frame at a first time point corresponding to a designated object at the first time point obtained through LiDAR, a tracking frame is generated by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point.

[0143] In operation S603, the vehicle control method according to the embodiment may include the following operation: based on obtaining a virtual frame at the second time point formed by multiple points obtained by a sensor (e.g., LiDAR) at the second time point, identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point.

[0144] For example, the vehicle control method may include the following operations: based on the fact that the tracking frame overlaps with each virtual frame in the virtual frame at the second time point by a specified ratio or greater, or the fact that the distances between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point are identified as being within the relevant distance, identifying a virtual frame in the virtual frame at the second time point that is associated with the tracking frame.

[0145] In operation S605, the vehicle control method according to an embodiment may include an operation of selectively merging all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether the virtual frames at the second time point are outside the edge of the road, the type of the virtual frames at the second time point, or any combination thereof.

[0146] For example, the vehicle control method may include an operation of generating a merged frame based on merging all or part of a virtual frame associated with a tracking frame. The vehicle control method may include an operation of outputting the generated merged frame.

[0147] For example, the vehicle control method may include not merging or selectively skipping merging all or part of the virtual frames associated with the tracking frame. The vehicle control method may output each virtual frame based on merging all or part of the virtual frames associated with the tracking frame.

[0148] Figure 7 An embodiment of a computing system associated with a vehicle control device according to an embodiment of the present disclosure is shown.

[0149] refer to Figure 7 , the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 connected to each other via a bus 1200 .

[0150] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the storage device 1600. Each of the memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a read-only memory (ROM) and a random access memory (RAM).

[0151] Therefore, the operation of the method or algorithm described in conjunction with the embodiments disclosed in the specification can be directly implemented using a hardware module, a software module, or a combination of a hardware module and a software module, which is executed by the processor 1100. The software module may reside on a storage medium (i.e., the memory 1300 and / or the storage device 1600), such as a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable and programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk drive, a removable disk, or a compact disk ROM (CD-ROM).

[0152] The storage medium may be coupled to the processor 1100. The processor 1100 may read information from the storage medium and may write information to the storage medium. Alternatively or additionally, the storage medium may be integrated with the processor 1100. The processor and the storage medium may be implemented using an application specific integrated circuit (ASIC). The ASIC may be provided in a user terminal. Alternatively or additionally, the processor and the storage medium may be implemented using separate components in a user terminal.

[0153] The present disclosure has been made to solve the above-mentioned problems occurring in the prior art while keeping the advantages achieved by the prior art intact.

[0154] An embodiment of the present disclosure provides a vehicle control apparatus and method thereof, which merge virtual frames generated by using LiDAR or unmerge merged frames.

[0155] One embodiment of the present disclosure provides a vehicle control apparatus and method thereof, which reduces a load of a processor by determining whether to merge virtual frames before merging virtual frames.

[0156] One embodiment of the present disclosure provides a vehicle control device and a method thereof, which reduces the load of a processor and shortens calculation time by not merging a virtual frame identified outside a road edge.

[0157] The technical problems to be solved by the present disclosure are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0158] According to an embodiment of the present disclosure, a vehicle control device may include light detection and ranging (LiDAR) and a processor. The processor may generate a tracking frame by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point based on a virtual frame at the first time point corresponding to a specified object at the first time point obtained by LiDAR, may identify a virtual frame associated with the tracking frame in the virtual frame at the second time point based on a virtual frame at the second time point formed by a plurality of points obtained at the second time point obtained by LiDAR, and may selectively merge all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether the virtual frames at the second time point are outside the edge of the road, and the type of the virtual frames at the second time point, or a combination of the distance between the virtual frames at the second time point, whether the virtual frames at the second time point are outside the edge of the road, and the type of the virtual frames at the second time point.

[0159] In an embodiment, the processor may identify a virtual box associated with the tracking box in the virtual box at the second time point based on the fact that the tracking box overlaps with each virtual box in the virtual box at the second time point by a specified ratio or greater, or the distance between multiple first points included in the tracking box and multiple second points included in the virtual box at the second time point is identified as being within the relevant distance.

[0160] In an embodiment, the processor can identify that the specified object is not obscured, and can identify the stationary state or the moving state of the specified object based on the type of the specified object being identified as a first specified type including pedestrians, or the type of the specified object being identified as a type different from a second specified type including special vehicles, and the number of virtual frames at the second time point is less than a specified number.

[0161] In one embodiment, the processor may identify that the specified object is in a stationary state, may identify that at least one of the width or the length of the tracking frame is greater than or equal to a first reference length, and based on the fact that the specified object is in a stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, may identify a first straight line in the virtual frame at the second time point that is perpendicular to a line segment that forms the first virtual frame from a first center point of the first virtual frame, and based on the fact that a distance between the first straight line and a second center point of a second virtual frame in the virtual frame at the second time point is greater than or equal to the second reference length, may not merge the associated virtual frames.

[0162] In an embodiment, the processor can identify that a road on which a main vehicle including a vehicle control device is traveling is a straight road, can identify that a specified object is in a stationary state among a stationary state and a moving state, and can identify all or part of an associated virtual frame within a first specified distance in a direction perpendicular to the front of the main vehicle.

[0163] In an embodiment, the processor may identify a first area including a road and a road edge, may identify a second area different from the first area including the road and the road edge, and may not merge all or part of the associated virtual boxes identified in the second area with all or part of the associated virtual boxes identified within a first specified distance.

[0164] In an embodiment, the processor may receive an entry signal indicating that a main vehicle including a vehicle control device has entered a tunnel, may identify that a designated object is in a stationary state among a stationary state and a moving state, may identify that an associated virtual frame is located within a second designated distance in a direction perpendicular to the designated object, may identify that a length of each of the associated virtual frames is greater than or equal to a reference length, and based on identifying that a number of points included in the associated virtual frame is greater than or equal to a first reference number, all or part of the associated virtual frames may not be merged.

[0165] In an embodiment, the processor may identify that the specified object is in a stationary state among a stationary state and a moving state, may identify that a size of the tracking frame is greater than or equal to a reference size, may identify that the number of points included in the tracking frame is greater than or equal to a second reference number, and may not merge all or part of the associated virtual frame based on the type of the associated virtual frame being identified as a third specified type.

[0166] In an embodiment, the processor may identify the type of the associated virtual frame based on the density of points included in the associated virtual frame.

[0167] In an embodiment, the processor may identify that the type of the associated virtual frame is the third specified type based on the density of the points being greater than or equal to a reference density.

[0168] According to an embodiment of the present disclosure, a vehicle control method may include: based on a virtual frame at a first time point corresponding to a designated object at the first time point obtained through LiDAR, converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point to generate a tracking frame; based on a virtual frame at a second time point formed by a plurality of points obtained at the second time point obtained through the LiDAR, identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point, and selectively merging all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether the virtual frame at the second time point is outside the edge of the road, and the type of the virtual frame at the second time point, or based on a combination of the distance between the virtual frames at the second time point, whether the virtual frame at the second time point is outside the edge of the road, and the type of the virtual frame at the second time point.

[0169] According to an embodiment, the vehicle control method may further include: based on the fact that the tracking frame overlaps with each virtual frame in the virtual frame at the second time point at a specified ratio or more, or the fact that the distances between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point are identified as being within the relevant distance, identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point.

[0170] According to an embodiment, the vehicle control method may further include: identifying that a specified object is not obscured, and based on the type of the specified object being identified as a first specified type including pedestrians, or the type of the specified object being identified as a type different from a second specified type including special vehicles, and the number of virtual frames at the second time point is less than a specified number, identifying the stationary state or the moving state of the specified object.

[0171] According to an embodiment, the vehicle control method may further include identifying that the specified object is in a stationary state, identifying that at least one of the width or the length of the tracking frame is greater than or equal to a first reference length, identifying a first straight line in the virtual frame at the second time point that is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame, and not merging associated virtual frames based on the fact that a distance between the first straight line and a second center point of a second virtual frame in the virtual frame at the second time point is greater than or equal to the second reference length.

[0172] According to one embodiment, the vehicle control method may also include: identifying that a road on which a main vehicle including a vehicle control device is traveling is a straight road; identifying that a specified object is in a stationary state among a stationary state and a moving state; and identifying all or part of an associated virtual frame within a first specified distance in a direction perpendicular to the front of the main vehicle.

[0173] According to one embodiment, the vehicle control method may also include: identifying a first area including a road and a road edge; determining a second area different from the first area including the road and the road edge; and not merging all or part of the associated virtual boxes identified in the second area among all or part of the associated virtual boxes identified within a first specified distance.

[0174] According to an embodiment, the vehicle control method may further include: receiving an entry signal indicating that a main vehicle including a vehicle control device enters a tunnel; identifying that a specified object is in a stationary state among a stationary state and a moving state; identifying that an associated virtual frame is located within a second specified distance in a direction perpendicular to the specified object, identifying that a length of each in the associated virtual frame is greater than or equal to a reference length, and based on identifying that the number of points included in the associated virtual frame is greater than or equal to a first reference number, not merging all or part of the associated virtual frame.

[0175] According to an embodiment, the vehicle control method may further include: identifying that the specified object is in a stationary state among a stationary state and a moving state; identifying that the size of the tracking frame is greater than or equal to a reference size; identifying that the number of points included in the tracking frame is greater than or equal to a second reference number; and based on the type of the associated virtual frame being identified as a third specified type, not merging all or part of the associated virtual frame.

[0176] According to an embodiment, the vehicle control method may further include identifying a type of the associated virtual frame based on a density of points included in the associated virtual frame.

[0177] According to an embodiment, the vehicle control method may further include: based on the density of the points being greater than or equal to a reference density, identifying that the type of the associated virtual frame is a third designated type.

[0178] The above description is merely an embodiment of the technical concept of the present disclosure, and various modifications and changes may be made by those skilled in the art without departing from the essential features of the present disclosure.

[0179] Therefore, the embodiments of the present disclosure are not intended to limit but to explain the technical concept of the present disclosure, and the scope and spirit of the present disclosure are not limited by the above embodiments. The protection scope of the present disclosure should be interpreted by the attached claims, and all equivalents thereof should be interpreted as included in the scope of the present disclosure.

[0180] According to an embodiment of the present disclosure, the vehicle control apparatus may merge virtual frames generated by using LiDAR or may unmerge merged frames.

[0181] Furthermore, according to an embodiment of the present disclosure, the vehicle control apparatus may reduce the load of a processor by determining whether to merge virtual frames before merging the virtual frames.

[0182] Furthermore, according to an embodiment of the present disclosure, the vehicle control apparatus can reduce the load on the processor and can shorten the calculation time by not merging the recognized virtual frame outside the edge of the road.

[0183] Furthermore, various effects understood directly or indirectly through the specification can be provided.

[0184] Although the present disclosure has been described above with reference to the embodiments and the accompanying drawings, the present disclosure is not limited thereto, but various modifications and changes may be made by those skilled in the art without departing from the spirit and scope of the present disclosure as claimed in the appended claims.

Claims

1. A device for controlling a vehicle, the device comprising: sensor; as well as processor, Wherein, the processor is configured to: generating a tracking frame by converting a virtual frame corresponding to the object at a first time point obtained by the sensor into a virtual frame at a second time point after the first time point; determining a virtual frame associated with the tracking frame among the plurality of virtual frames at the second time point based on a plurality of virtual frames at the second time point formed by a plurality of points obtained by the sensor at the second time point; and Selectively merging all or part of the determined virtual frames based on at least one of the following: The distance between the plurality of virtual frames at the second time point, whether the plurality of virtual frames at the second time point are outside the edge of the road, or types of the plurality of virtual frames at the second time point; and A signal indicating the selectively merged virtual frame is output.

2. The device according to claim 1, wherein: The processor is configured to: Determining a virtual frame associated with the tracking frame among the multiple virtual frames at a second time point is further based on: The tracking frame overlaps each of the plurality of virtual frames at the second time point at a specified ratio or more, or Distances between a plurality of first points included in the tracking frame and a plurality of second points included in the plurality of virtual frames at the second time point are within a correlation distance.

3. The device according to claim 2, wherein: The processor is configured to: determining that the object is not occluded; and The stationary state or moving state of the object is determined based on: The type of the object is determined to be: The first type includes pedestrians, or a third type different from the second type comprising special vehicles; and The number of the plurality of virtual frames at the second time point is less than a specified number.

4. The apparatus of claim 3, wherein the processor is configured to: determining that the object is in the stationary state; Determining that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length; determining, based on the object being in the stationary state and at least one of a width or a length of the tracking frame being greater than or equal to the first reference length, a first straight line from a first center point of a first virtual frame to a line segment forming the first virtual frame among the multiple virtual frames at the second time point; as well as Based on the distance between the second center point of the second virtual frame among the multiple virtual frames at the second time point and the first straight line being greater than or equal to a second reference length, at least a portion of the virtual frames determined by merging are selectively skipped.

5. The device according to claim 1, wherein: The processor is configured to: determining that a road on which the vehicle including the device is traveling is a straight road; determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; as well as All or part of the determined virtual frame is determined within a first specified distance in a direction perpendicular to the front of the vehicle.

6. The device according to claim 5, wherein: The processor is configured to: determining a first area including the road and the edge of the road; determining a second area different from the first area including the road and the road edge; as well as Selectively skipping merging all or part of the determined virtual frame detected in the second area from all or part of the determined virtual frame detected within the first specified distance.

7. The device according to claim 1, wherein: The processor is configured to: receiving an entry signal indicating that a vehicle including the device has entered the enclosed space; determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; determining that the determined virtual frame is within a second specified distance in a direction perpendicular to the object; Determining that the length of each determined virtual frame is greater than or equal to a reference length; as well as Based on determining that the number of points contained in the determined virtual frame is greater than or equal to the first reference number, selectively skipping merging all or part of the determined virtual frame.

8. The device according to claim 1, wherein: The processor is configured to: determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; Determine that the size of the tracking frame is greater than or equal to a reference size; determining that the number of points included in the tracking frame is greater than or equal to a second reference number; as well as Based on the determined type of the virtual frame being determined as a third type, selectively skipping merging all or part of the determined virtual frames, wherein the third type is different from a first type associated with a pedestrian, and the third type is different from a second type associated with a vehicle.

9. The device according to claim 8, wherein: The processor is configured to: The type of the determined virtual frame is determined based on the density of points included in the determined virtual frame.

10. The device according to claim 9, wherein: The processor is configured to: Based on the density of the points being greater than or equal to a reference density, the type of the determined virtual frame is determined to be the third type.

11. A method for controlling a vehicle, the method comprising: generating, by the device, a tracking frame based on a virtual frame corresponding to the object at a first time point obtained by a sensor, by converting the virtual frame into a virtual frame at a second time point after the first time point; determining a virtual frame associated with the tracking frame among the multiple virtual frames at the second time point based on obtaining multiple virtual frames at the second time point formed by multiple points obtained by the sensor at the second time point; as well as Selectively merging all or part of the determined virtual frames based on at least one of the following: The distance between the plurality of virtual frames at the second time point, whether the plurality of virtual frames at the second time point are outside the edge of the road, or types of the plurality of virtual frames at the second time point; as well as A signal indicating the selectively merged virtual frame is output.

12. The method according to claim 11, wherein: Determining a virtual frame associated with the tracking frame among the multiple virtual frames at a second time point is further based on: The tracking frame overlaps each of the plurality of virtual frames at the second time point at a specified ratio or more, or Distances between a plurality of first points included in the tracking frame and a plurality of second points included in the plurality of virtual frames at the second time point are within a correlation distance.

13. The method according to claim 12, further comprising: Determining that the object is not occluded; as well as The stationary state or moving state of the object is determined based on: The type of the object is determined to be: The first type includes pedestrians, or a third type different from the second type comprising special vehicles; and The number of the plurality of virtual frames at the second time point is less than a specified number.

14. The method according to claim 13, further comprising: determining that the object is in the stationary state; Determining that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length; determining, based on the object being in the stationary state and at least one of a width or a length of the tracking frame being greater than or equal to the first reference length, a first straight line from a first center point of a first virtual frame to a line segment forming the first virtual frame among the multiple virtual frames at the second time point; as well as Based on the distance between the second center point of the second virtual frame among the multiple virtual frames at the second time point and the first straight line being greater than or equal to a second reference length, at least a portion of the virtual frames determined by merging are selectively skipped.

15. The method according to claim 11, further comprising: determining that a road on which the vehicle including the device is traveling is a straight road; determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; as well as All or part of the determined virtual frame is determined within a first specified distance in a direction perpendicular to the front of the vehicle.

16. The method according to claim 15, further comprising: determining a first area including the road and the edge of the road; determining a second area different from the first area including the road and the road edge; as well as Selectively skipping merging all or part of the determined virtual frame detected in the second area from all or part of the determined virtual frame detected within the first specified distance.

17. The method according to claim 11, further comprising: receiving an entry signal indicating that a vehicle has entered an enclosed space; determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; determining that the determined virtual frame is within a second specified distance in a direction perpendicular to the object; Determining that the length of each determined virtual frame is greater than or equal to a reference length; as well as Based on determining that the number of points contained in the determined virtual frame is greater than or equal to the first reference number, selectively skipping merging all or part of the determined virtual frame.

18. The method according to claim 11, further comprising: determining that the object is in the stationary state among a plurality of states including a stationary state and a moving state; Determine that the size of the tracking frame is greater than or equal to a reference size; determining that the number of points included in the tracking frame is greater than or equal to a second reference number; as well as Based on the determined type of the virtual frame being determined as a third type, selectively skipping merging all or part of the determined virtual frames, wherein the third type is different from a first type associated with a pedestrian, and the third type is different from a second type associated with a vehicle.

19. The method according to claim 18, further comprising: The type of the determined virtual frame is determined based on the density of points included in the determined virtual frame.

20. The method according to claim 19, further comprising: Based on the density of the points being greater than or equal to a reference density, the type of the determined virtual frame is determined to be the third type.

Images