Apparatus for controlling vehicle and method thereof

Abstract

An apparatus for vehicle control is presented. The apparatus includes a sensor and a processor configured to generate a tracking frame based on a virtual frame obtained from the sensor at a first point in time, convert it into a virtual frame at a second point in time, and determine a virtual frame associated with the tracking frame from a set of virtual frames at the second point in time obtained from sensor data. The processor determines whether to merge some or all of the virtual frames based on mutual distances of the virtual frames, whether the virtual frames correspond to objects separate from the road edge, or whether they are included in a tracking frame. The merge process is controlled based on the generated merge box size, a difference between the size based on the identified virtual box, or an object type. The device outputs a signal to indicate a result of the controlled merging.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority from Korean Patent Application No. 10-2023-0120570 filed on September 11, 2023 in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a vehicle control apparatus and method thereof, and more particularly, to a technology using a sensor (eg, light detection and ranging (LiDAR)). Background Art

[0004] Various studies are being conducted to assist a host vehicle in recognizing external objects while driving by using various sensors.

[0005] In particular, when the host vehicle is driving in a driving assistance device activation mode or an autonomous driving mode, external objects may be recognized by using LiDAR.

[0006] In the process of merging virtual boxes generated using LiDAR, the virtual boxes may be merged incorrectly. If the merged box obtained by merging the virtual boxes incorrectly is mapped to an external object, the type, state, or size of the external object may be incorrectly identified. If the type, state, or size of the external object is incorrectly identified, the driving route of the host vehicle may be significantly or incorrectly changed. Summary of the invention

[0007] According to the present disclosure, a device for controlling a vehicle may include a sensor and a processor, wherein the processor is configured to generate a tracking frame based on a virtual frame corresponding to an object obtained by the sensor at a first time point, by converting the virtual frame into a virtual frame at a second time point after the first time point, determine a virtual frame associated with the tracking frame in the multiple virtual frames at the second time point based on obtaining multiple virtual frames at the second time point, wherein the multiple virtual frames at the second time point are formed by multiple points obtained by the sensor at the second time point, merge all or part of the determined virtual frames based on at least one of the following conditions: the distance between the multiple virtual frames at the second time point, whether at least one of the multiple virtual frames at the second time point corresponds to an object separated from a road edge, and whether the multiple virtual frames at the second time point are included in the tracking frame, and control merging based on at least one of the size of the merged frame generated by the merging, the difference between the size of the merged frame and the size of the determined virtual frame, and the type of the object, and output a signal indicating that the merging is controlled.

[0008] In the device, the processor is configured to determine a virtual frame associated with the tracking frame in the multiple virtual frames at the second time point further based on: the tracking frame overlaps with each of the multiple virtual frames at the second time point at a specified ratio or greater, or the distances between the multiple first points included in the tracking frame and the multiple second points included in the multiple virtual frames at the second time point are within the relevant distances.

[0009] In the device, the processor is configured to determine the stationary state or moving state of the object based on: the object is not blocked, the object type is determined to be a third type different from the first type including a special vehicle, and at a second time point, the number of multiple virtual frames is less than a specified number.

[0010] In the device, the processor is configured to: determine that the object is in a stationary state, determine that at least one of the width or the length of the tracking frame is greater than or equal to a first reference length, determine a first straight line in multiple virtual frames at a second time point that is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame based on the object being in a stationary state and at least one of the width or the length of the tracking frame being greater than or equal to the first reference length, and skip merging at least a portion of the determined virtual frames based on a distance between the first straight line and a second center point of a second virtual frame in the multiple virtual frames at the second time point being greater than or equal to the second reference length.

[0011] In the device, the processor is configured to determine, at a second time point, a virtual frame with a first identifier from among multiple virtual frames, the first identifier indicating that: the virtual frame with the first identifier is closest to a vehicle including the device and has a width and a length, each of which is greater than or equal to a third reference length, determine, at the second time point, a virtual frame with a second identifier from among multiple virtual frames, the second identifier indicating an object separated from an edge of a road, and skip merging the virtual frame with the first identifier and the virtual frame with the second identifier.

[0012] In the device, the processor is configured to: determine a virtual frame not overlapping the tracking frame from among the plurality of virtual frames at the second time point, and merge a portion of the determined virtual frame excluding the virtual frame not overlapping the tracking frame.

[0013] In the device, the processor is configured to: determine at least one of a width of each of the determined virtual frames or a length of each of the determined virtual frames, compare a sum of the widths of the determined virtual frames with a width of the tracking frame, or compare a sum of the lengths of the determined virtual frames with a length of the tracking frame, and skip merging the determined virtual frames based on the sum of the widths of the determined virtual frames exceeding the width of the tracking frame, or the sum of the lengths of the determined virtual frames exceeding the length of the tracking frame.

[0014] In the device, the processor is configured to cancel at least a portion of the merging based on the size of the merging box being greater than or equal to a first reference size, or at least one of a width and a length of the merging box being greater than or equal to a fourth reference length.

[0015] In the device, the processor is configured to: determine a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among multiple points included in a determined virtual box determined on a plane formed by the x-axis and the y-axis, wherein the x-axis is a moving direction of the vehicle and the y-axis is perpendicular to the x-axis, and based on the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis being greater than or equal to a second reference size, cancel at least a portion of the merge.

[0016] In the device, the processor is configured to: determine that the object is in a moving state among a plurality of states including a stationary state and a moving state, and determine that the merged frame is located in an area exceeding a specified distance from a field of view boundary of the sensor. The processor also cancels at least a portion of the merge based on one of the following: when the type of the object is a first type including a special vehicle, the width of the merged frame is greater than or equal to the fifth reference length, or the length of the merged frame is greater than or equal to the sixth reference length, when the type of the object is a second type including a passenger car, the width of the merged frame is greater than or equal to the fifth reference length, and the length of the merged frame is greater than or equal to the sixth reference length, when the object is a third type including a pedestrian, the width or length of the merged frame is greater than or equal to the seventh reference length; and when the type of the object is not determined, the width or length of the merged frame is greater than or equal to the sixth reference length.

[0017] According to the present disclosure, a method for controlling a vehicle may include: generating a tracking frame by converting the virtual frame into a virtual frame at a second time point after the first time point based on a virtual frame corresponding to an object obtained by a sensor at a first time point by a device, determining a virtual frame associated with the tracking frame in the multiple virtual frames at the second time point based on obtaining multiple virtual frames at the second time point, the multiple virtual frames at the second time point being formed by multiple points obtained by the sensor at the second time point, merging all or part of the determined virtual frames based on at least one of the following: a distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from a road edge, and whether the virtual frame at the second time point is included in the tracking frame, and controlling merging based on at least one of a size of a merged frame generated by merging, a difference between the size of the merged frame and the size of the virtual frame determined, and a type of the object, and outputting a signal indicating that the merging is controlled

[0018] In the method, a virtual frame associated with the tracking frame in the multiple virtual frames at the second time point is determined based on the fact that the tracking frame overlaps with each of the multiple virtual frames at a specified or greater ratio at the second time point, or that the distances between the multiple first points included in the tracking frame and the multiple second points included in the multiple virtual frames at the second time point are within the relevant distances.

[0019] The method may further include determining the stationary state or moving state of the object based on: the object is not blocked; the object type is determined to be a first type including pedestrians; and the number of the plurality of virtual frames at the second time point is less than a specified number.

[0020] The method may further include: determining that the object is in a stationary state, determining that at least one of the width or the length of the tracking frame is greater than or equal to a first reference length, determining a first straight line in a plurality of virtual frames at a second time point that is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame based on the object being in a stationary state and at least one of the width or the length of the tracking frame being greater than or equal to the first reference length, and skipping merging at least a portion of the determined virtual frames based on a distance between the first straight line and a second center point of a second virtual frame in the plurality of virtual frames at the second time point being greater than or equal to the second reference length.

[0021] The method may also include: determining a virtual frame with a first identifier from multiple virtual frames at a second time point, the first identifier indicating that the virtual frame with the first identifier is closest to a vehicle with a width and a length, each of which is greater than or equal to a third reference length, determining a virtual frame with a second identifier from multiple virtual frames at the second time point, the second identifier indicating an object separated from the edge of the road, and merging the virtual frame with the first identifier and the virtual frame with the second identifier.

[0022] The method may further include determining a virtual frame that does not overlap the tracking frame from among the plurality of virtual frames at a second time point, and merging a portion of the determined virtual frame that excludes the virtual frame that does not overlap the tracking frame.

[0023] The method may further include determining at least one of a width of each of the determined virtual frames or a length of each of the determined virtual frames, comparing a sum of the widths of the determined virtual frames with a width of the tracking frame, or comparing a sum of the lengths of the determined virtual frames with a length of the tracking frame, and merging the determined virtual frames based on the sum of the widths of the determined virtual frames exceeding the width of the tracking frame, or the sum of the lengths of the determined virtual frames exceeding the length of the tracking frame.

[0024] The method may further include canceling at least a portion of the merging based on the size of the merging box being greater than or equal to the first reference size or at least one of the width and the length of the merging box being greater than or equal to a fourth reference length.

[0025] The method may also include determining a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in a determined virtual box determined on a plane formed by the x-axis and the y-axis, wherein the x-axis is a moving direction of the vehicle and the y-axis is perpendicular to the x-axis, and based on the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis being greater than or equal to a second reference size, canceling at least a portion of the merge.

[0026] The method may also include: determining that the object is in a moving state among multiple states including a stationary state and a moving state, and determining that the merge box is located in an area exceeding a specified distance from a boundary of the field of view of the sensor, and canceling at least a portion of the merge based on one of the following: when the type of the object is a first type including a special-purpose vehicle, the width of the merge box is greater than or equal to the fifth reference length, or the length of the merge box is greater than or equal to the sixth reference length, when the type of the object is a second type including a passenger car, the width of the merge box is greater than or equal to the fifth reference length, and the length of the merge box is greater than or equal to the sixth reference length, when the object is a third type including a pedestrian, the width or length of the merge box is greater than or equal to the seventh reference length; and when the type of the object is not determined, the width or length of the merge box is greater than or equal to the sixth reference length. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The above and other objects, features and advantages of the present disclosure will become more apparent through the following detailed description in conjunction with the accompanying drawings:

[0028] Figure 1 An example of a block diagram of a vehicle control device according to an example of the present disclosure is shown;

[0029] Figure 2 An example of generating a merged frame by using a virtual frame in an example of the present disclosure is shown;

[0030] Figure 3 An example of identifying distances between virtual frames in an example of the present disclosure is shown;

[0031] Figure 4 An example of comparing the width or length of a virtual frame with the width or length of a tracking frame in an example of the present disclosure is shown;

[0032] Figure 5 An example of comparing the size of a virtual box with the size of a merged box in an example of the present disclosure is shown;

[0033] Figure 6 An example of comparing the size of a virtual box with the size of a merged box in an example of the present disclosure is shown;

[0034] Figure 7 An example of a flowchart showing a vehicle control method according to another example of the present disclosure; and

[0035] Figure 8 An example of a computing system associated with a vehicle control device according to an example of the present disclosure is shown. DETAILED DESCRIPTION

[0036] Hereinafter, some examples of the present disclosure will be described in detail with reference to the accompanying drawings. When adding reference numerals to the components of each of the accompanying drawings, it should be noted that the same components have the same reference numerals even though the same components are indicated on another drawing. In addition, when describing examples of the present disclosure, if detailed descriptions associated with well-known functions or configurations may make the subject matter of the present disclosure unnecessarily obscure, these detailed descriptions will be omitted.

[0037] When describing the elements of the examples of the present disclosure, the terms first, second, A, B, (a), (b), etc. may be used herein. These terms are only used to distinguish one element from another element, but do not limit the corresponding element, regardless of the nature, order or priority of the corresponding element. In addition, unless otherwise defined, all terms including technical and scientific terms used herein should be interpreted as the conventions in the field to which the present disclosure belongs. It should be understood that the terms used herein should be interpreted as having a meaning consistent with their meaning in the context of the present disclosure and the related art, and unless clearly defined herein, will not be interpreted in an idealized or overly formal sense.

[0038] In the following, reference will be made to Figures 1 to 8 Various examples of the present disclosure are described in detail.

[0039] Figure 1 An example of a block diagram of a vehicle control device according to an example of the present disclosure is shown.

[0040] refer to Figure 1 The vehicle control device 100 according to the example of the present disclosure may be implemented inside or outside the vehicle, and some of the components included in the vehicle control device 100 may be implemented inside or outside the vehicle. In this case, the vehicle control device 100 may be integrated with the internal control unit of the vehicle, and may be implemented with a separate device so as to be connected to the control unit of the vehicle by means of a separate connection measure. For example, the vehicle control device 100 may also include Figure 1 Components not shown.

[0041] refer to Figure 1 According to an example, the vehicle control device 100 may include a processor 110 and a sensor (eg, LiDAR 120). The processor 110 or the LiDAR 120 may be electrically or operably connected to each other through electronic components including a communication bus.

[0042] Hereinafter, the fact that multiple pieces of hardware are operably coupled may include the fact that a direct or indirect connection between the multiple pieces of hardware is established by wire or wirelessly, so that the second hardware is controlled by the first hardware among the multiple pieces of hardware. Although different blocks are shown, the examples are not limited thereto.

[0043] Figure 1 Some of the multiple pieces of hardware in the vehicle control device 100 may be included in a single integrated circuit including a system on a chip (SoC). The type or number of hardware included in the vehicle control device 100 is not limited to Figure 1 For example, the vehicle control device 100 may only include Figure 1 Some of the many pieces of hardware shown.

[0044] The vehicle control device 100 according to an example may include hardware for processing data based on one or more instructions. The hardware for processing data may include a processor 110 .

[0045] For example, the hardware for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The processor 110 may have a structure of a single-core processor, or may have a structure of a multi-core processor including a dual-core, quad-core, hexa-core, or octa-core.

[0046] The LiDAR 120 included in the vehicle control apparatus 100 according to the example may obtain a data set from identifying objects around the vehicle control apparatus 100. For example, based on a pulse laser signal emitted from the LiDAR 120, reflected by the surrounding objects, and returned, the LiDAR 120 may identify at least one of the position of the surrounding objects, the moving direction of the surrounding objects, the speed of the surrounding objects, or any combination thereof.

[0047] The processor 110 included in the vehicle control device 100 according to the example may obtain a virtual frame at a first time point corresponding to an object specified at the first time point through the LiDAR 120. For example, the processor 110 may obtain the virtual frame at the first time point by using a plurality of points obtained at the first time point through the LiDAR 120.

[0048] The processor 110 may convert the virtual frame at the first time point into a virtual frame at a second time point after the first time point to generate a tracking frame.

[0049] For example, at least one of the first time point, the second time point, or any combination thereof may include a time point for obtaining a data set by LiDAR 120. For example, the second time point may be referred to as a current time point (e.g., time point "t"). For example, the second time point may include a time point at which a current frame is obtained. For example, the first time point may include a time point immediately before the current frame is obtained. For example, the first time point may include a time point "t-1". It indicates that the time point is at least one of the first time point, the second time point, or any combination thereof. However, at least one of the first time point, the second time point, or any combination thereof may represent a frame number. For example, when the second time point represents the tth frame, the first time point may represent the (t-1)th frame.

[0050] In an example, the processor 110 may obtain a plurality of points at a second point in time through the LiDAR 120. For example, the processor 110 may obtain a virtual frame at the second point in time formed by the plurality of points obtained at the second point in time through the LiDAR 120. For example, the virtual frame at the second point in time may be referred to as a “meta box”. For example, the virtual frame at the second point in time may include a frame formed based on points obtained when light emitted from the LiDAR 120 is reflected from an external object.

[0051] In an example, the processor 110 may identify a virtual frame associated with the tracking frame in the virtual frame at the second time point based on obtaining the virtual frame at the second time point.

[0052] For example, the processor 110 may identify a ratio at which the tracking frame overlaps each of the virtual frames at the second time point. For example, the processor 110 may identify that the tracking frame overlaps each of the virtual frames obtained at the second time point at a specified or greater ratio (e.g., approximately 25%).

[0053] For example, the processor 110 may identify the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point. For example, the processor 110 may determine whether the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point is within the relevant distance. For example, the relevant distance may be referred to as the "Mahalanobis distance".

[0054] In an example, the processor 110 may identify a virtual box associated with the tracking box in the virtual box at the second time point based on the fact that the tracking box overlaps with each of the virtual boxes obtained at the second time point at a specified ratio or greater, or the distances between multiple first points included in the tracking box and multiple second points included in the virtual box at the second time point are identified as being within the relevant distance.

[0055] For example, the processor 110 may identify a virtual frame associated with the tracking frame by identifying that the tracking frame overlaps with each of the virtual frames obtained at the second time point at a specified ratio or greater, or that distances between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point are within a relevant distance.

[0056] In an example, processor 110 may merge all or a portion of the associated virtual frames based on at least one of: the distance between the virtual frames at the second point in time, whether at least one of the virtual frames at the second point in time corresponds to an object separated from the edge of the road, whether the virtual frame at the second point in time is included in the tracking frame, and any combination thereof.

[0057] In an example, the processor 110 may identify whether a specified object is occluded. The processor 110 may identify that a particular object is not occluded.

[0058] In an example, the processor 110 may identify the type of the designated object. For example, the processor 110 may identify that the type of the designated object is a different type than the first designated type including a dedicated vehicle.

[0059] In an example, the processor 110 may identify the number of virtual frames at the second time point. For example, the processor 110 may identify the number of virtual frames associated with the tracking frame in the virtual frame at the second time point. The processor 110 may identify that the number of virtual frames associated with the tracking frame in the virtual frame at the second time point is less than a specified number (e.g., 3).

[0060] In an example, the processor 110 can identify the stationary state or moving state of the specified object based on the fact that the specified object is not obscured, the type of the specified object is identified as different from the first specified type including the special vehicle, and the number of virtual frames at the second time point is less than the specified number.

[0061] For example, the specialized vehicle may include at least one of a 1-ton truck, a container truck, a heavy truck, a trailer, or any combination thereof.

[0062] For example, the processor 110 can identify the stationary state or the moving state of the specified object based on the fact that the specified object is not obscured, the type of the specified object is identified as different from the first specified type including the special vehicle, and the number of virtual frames associated with the tracking frame in the virtual frame at the second time point is less than the specified number.

[0063] In an example, the processor 110 may identify that the designated object is in a stationary state among the stationary state and the moving state. The processor 110 may identify at least one of a width of the tracking frame, a length of the tracking frame, or any combination thereof based on identifying that the designated object is in a stationary state.

[0064] The processor 110 may recognize that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length (eg, approximately 30 m) based on the designated object being in a stationary state.

[0065] The processor 110 may identify a first center point of the first virtual frame in the virtual frame at the second time point. The processor 110 may identify a first straight line in the virtual frame at the second time point that is perpendicular to a line segment forming the first virtual frame from the first center point of the first virtual frame.

[0066] The processor 110 may identify a second center point of the second virtual frame in the virtual frame at the second time point.

[0067] In an example, the processor 110 may identify a first straight line in a virtual frame at a second time point that is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame based on the fact that the designated object is stationary and at least one of a width or a length of the tracking frame is greater than or equal to a first reference length.

[0068] The processor 110 may identify a distance between the first straight line and a second center point of the second virtual frame in the virtual frame at the second time point. Based on the fact that the distance between the first straight line and the second center point of the second virtual frame in the virtual frame at the second time point is greater than or equal to a second reference length (e.g., about 20 m), the processor 110 may not merge the virtual frame associated with the tracking frame in the virtual frame at the second time point.

[0069] In an example, the processor 110 may merge the associated virtual frames based on the fact that the distance between the second center point of the second virtual frame in the virtual frame at the second time point and the first straight line is less than the second reference length. The processor 110 may obtain a merged frame based on merging the associated virtual frames.

[0070] In an example, the processor 110 may identify a virtual frame having an assigned first identifier from the virtual frame at the second time point, the first identifier indicating that the virtual frame is closest to the host vehicle including the vehicle control device 100 and has a width and length greater than or equal to a third reference length (e.g., approximately 1 m). For example, the first identifier may be referred to as a “primary index”.

[0071] For example, the processor 110 may identify a virtual frame having an assigned second identifier from the virtual frame at the second time point, the second identifier indicating an object separated from the edge of the road. For example, the second identifier may be called "ContourSplitFlag".

[0072] In an embodiment, the processor 110 may not merge the virtual frame assigned the first identifier and the virtual frame assigned the second identifier. For example, when trying to merge the virtual frame assigned the first identifier and the virtual frame assigned the second identifier, the processor 110 may not merge the virtual frame assigned the first identifier and the virtual frame assigned the second identifier. For example, the processor 110 may exclude the virtual frame assigned the second identifier from the merge candidates.

[0073] Hereinafter, an example of determining whether a virtual frame at the second time point is included in a tracking frame based on at least one of a width, a length, or any combination thereof of a virtual frame associated with a tracking frame in the virtual frame at the second time point will be described later.

[0074] In an example, the processor 110 may identify at least one of a width, a length, or any combination thereof of each of the associated virtual frames.

[0075] For example, the processor 110 may obtain the sum of the associated virtual frame widths based on identifying the widths of the associated virtual frames.

[0076] For example, the processor 110 may obtain the sum of the associated virtual frame lengths based on identifying the lengths of the associated virtual frames.

[0077] The processor 110 may compare the sum of the widths of the associated virtual frames with the width of the tracking frame. The processor 110 may compare the sum of the lengths of the associated virtual frames with the length of the tracking frame.

[0078] For example, the processor 110 may compare the sum of the widths of the associated virtual frames with the width of the tracking frame, or may compare the sum of the lengths of the associated virtual frames with the length of the tracking frame.

[0079] In an embodiment, the processor 110 may identify that the sum of the widths of the associated virtual frames exceeds the width of the tracking frame. The processor 110 may not merge the associated virtual frames based on the sum of the widths of the associated virtual frames exceeding the width of the tracking frame.

[0080] For example, the fact that the sum of the widths of the associated virtual boxes exceeds the width of the tracking box may include the fact that the associated virtual boxes are not included in the tracking box.

[0081] The processor 110 may identify that the sum of the widths of the associated virtual frames is less than or equal to the width of the tracking frame. The processor 110 may merge the associated virtual frames based on the sum of the widths of the associated virtual frames being less than or equal to the width of the tracking frame.

[0082] In an embodiment, the processor 110 may identify that the sum of the lengths of the associated virtual frames exceeds the length of the tracking frame. The processor 110 may not merge the associated virtual frames based on the sum of the lengths of the associated virtual frames exceeding the length of the tracking frame.

[0083] For example, the fact that the sum of the lengths of the associated virtual frames exceeds the length of the tracking frame may include the fact that the associated virtual frames are not included in the tracking frame.

[0084] The processor 110 may identify that the sum of the lengths of the associated virtual frames is less than or equal to the length of the tracking frame. The processor 110 may merge the associated virtual frames based on the sum of the lengths of the associated virtual frames being less than or equal to the length of the tracking frame.

[0085] In an embodiment, the processor 110 may not merge the associated virtual frames based on the sum of the widths of the associated virtual frames exceeding the width of the tracking frame, or the sum of the lengths of the associated virtual frames exceeding the length of the tracking frame.

[0086] The processor 110 may merge the associated virtual frames based on the sum of the widths of the associated virtual frames being less than or equal to the width of the tracking frame, and the sum of the lengths of the associated virtual frames being less than or equal to the length of the tracking frame.

[0087] As described above, the processor 110 can merge all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from the edge of the road, whether the virtual frame at the second time point is included in the tracking frame, or any combination.

[0088] Hereinafter, an example of maintaining or canceling merging based on at least one of the size of a merged box generated by merging, a difference between the size of the merged box and a size based on an associated virtual box, a type of a specified object, or any combination thereof will be described later.

[0089] In an embodiment, the processor 110 may merge all or part of the associated virtual frames. The processor 110 may generate a merged frame based on merging all or part of the associated virtual frames.

[0090] In an embodiment, the processor 110 may identify the size of the merged box. The processor 110 may identify that the size of the merged box is greater than or equal to the first reference size (eg, approximately 90 m 2 ).

[0091] In an embodiment, the processor 110 may identify at least one of the width of the merged box, the length of the merged box, or any combination thereof. The processor 110 may identify that one of the width and the length of the merged box is greater than or equal to a fourth reference length (eg, approximately 20 m).

[0092] In an embodiment, the processor 110 may cancel the merging based on the size of the merged box being greater than or equal to the first reference size, or one of the width and length of the merged box being greater than or equal to the fourth reference length. For example, the processor 110 may separate the merged box into virtual boxes at the second time point based on the size of the merged box being greater than or equal to the first reference size, or one of the width and length of the merged box being greater than or equal to the fourth reference length.

[0093] In an embodiment, the processor 110 may maintain the consolidation based on the size of the consolidation box being smaller than the first reference size, or the width or length of the consolidation box being smaller than a fourth reference length.

[0094] The processor 110 may output the merged box based on the size of the merged box being smaller than the first reference size, or the width or length of the merged box being smaller than a fourth reference length.

[0095] The processor 110 may map the merged box to the external object based on the size of the merged box being smaller than the first reference size or the width or length of the merged box being smaller than the fourth reference length. Hereinafter, the operation of outputting the merged box may include the operation of mapping the merged box to the external object.

[0096] In an embodiment, the processor 110 may identify a virtual frame associated with the tracking frame in a second virtual frame on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis. The processor 110 may identify a plurality of points included in the associated virtual frame based on identifying the associated virtual frame on the plane formed by the x-axis and the y-axis.

[0097] The processor 110 may identify a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in the associated virtual frame.

[0098] The processor 110 can obtain a quadrilateral based on the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis. The processor 110 can obtain a quadrilateral including the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis.

[0099] For example, processor 110 may identify a first line segment that is parallel to the y-axis and includes a minimum value on the x-axis. Processor 110 may identify a second line segment that is parallel to the y-axis and includes a maximum value on the x-axis. Processor 110 may identify a third line segment that is parallel to the x-axis and includes a minimum value on the y-axis. Processor 110 may identify a fourth line segment that is parallel to the x-axis and includes a maximum value on the y-axis.

[0100] The processor 110 may obtain a quadrilateral based on the first line segment, the second line segment, the third line segment, and the fourth line segment.

[0101] In an embodiment, the processor 110 may identify the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis. The processor 110 may identify the difference between the size of the merged box and the size of the quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis.

[0102] The processor 110 may recognize that the difference between the size of the merged frame and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to a second reference size (e.g., about 2 m 2 ).

[0103] The processor 110 may determine the size of the merged frame based on the difference between the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis being greater than or equal to a second reference size (e.g., about 2 m). 2 ) to unmerge the merge box.

[0104] In an embodiment, the processor 110 may identify that the designated object is in a moving state among the stationary state and the moving state. The processor 110 may identify that the merged frame is located in an area exceeding a designated distance from a field of view (FOV) boundary of the LiDAR 120.

[0105] In an embodiment, the processor 110 may recognize that the designated object is in a moving state among the stationary state and the moving state, and that the merged frame is located in an area exceeding a designated distance from the FOV boundary of the LiDAR 120 .

[0106] In an embodiment, the processor 110 may identify the type of the designated object. The processor 110 may identify at least one of a width of the merged box, a length of the merged box, or any combination thereof.

[0107] In an embodiment, when the type of the designated object is the first designated type including a special vehicle, the processor 110 may recognize that the width of the merged box is greater than or equal to the fifth reference length (e.g., about 4.5 m), or the length of the merged box is greater than or equal to the sixth reference length (e.g., about 20 m). When the type of the designated object is the first designated type including a special vehicle, the processor 110 may cancel the merge based on the width of the merged box being greater than or equal to the fifth reference length or the length of the merged box being greater than or equal to the sixth reference length.

[0108] In an embodiment, when the type of the designated object is the first designated type including a special vehicle, the processor 110 may maintain the merging based on the width of the merging box being less than the fifth reference length and the length of the merging box being less than the sixth reference length. When the type of the designated object is the first designated type including a special vehicle, the processor 110 may output the merging box based on the width of the merging box being less than the fifth reference length and the length of the merging box being less than the sixth reference length.

[0109] In an embodiment, when the type of the designated object is the second designated type including a passenger car, the processor 110 may recognize that the width of the merged box is greater than or equal to a fifth reference length (e.g., about 4.5 m), and the length of the merged box is greater than or equal to a sixth reference length (e.g., about 20 m). When the type of the designated object is the second designated type including a passenger car, the processor 110 may cancel the merge based on the width of the merged box being greater than or equal to the fifth reference length and the length of the merged box being greater than or equal to the sixth reference length.

[0110] In an embodiment, when the type of the designated object is the second designated type including a passenger car, the processor 110 may maintain the merging based on the width of the merging box being less than the fifth reference length and the length of the merging box being less than the sixth reference length. When the type of the designated object is the second designated type including a passenger car, the processor 110 may output the merging box based on the width of the merging box being less than the fifth reference length and the length of the merging box being less than the sixth reference length.

[0111] In an embodiment, the processor 110 may recognize that the type of the specified object is the third specified type including pedestrians. When the type of the specified object is the third specified type including pedestrians, the processor 110 may recognize that the width or length of the merged frame is greater than or equal to a seventh reference length (e.g., about 1.5 m). When the type of the specified object is the third specified type including pedestrians, the processor 110 may cancel the merge based on the width or length of the merged frame being greater than or equal to the seventh reference length.

[0112] In an embodiment, when the type of the designated object is the third designated type including pedestrians, the processor 110 may maintain the merging based on the width and length of the merged box being less than the seventh reference length. For example, when the type of the designated object is the third designated type including pedestrians, the processor 110 may output the merged box based on the width and length of the merged box being less than the seventh reference length.

[0113] In an embodiment, when the type of the specified object is not identified, the processor 110 may identify that the width or length of the merged box is greater than or equal to the sixth reference length. When the type of the specified object is not identified, the processor 110 may cancel the merge based on the width or length of the merged box being greater than or equal to the sixth reference length.

[0114] In an embodiment, when the type of the specified object is not identified, the processor 110 may identify that the width and length of the merged box are less than the sixth reference length. When the type of the specified object is not identified, the processor 110 may cancel the merge based on the width and length of the merged box being less than the sixth reference length. When the type of the specified object is not identified, the processor 110 may output the merged box based on the width and length of the merged box being less than the sixth reference length.

[0115] As described above, the processor 110 included in the vehicle control device 100 according to the embodiment can determine whether to merge the virtual frame associated with the tracking frame in the second virtual frame. The processor 110 can obtain a merged frame based on merging all or part of the associated virtual frames, and then can determine the appropriateness of the obtained merged frame.

[0116] Figure 2 An example of generating a merged frame by using a virtual frame in an embodiment of the present disclosure is shown.

[0117] Figure 2 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 2 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0118] refer to Figure 2 In the first example 201, the processor included in the vehicle control device according to the embodiment can obtain a virtual frame 211 at a first time point corresponding to a specified object at a first time point through LiDAR. For example, the processor can obtain the virtual frame 211 by using a plurality of points obtained at the time point "t-1".

[0119] refer to Figure 2In the second example 203, in an embodiment, the processor may convert the virtual frame 211 at the first time point into a virtual frame at a second time point after the first time point to generate a tracking frame 213. For example, the second time point may be referred to as a current frame.

[0120] refer to Figure 2 In the third example 205 , in an embodiment, the processor may obtain virtual frames 221 , 223 , 225 , and 227 at a second time point through LiDAR, where the virtual frames are formed by a plurality of points obtained at the second time point.

[0121] The processor may identify a virtual frame associated with the tracking frame 213 among the virtual frames 221 , 223 , 225 , and 227 at the second time point based on obtaining the virtual frames 221 , 223 , 225 , and 227 at the second time point formed by the plurality of points obtained at the second time point.

[0122] For example, the processor may determine whether the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on at least one of the tracking point, the output point, the center point, the ratio of overlap of the tracking frame 213 with the virtual frames 221, 223, 225, and 227 at the second time point, or any combination thereof.

[0123] For example, the processor may determine whether the tracking frame 213 is associated with the virtual frames 221 , 223 , 225 , and 227 at the second time point based on the correlation distances between the tracking frame 213 and the virtual frames 221 , 223 , 225 , and 227 at the second time point.

[0124] For example, the processor may determine whether the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on the correlation distance obtained based on the Mahalanobis distance. For example, when the Mahalanobis distance is less than or equal to the reference distance (e.g., about 0.5 m), the processor may determine that the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point.

[0125] For example, the processor may determine that the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on the fact that the tracking frame 213 overlaps with the virtual frames 221, 223, 225, and 227 at the second time point at a reference ratio (e.g., approximately 25%) or more.

[0126] refer to Figure 2 In the fourth example 207 , in an embodiment, the processor may generate a merged frame 230 based on the associated virtual frames 221 , 223 , 225 , and 227 .

[0127] For example, the processor may merge the virtual boxes 221, 223, 225, and 227 associated with the tracking box 213. For example, the processor may merge all or part of the virtual boxes 221, 223, 225, and 227 associated with the tracking box 213.

[0128] The processor may generate a merged frame 230 based on merging all or a portion of the associated virtual frames 221 , 223 , 225 , and 227 .

[0129] Hereinafter, an embodiment of a vehicle control apparatus for generating the merging box 230 will be described later.

[0130] Figure 3 An example of identifying the distance between virtual frames in an embodiment of the present disclosure is shown.

[0131] Figure 3 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 3 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0132] Figure 3 Example 301 in FIG. 4 shows an example of merging all or part of a virtual frame associated with a tracking frame based on a distance between virtual frames at a second time point.

[0133] Referring to example 301, the processor included in the vehicle control device according to the embodiment can identify virtual frames 311 and 313 associated with the tracking frame 310 in the virtual frame at the second time point. For example, when the specified object corresponding to the tracking frame 310 is identified as being in a stationary state, the processor can perform the operation described in example 301.

[0134] In an embodiment, the processor 110 may determine whether the specified object is in a stationary state or a moving state based on at least one of whether the specified object corresponding to the tracking frame 310 is occluded, the type of the specified object, the number of virtual frames at the second time point, or any combination thereof.

[0135] The processor can identify the stationary state or moving state of the specified object based on the fact that the specified object is not obscured, the type of the specified object is identified as different from the first specified type including the special vehicle, and the number of virtual frames at the second time point is less than a specified number (e.g., approximately 3).

[0136] The processor may recognize that the designated object is stationary, and may recognize that at least one of the width 319 of the tracking frame 310 , the length of the tracking frame 310 , or any combination thereof is greater than or equal to a first reference length (eg, approximately 30 m).

[0137] For example, when the designated object corresponding to the tracking frame 310 is stationary, the width 319 of the tracking frame 310 is greater than or equal to 30 m, and the distance between the associated virtual frames is greater than or equal to 20 m, the processor may not merge the associated virtual frames.

[0138] For example, based on the fact that the first virtual frame 311 is associated with the second virtual frame 313 , the processor may identify a first straight line 315 that is perpendicular to a line segment forming the first virtual frame 311 from a center point of the first virtual frame 311 .

[0139] The processor may identify a distance 317 between the first straight line 315 and the center point of the second virtual frame 313. Based on identifying that the distance between the first straight line 315 and the center point of the second virtual frame 313 is greater than or equal to a reference distance (e.g., about 20 m), the processor may not merge the first virtual frame 311 and the second virtual frame 313.

[0140] The operation of the processor not to merge the first virtual frame 311 and the second virtual frame 313 may include an operation of the processor excluding at least one of the first virtual frame 311 , the second virtual frame 313 , or any combination thereof from merge candidates.

[0141] Figure 4 An example of comparing the width or length of the virtual frame with the width or length of the tracking frame in an embodiment of the present disclosure is shown.

[0142] Figure 4 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 4 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0143] refer to Figure 4 , in the first example 401, the processor included in the vehicle control apparatus according to the embodiment may identify virtual frames 411 and 413 associated with the tracking frame 410 in the virtual frame at the second time point.

[0144] In an embodiment, the processor may recognize an array of points included in each of the associated virtual frames 411 and 413. For example, the processor may recognize that the array of points included in each of the associated virtual frames 411 and 413 has a specified shape. For example, the specified shape may include an L-shape among an L-shape and an I-shape.

[0145] In an embodiment, the processor may obtain at least one of a width, a length, or any combination thereof of each of the associated virtual frames 411 and 413 based on a point array in each of the associated virtual frames 411 and 413 having a specified shape.

[0146] In an embodiment, the processor may obtain at least one of a first width, a first length, or any combination thereof of a first virtual frame 411 of the associated virtual frames 411 and 413. The processor may obtain at least one of a second width, a second length, or any combination thereof of a second virtual frame 413 of the associated virtual frames 411 and 413.

[0147] The processor may obtain the sum of the first width of the first virtual frame 411 and the second width of the second virtual frame 413. The processor may obtain the sum of the first length of the first virtual frame 411 and the second length of the second virtual frame 413.

[0148] The processor may compare the sum of the first width and the second width with the width of the tracking frame 410. The processor may compare the sum of the first length and the second length with the length of the tracking frame 410.

[0149] Based on the sum of the first width and the second width exceeding the width of the tracking frame 410, or the sum of the first length and the second length exceeding the length of the tracking frame 410, the processor may not merge the associated virtual frames 411 and 413. The first example 401 may include an example where the sum of the first length and the second length exceeds the length of the tracking frame 410.

[0150] In the second example 403 , the processor included in the vehicle control apparatus according to the embodiment may identify virtual frames 421 and 423 associated with the tracking frame 420 in the virtual frame at the second time point.

[0151] In an embodiment, the processor may recognize that the dot array included in each of the associated virtual frames 421 and 423 has a specified shape. For example, the processor may recognize that the dot array included in each of the associated virtual frames 421 and 423 has a specified shape. For example, the specified shape may include an L-shape among an L-shape and an I-shape.

[0152] In an embodiment, the processor may obtain at least one of a width, a length, or any combination thereof of each of the associated virtual frames 421 and 423 based on a point array included in each of the associated virtual frames 421 and 423 having a specified shape.

[0153] In an embodiment, the processor may obtain at least one of a third width, a third length, or any combination thereof of a third virtual frame 421 of the associated virtual frames 421 and 423. The processor may obtain at least one of a fourth width, a fourth length, or any combination thereof of a fourth virtual frame 423 of the associated virtual frames 421 and 423.

[0154] The processor may obtain the sum of the third width of the third virtual frame 421 and the fourth width of the fourth virtual frame 423. The processor may obtain the sum of the third length of the third virtual frame 421 and the fourth length of the fourth virtual frame 423.

[0155] The processor may compare the sum of the third width and the fourth width with the width of the tracking frame 420. The processor may compare the sum of the third length and the fourth length with the length of the tracking frame 420.

[0156] Based on the sum of the third width and the fourth width exceeding the width of the tracking frame 420, or the sum of the third length and the fourth length exceeding the length of the tracking frame 420, the processor may not merge the associated virtual frames 421 and 423. The second example 403 may include an example where the sum of the third width and the fourth width exceeds the width of the tracking frame 420.

[0157] Figure 5 An example of comparing the size of the virtual frame with the size of the merged frame in an embodiment of the present disclosure is shown.

[0158] Figure 5 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 5 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0159] refer to Figure 5 , the processor included in the vehicle control apparatus according to the embodiment may generate a merged frame 510 based on the associated virtual frames 511 and 513 .

[0160] In an embodiment, the processor may identify the size of the merged box 510. For example, the processor may identify the width and length of the merged box 510. The processor may identify the size of the merged box 510 based on identifying the width and length of the merged box 510.

[0161] For example, the processor may identify the merged frame 510 on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis. The processor may identify the size of the merged frame 510 on the plane formed by the x-axis and the y-axis.

[0162] For example, the processor may identify coordinates of each of the vertices included in the merged box 510 on a plane formed by the x-axis and the y-axis. The processor may identify a line segment forming the merged box 510 based on the coordinates of each of the vertices included in the merged box 510.

[0163] The processor may identify the length of each of the line segments forming the merged frame 510. For example, among the line segments forming the merged frame 510, each of the line segments corresponding to the y-axis direction may be referred to as a “width.” For example, among the line segments forming the merged frame 510, each of the line segments corresponding to the x-axis direction may be referred to as a “length.”

[0164] In an embodiment, the processor may determine whether the length 515 of the merged frame 510 exceeds a reference length (e.g., about 20 m). The processor may cancel the merge based on the length 515 of the merged frame 510 exceeding the reference length. For example, the processor may separate the merged frame 510 into virtual frames 511 and 513 based on the length of the merged frame 510 exceeding the reference length.

[0165] In an embodiment, the processor may maintain the merging based on the length of the merged box 510 being less than or equal to the reference length. For example, the processor may output the merged box 510 based on the length 515 of the merged box 510 being less than or equal to the reference length.

[0166] The processor may identify the width and length of the merged box 510 based on coordinates on a plane formed by the x-axis and the y-axis included in the merged box 510. The processor may identify the size of the merged box 510 based on the identified width and the identified length of the merged box 510.

[0167] In an embodiment, based on identifying the size of the merged box 510 , the processor may maintain or cancel the merge of the merged box 510 .

[0168] For example, the processor may determine whether the size of the merged box 510 exceeds a reference size (eg, 90 m 2 ).

[0169] For example, the processor may cancel the merging of the merging box 510 based on the size of the merging box 510 exceeding the reference size.

[0170] For example, the processor may maintain the merging of the merged box 510 based on the size of the merged box 510 being less than or equal to the reference size. When maintaining the merged box 510, the processor may output the merged box 510.

[0171] Figure 6 An example of comparing the size of the virtual frame with the size of the merged frame in an embodiment of the present disclosure is shown.

[0172] Figure 6 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 6 The operation can be included in Figure 1 Executed by the processor 110 in the vehicle control device 100.

[0173] refer to Figure 6 , the processor included in the vehicle control device according to the embodiment may compare the size of the merged boxes 610 and 630 with the size of the virtual boxes 611, 613, 631, and 633. For example, the processor may generate a quadrilateral based on the points forming the virtual boxes 611, 613, 631, and 633. The processor may identify the size of the generated quadrilateral and the size of the merged box 610 or 630. Based on the size difference between the size of the generated quadrilateral and the merged boxes 610 and 630, the processor may maintain the merge of the merged boxes 610 or 630, or may cancel the merge of the merged boxes 610 or 630.

[0174] refer to Figure 6 In the first example 601 in FIG. 1 , in an embodiment, the processor may generate a merged box 610 based on virtual boxes 611 and 613 associated with the tracking box. The processor may identify the size of the merged box 610 based on generating the merged box 610 .

[0175] The processor may identify vertices of the virtual frames 611 and 613. The processor may identify at least one of a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, a maximum value on the y-axis, or any combination thereof among the vertices of the virtual frames 611 and 613.

[0176] Specifically, the processor may identify a plurality of points included in the associated virtual frames 611 and 613 identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis. The processor may identify a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in the associated virtual frames 611 and 613 identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis.

[0177] The processor may form a quadrilateral based on the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis. The processor may identify the size of the quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis.

[0178] The processor may recognize that the difference between the size of the merged box 610 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to a reference size (e.g., 2 m 2 ).

[0179] The processor may unmerge the merging box 610 based on the fact that the difference between the size of the merging box 610 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to the reference size.

[0180] The processor may maintain the merging of the merging box 610 based on the fact that the difference 620 between the size of the merging box 610 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is smaller than the reference size. The processor may maintain the merging of the merging box 610 based on the fact that the difference 620 between the size of the merging box 610 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is smaller than the reference size, and may output the merging box 610.

[0181] refer to Figure 6 In the second example 603, in an embodiment, the processor may identify that the virtual boxes 631 and 633 associated with the tracking box are not parallel to at least one of the x-axis, the y-axis, or any combination thereof. Although the virtual boxes 631 and 633 associated with the tracking box are not parallel to at least one of the x-axis, the y-axis, or any combination thereof, the processor may identify differences 641 and 643 between the size of the merged box 630 and the sizes of the virtual boxes 631 and 633.

[0182] For example, the processor may identify a plurality of points included in the associated virtual frames 631 and 633 identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis. The processor may identify a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in the associated virtual frames 631 and 633 identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis.

[0183] The processor may form a quadrilateral based on the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis. The processor may identify the size of the quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis.

[0184] The processor may recognize that the difference between the size of the merged box 630 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to a reference size (e.g., 2 m 2 ).

[0185] The processor may unmerge the merging box 630 based on the fact that the difference between the size of the merging box 630 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to the reference size.

[0186] The processor may maintain the merging of the merging box 630 based on the fact that the difference 641 or 643 between the size of the merging box 630 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is smaller than the reference size. The processor may maintain the merging of the merging box 630 based on the fact that the difference 641 or 643 between the size of the merging box 630 and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is smaller than the reference size, and may output the merging box 630.

[0187] Figure 7 An example of a flowchart of a vehicle control method according to another embodiment of the present disclosure is shown.

[0188] In the following, it is assumed Figure 1 The vehicle control device 100 executes Figure 7 In addition, Figure 7 In the description, it can be understood that the operations described as being performed by the device are controlled by the processor 110 of the vehicle control device 100.

[0189] refer to Figure 7 In operation S701, the vehicle control method according to the embodiment may include an operation of generating a tracking frame by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point based on obtaining a virtual frame at a first time point corresponding to a designated object at the first time point through LiDAR.

[0190] In operation S703, the vehicle control method according to the embodiment may include: based on obtaining a virtual frame at a second time point through LiDAR, identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point, the virtual frame at the second time point being formed by a plurality of points obtained at the second time point;

[0191] For example, the vehicle control method can identify a virtual box associated with the tracking box in the virtual box at the second time point based on at least one of an area overlapping with the tracking box and each of the virtual boxes at the second time point, a related distance between points included in each of the virtual boxes at the second time point and points included in the tracking box, or any combination thereof.

[0192] In operation S705, the vehicle control method according to an embodiment may include: merging all or part of the operations in the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from the edge of the road, or any combination thereof.

[0193] In operation S707, the vehicle control method according to an embodiment may include: maintaining or canceling the merging operation based on at least one of the size of the merged box generated by the merging, the difference between the size of the merged box and the size based on the associated virtual box, the type of the specified object, or any combination thereof.

[0194] Figure 8 A computing system associated with a vehicle control device according to an embodiment of the present disclosure is shown.

[0195] refer to Figure 8 , the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 , which are connected to each other via a bus 1200 .

[0196] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the storage device 1600. Each of the memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a read-only memory (ROM) and a random access memory (RAM).

[0197] Therefore, the operation of the method or algorithm described in conjunction with the embodiments disclosed in the specification can be directly implemented using a hardware module, a software module, or a combination of a hardware module and a software module executed by the processor 1100. The software module can reside on a storage medium (i.e., the memory 1300 and / or the storage device 1600), such as a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk drive, a removable disk, or a compact disk ROM (CD-ROM).

[0198] The storage medium may be coupled to the processor 1100. The processor 1100 may read information from the storage medium and may write information in the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor and the storage medium may be implemented with an application specific integrated circuit (ASIC). The ASIC may be provided in a user terminal. Alternatively, the processor and the storage medium may be implemented with separate components in a user terminal.

[0199] The present disclosure aims to solve the above-mentioned problems occurring in the prior art while maintaining the advantages achieved by the prior art unchanged.

[0200] An example of the present disclosure provides a vehicle control apparatus and method thereof, which merges a virtual frame generated by using a sensor (eg, LiDAR) or cancels or cancels the merged frame.

[0201] An example of the present disclosure provides a vehicle control apparatus and method thereof, which reduces a load on a processor by determining whether to merge virtual frames before merging the virtual frames.

[0202] An example of the present disclosure provides a vehicle control device and a method thereof, which determines whether to merge a virtual frame by using the size of the virtual frame and the size of the tracking frame, or determines whether to cancel the merge by using the size of the virtual frame and the size of the merged frame, and outputs the merged frame to output a correct merged frame.

[0203] The technical problems to be solved by the present disclosure are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs through the following description.

[0204] According to an example of the present disclosure, a vehicle control device may include light detection and ranging (LiDAR) and a processor. The processor may generate a tracking frame based on a virtual frame at a first time point corresponding to a specified object at the first time point obtained by LiDAR, by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point; may identify or determine a virtual frame associated with the tracking frame in the virtual frame at the second time point based on the virtual frame obtained at the second time point, the virtual frame at the second time point being formed by a plurality of points obtained by LiDAR at the second time point; may merge all or part of the associated virtual frames based on at least one of the following: the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to a virtual frame at the second time point, or whether the virtual frame at the second time point corresponds to a virtual frame at the second time point. an object separated from the edge of the road (for example, a boundary line, a guardrail, a fence or a road wall, etc.), whether the virtual frame at the second time point is included in the tracking frame, or the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from the edge of the road, and whether the virtual frame at the second time point is included in the tracking frame; and the merging can be maintained or canceled based on at least one of the following: the size of the merged box generated by the merging, the difference between the size of the merged box and the size based on the associated virtual box, the type of the specified object, or the combination of the size of the merged box, the difference between the size of the merged box and the size based on the associated virtual box, and the type of the specified object.

[0205] In an example, the processor may identify or determine a virtual box in the virtual box at the second time point that is associated with the tracking box based on the fact that the tracking box overlaps with each of the virtual boxes at the second time point at a specified ratio or greater, or the distances between a plurality of first points included in the tracking box and a plurality of second points included in the virtual box at the second time point are identified as being within the relevant distance.

[0206] In an example, the processor can identify or determine the stationary or moving state of the specified object based on the fact that the specified object is not obscured, the type of the specified object is identified as a type different from the first specified type including the special vehicle, and the number of virtual frames at the second time point is less than a specified number.

[0207] In an example, the processor may identify or determine that the specified object is stationary, may identify or determine that at least one of the width or the length of the tracking frame is less than or equal to a first reference length, may identify or determine a first straight line in the virtual frame at the second time point that is perpendicular to a line segment that forms the first virtual frame from a first center point of the first virtual frame based on the fact that the specified object is stationary and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, and may not merge the associated virtual frames based on the fact that a distance between the first straight line and a second center point of the second virtual frame in the virtual frame at the second time point is greater than or equal to the second reference length.

[0208] In the example, the processor can identify or determine a virtual frame with a first identifier from the virtual frame at the second time point, the first identifier indicating that: the virtual frame is closest to the main vehicle including the vehicle control device and has a width and a length, each of which is greater than or equal to a third reference length, the processor can identify or determine a virtual frame with a second identifier from the virtual frame at the second time point, the second identifier indicating an object separated from the edge of the road, and the processor may not merge the virtual frame with the first identifier and the virtual frame with the second identifier.

[0209] In an example, the processor may identify or determine a virtual frame that does not overlap with the tracking frame from the virtual frame at the second time point, and may merge a portion of the associated virtual frame except for the virtual frame that does not overlap thus.

[0210] In an example, the processor may identify or determine at least one of a width of each of the associated virtual frames, a length of each of the associated virtual frames, or a combination of the width of each of the associated virtual frames and the length of each of the associated virtual frames, may compare the sum of the widths of the associated virtual frames with the width of the tracking box, or may compare the sum of the lengths of the associated virtual frames with the length of the tracking box, and may not merge the associated virtual frames based on the sum of the widths of the associated virtual frames exceeding the width of the tracking box or the sum of the lengths of the associated virtual frames exceeding the length of the tracking box.

[0211] In an example, the processor may cancel the merging based on that the size of the merging box is greater than or equal to the first reference size, or one of the width and the length of the merging box is greater than or equal to the fourth reference length.

[0212] In an example, the processor can identify or determine the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis among multiple points included in an associated virtual box identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis, and can cancel the merger based on the fact that the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to a second reference size.

[0213] In an example, the processor can identify that the specified object is in a moving state in a stationary state and a moving state, and the merge box is located in an area that is more than a specified distance from the boundary of the field of view (FOV) of the LiDAR. If the type of the specified object is a first specified type including special vehicles, the merge can be canceled based on the width of the merge box being greater than or equal to the fifth reference length, or the length of the merge box being greater than or equal to the fifth reference length. If the type of the specified object is a second specified type including passenger cars, the merge can be canceled based on the width of the merge box being greater than or equal to the fifth reference length, and the length of the merge box being greater than or equal to the sixth reference length. If the type of the specified object is a third specified type including pedestrians, the merge can be canceled based on the width or length of the merge box being greater than or equal to the seventh reference length. If the type of the specified object is not identified, the merge can be canceled based on the width or length of the merge box being greater than or equal to the sixth reference length.

[0214] According to an example of the present disclosure, a vehicle control method may include: generating a tracking frame based on a virtual frame at a first time point corresponding to a specified object at the first time point obtained by LiDAR, by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point; identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point based on the virtual frame obtained at the second time point, the virtual frame at the second time point being formed by a plurality of points obtained by LiDAR at the second time point; merging all or part of the associated virtual frames based on at least one of the following conditions: a distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from a road edge, whether the virtual frame at the second time point is included in the tracking frame, or a combination of a distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from a road edge, and whether the virtual frame at the second time point is included in the tracking frame; and maintaining or canceling the merging based on at least one of the following conditions: a size of a merged frame generated by merging, a difference between a size of the merged frame and a size based on the associated virtual frame, a type of the specified object, or a combination of a size of the merged frame, a difference between a size of the merged frame and a size based on the associated virtual frame, and a type of the specified object.

[0215] According to an example, the vehicle control method may further include identifying a virtual box in the virtual box at the second time point that is associated with the tracking box based on the fact that the tracking box overlaps with each of the virtual boxes at the second time point at a specified ratio or greater, or the distance between multiple first points included in the tracking box and multiple second points included in the virtual box at the second time point is identified as being within the relevant distance.

[0216] According to the example, the vehicle control method may also include identifying the stationary or moving state of the specified object based on the fact that the specified object is not obscured, the type of the specified object is identified as a type different from a first specified type including a special-purpose vehicle, and the number of virtual frames at the second time point is less than a specified number.

[0217] According to an example, the vehicle control method may further include identifying that the specified object is in a stationary state, identifying that at least one of the width or the length of the tracking frame is less than or equal to a first reference length, identifying a first straight line in the virtual frame at the second time point that is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame based on the fact that the specified object is in a stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, and not merging the associated virtual frames based on the fact that a distance between the first straight line and a second center point of the second virtual frame in the virtual frame at the second time point is greater than or equal to the second reference length.

[0218] According to the example, the vehicle control method may also include identifying a virtual frame with a first identifier from the virtual frame at the second time point, the first identifier indicating that the virtual frame is closest to the main vehicle including the vehicle control device and has a width and a length, each of which is greater than or equal to a third reference length, identifying or determining a virtual frame with a second identifier from the virtual frame at the second time point, the second identifier indicating an object separated from the edge of the road, and merging the virtual frame with the first identifier and the virtual frame with the second identifier.

[0219] According to an example, the vehicle control method may further include identifying a virtual frame that does not overlap with the tracking frame from among the virtual frames at the second time point, and merging a portion of the associated virtual frame except for the virtual frame that does not overlap thereby.

[0220] According to an example, the vehicle control method may also include identifying at least one of a width of each of the associated virtual frames, a length of each of the associated virtual frames, or a combination of a width of each of the associated virtual frames and a length of each of the associated virtual frames, comparing the sum of the widths of the associated virtual frames with the width of the tracking frame, or comparing the sum of the lengths of the associated virtual frames with the length of the tracking frame, and merging the associated virtual frames based on the sum of the widths of the associated virtual frames exceeding the width of the tracking frame or the sum of the lengths of the associated virtual frames exceeding the length of the tracking frame.

[0221] According to an example, the vehicle control method may further include canceling merging based on that the size of the merging box is greater than or equal to the first reference size, or one of the width and the length of the merging box is greater than or equal to a fourth reference length.

[0222] According to an example, the vehicle control method may also include identifying the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis among multiple points included in an associated virtual box identified on a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis, and canceling the merging based on the fact that the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis, and the maximum value on the y-axis is greater than or equal to a second reference size.

[0223] According to an example, the vehicle control method may also include identifying that a specified object is in a moving state in a stationary state and a moving state, and that the merge box is located in an area that is more than a specified distance from a boundary of the FOV of the LiDAR, and if the type of the specified object is a first specified type including special-purpose vehicles, canceling the merging based on that the width of the merge box is greater than or equal to a fifth reference length, or the length of the merge box is greater than or equal to the fifth reference length, if the type of the specified object is a second specified type including a passenger car, canceling the merging based on that the width of the merge box is greater than or equal to the fifth reference length, and the length of the merge box is greater than or equal to a sixth reference length, if the type of the specified object is a third specified type including pedestrians, canceling the merging based on that the width or length of the merge box is greater than or equal to a seventh reference length, and if the type of the specified object is not identified, canceling the merging based on that the width or length of the merge box is greater than or equal to the sixth reference length.

[0224] The above description is merely an example of the technical idea of ​​the present disclosure, and various modifications and changes may be made by those skilled in the art without departing from the essential characteristics of the present disclosure.

[0225] Therefore, the embodiments of the present disclosure are not intended to limit but to explain the technical ideas of the present disclosure, and the scope and spirit of the present disclosure are not limited by the above embodiments. The protection scope of the present disclosure should be interpreted by the attached claims, and all equivalent forms thereof should be interpreted as included in the scope of the present disclosure.

[0226] According to an embodiment of the present disclosure, the vehicle control apparatus may merge a virtual frame generated by using LiDAR, or may unmerge a merged frame.

[0227] Furthermore, according to an embodiment of the present disclosure, the vehicle control apparatus may reduce a load on a processor by determining whether to merge virtual frames before merging the virtual frames.

[0228] In addition, according to an embodiment of the present disclosure, the vehicle control device can determine whether to merge the virtual frame by using the size of the virtual frame and the size of the tracking frame, or determine whether to cancel the merge by using the size of the virtual frame and the size of the merged box, and can output the merged box to output the correct merged box, and a method thereof.

[0229] Furthermore, various effects understood directly or indirectly through the specification can be provided.

[0230] Although the present disclosure has been described above with reference to the exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto but may be variously modified and altered by those skilled in the art without departing from the spirit and scope of the present disclosure as claimed in the appended claims.

Claims

1. A device for controlling a vehicle, the device comprising: sensor; as well as processor, The processor is configured to: generating a tracking frame by converting a virtual frame corresponding to the object at a first time point into a virtual frame at a second time point after the first time point based on obtaining the virtual frame corresponding to the object at a first time point through the sensor; determining, based on obtaining a plurality of virtual frames at the second time point, a virtual frame associated with the tracking frame among the plurality of virtual frames at the second time point, wherein the plurality of virtual frames at the second time point are formed by a plurality of points obtained by the sensor at the second time point; Merging all or part of the determined virtual frames based on at least one of the following conditions: the distances between the plurality of virtual frames at the second time point, whether at least one of the plurality of virtual frames at the second time point corresponds to an object separated from a road edge, and whether the plurality of virtual frames at the second time point are included in the tracking frame; and The merging is controlled based on at least one of: the size of the merge box generated by the merge, a difference between the size of the merged frame and the size of the virtual frame based on the determination, and the type of object; and Outputs a signal indicating the controlled merge.

2. The apparatus of claim 1 , wherein the processor is configured to: Determining the virtual frame associated with the tracking frame in the plurality of virtual frames at the second time point is further based on: The tracking frame overlaps with each of the plurality of virtual frames at a specified ratio or greater at the second time point, or A distance between a plurality of first points included in the tracking frame and a plurality of second points included in the plurality of virtual frames at the second time point is within a relevant distance.

3. The apparatus of claim 1 , wherein the processor is configured to: based on: said object is not obstructed, The type of the object is determined to be a third type that is different from the first type including a special-purpose vehicle, and The number of the plurality of virtual frames at the second time point is less than a specified number to determine the stationary state or the moving state of the object.

4. The apparatus of claim 3, wherein the processor is configured to: determining that the object is at rest; Determining that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length; based on: the object is at rest, and At least one of a width or a length of the tracking frame is greater than or equal to the first reference length, to determine a first straight line, in the plurality of virtual frames at the second time point, which is perpendicular to a line segment forming the first virtual frame from a first center point of the first virtual frame; as well as Based on the distance between the first straight line and a second center point of a second virtual frame among the multiple virtual frames at the second time point being greater than or equal to a second reference length, merging at least a portion of the determined virtual frames is skipped.

5. The apparatus of claim 1 , wherein the processor is configured to: determining, at the second time point, from among the plurality of virtual frames, a virtual frame having a first identifier, the first identifier indicating that the virtual frame having the first identifier is closest to a vehicle including the device and has a width and a length, each of the width and the length being greater than or equal to a third reference length; determining, from among the plurality of virtual frames at the second time point, a virtual frame having a second identifier indicating an object separated from the road edge; as well as Merging the virtual frame having the first identifier and the virtual frame having the second identifier is skipped.

6. The apparatus of claim 1, wherein the processor is configured to: determining a virtual frame that does not overlap with the tracking frame from among the plurality of virtual frames at the second time point; and A portion of the determined virtual frame is merged, the portion excluding the virtual frame that does not overlap with the tracking frame.

7. The apparatus of claim 1, wherein the processor is configured to: determining at least one of a width of each of the determined virtual frames or a length of each of the determined virtual frames; comparing the sum of the determined widths of the virtual frames with the width of the tracking frame, or comparing the sum of the determined lengths of the virtual frames with the length of the tracking frame; and Based on the sum of the widths of the determined virtual frames exceeding the width of the tracking frame, or the sum of the lengths of the determined virtual frames exceeding the length of the tracking frame, merging the determined virtual frames is skipped.

8. The apparatus of claim 1, wherein the processor is configured to: At least a portion of the merging is cancelled based on the size of the merging frame being greater than or equal to a first reference size, or one of a width and a length of the merging frame being greater than or equal to a fourth reference length.

9. The apparatus of claim 1, wherein the processor is configured to: determining a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in the determined virtual frame determined on a plane formed by an x-axis and a y-axis, wherein the x-axis is a moving direction of the vehicle and the y-axis is perpendicular to the x-axis; and Based on the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis and the maximum value on the y-axis being greater than or equal to a second reference size, at least a portion of the merge is canceled.

10. The device of claim 1, wherein the processor is configured to: determining that the object is in a moving state among a plurality of states including a stationary state and a moving state, and determining that the merge frame is located in an area exceeding a specified distance from a boundary of a field of view of the sensor; as well as At least a portion of the merge is cancelled based on one of the following: When the type of the object is a first type including a special vehicle, the width of the merged frame is greater than or equal to a fifth reference length, or the length of the merged frame is greater than or equal to a sixth reference length; When the type of the object is a second type including a passenger car, the width of the merged frame is greater than or equal to the fifth reference length, and the length of the merged frame is greater than or equal to the sixth reference length; When the type of the object is a third type including a pedestrian, the width or the length of the merged frame is greater than or equal to a seventh reference length; as well as When the type of the object is not determined, the width or the length of the merged frame is greater than or equal to the sixth reference length.

11. A method for controlling a vehicle, the method comprising the steps of: generating, by a device and based on obtaining a virtual frame corresponding to the object at a first time point through a sensor, a tracking frame by converting the virtual frame into a virtual frame at a second time point after the first time point; determining, based on obtaining a plurality of virtual frames at the second time point, a virtual frame associated with the tracking frame among the plurality of virtual frames at the second time point, wherein the plurality of virtual frames at the second time point are formed by a plurality of points obtained by the sensor at the second time point; Merging all or part of the determined virtual frames based on at least one of the following: the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from a road edge, and whether the virtual frame at the second time point is included in the tracking frame; and The merging is controlled based on at least one of: the size of the merge box generated by the merge, a difference between the size of the merged frame and a size of a virtual frame based on the determination, and the type of object; and Outputs a signal indicating the controlled merge.

12. The method according to claim 11, wherein determining the virtual frame associated with the tracking frame among the plurality of virtual frames at the second time point is further based on: The tracking frame overlaps with each of the plurality of virtual frames at a specified ratio or greater at the second time point, or a distance between a plurality of first points included in the tracking frame and a plurality of second points included in the plurality of virtual frames at the second time point is within a relevant distance, To determine the virtual frame that is associated with the tracking frame among the multiple virtual frames at the second time point.

13. The method according to claim 11, further comprising: based on: said object is not obstructed, The type of the object is determined to be a first type including a pedestrian, and The number of the plurality of virtual frames at the second time point is less than a specified number to determine the stationary state or the moving state of the object.

14. The method according to claim 13, further comprising: determining that the object is at rest; Determining that at least one of a width or a length of the tracking frame is greater than or equal to a first reference length; based on: the object is at rest, and At least one of a width or a length of the tracking frame is greater than or equal to the first reference length, to determine a first straight line in the plurality of virtual frames at the second time point that is perpendicular to a line segment that forms the first virtual frame from a first center point of the first virtual frame, and Based on the distance between the first straight line and a second center point of a second virtual frame among the multiple virtual frames at the second time point being greater than or equal to a second reference length, merging at least a portion of the determined virtual frames is skipped.

15. The method according to claim 11, further comprising: determining, at the second time point, from among the plurality of virtual frames, a virtual frame having a first identifier, the first identifier indicating that the virtual frame having the first identifier is closest to a vehicle having a width and a length, each of the width and the length being greater than or equal to a third reference length; determining, from among the plurality of virtual frames at the second time point, a virtual frame having a second identifier indicating an object separated from the road edge; as well as The virtual frame having the first identifier and the virtual frame having the second identifier are merged.

16. The method according to claim 11, further comprising: determining, at the second time point, a virtual frame from among the plurality of virtual frames that does not overlap with the tracking frame; as well as A portion of the determined virtual frame is merged, the portion excluding the virtual frame that does not overlap with the tracking frame.

17. The method according to claim 11, further comprising: determining at least one of a width of each of the determined virtual frames or a length of each of the determined virtual frames; Comparing the sum of the widths of the determined virtual frames with the width of the tracking frame, or comparing the sum of the lengths of the determined virtual frames with the length of the tracking frame; as well as The determined virtual frames are merged based on the sum of the widths of the determined virtual frames exceeding the width of the tracking frame, or the sum of the lengths of the determined virtual frames exceeding the length of the tracking frame.

18. The method according to claim 11, further comprising: At least a portion of the merging is cancelled based on the size of the merging frame being greater than or equal to a first reference size, or one of a width and a length of the merging frame being greater than or equal to a fourth reference length.

19. The method according to claim 11, further comprising: determining a minimum value on the x-axis, a maximum value on the x-axis, a minimum value on the y-axis, and a maximum value on the y-axis among a plurality of points included in the determined virtual frame determined on a plane formed by an x-axis and a y-axis, wherein the x-axis is a moving direction of the vehicle and the y-axis is perpendicular to the x-axis; and Based on the difference between the size of the merged box and the size of a quadrilateral formed by the minimum value on the x-axis, the maximum value on the x-axis, the minimum value on the y-axis and the maximum value on the y-axis being greater than or equal to a second reference size, at least a portion of the merge is canceled.

20. The method according to claim 11, further comprising: determining that the object is in a moving state among a plurality of states including a stationary state and a moving state, and determining that the merge frame is located in an area exceeding a specified distance from a boundary of a field of view of the sensor; as well as At least a portion of the merge is cancelled based on one of the following: When the type of the object is a first type including a special vehicle, the width of the merged frame is greater than or equal to a fifth reference length, or the length of the merged frame is greater than or equal to a sixth reference length; When the type of the object is a second type including a passenger car, the width of the merged frame is greater than or equal to the fifth reference length, and the length of the merged frame is greater than or equal to the sixth reference length; When the type of the object is a third type including a pedestrian, the width or the length of the merged frame is greater than or equal to a seventh reference length; and When the type of the object is not determined, the width or the length of the merged frame is greater than or equal to the sixth reference length.

Images