Vehicle control apparatus and method thereof

Abstract

The invention relates to a vehicle control apparatus and a method thereof. A vehicle control device may include a sensor and a processor. The processor may convert a first virtual frame associated with the first object and obtained via the sensor at a first time into a second virtual frame associated with a second time later than the first time, determine a plurality of third virtual frames associated with the second virtual frame based on a plurality of data points obtained via the sensor at the second time, merging at least a portion of the plurality of third virtual frames into the merged frame based on at least one of a distance between two or more of the plurality of third virtual frames at a second time, or whether at least one of the plurality of third virtual frames at the second time corresponds to a second object separated from the road boundary, the merge is maintained or canceled based on at least one of: a distance between two or more data points included in the merge box, or a type of the first object, and a signal indicating a result of the maintenance or cancellation of the merge is output.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority from Korean Patent Application No. 10-2023-0120571 filed on September 11, 2023, in the Korean Intellectual Property Office, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to a vehicle control device and method thereof, and more particularly, to a technology for using light detection and ranging (LiDAR). Background Art

[0004] Various studies are being conducted to assist driving of a host vehicle by recognizing external objects relative to the host vehicle using various sensors.

[0005] Specifically, when the host vehicle is driven in a driving assist device activation mode or an autonomous driving mode, external objects may be recognized by using LiDAR.

[0006] In the process of merging virtual frames generated using LiDAR, two or more virtual frames may be incorrectly merged. When the merged frame obtained by incorrectly merging virtual frames is mapped to an external object, the type, state, or size of the external object may be incorrectly identified. When the type, state, or size of the external object is incorrectly identified, the driving route of the host vehicle may be unexpectedly and unnecessarily changed. Summary of the invention

[0007] The present disclosure is directed to solving the above-mentioned problems occurring in the prior art while keeping the advantages achieved by the prior art intact.

[0008] An aspect of the present disclosure provides a vehicle control apparatus and method thereof which merges a virtual frame generated by using a sensor (eg, LiDAR) or unmerges a merged frame.

[0009] An aspect of the present disclosure provides a vehicle control apparatus and method thereof which reduce a load of a processor by determining whether to merge virtual frames before merging the virtual frames.

[0010] An aspect of the present disclosure provides a vehicle control device and method thereof that accurately recognizes a pedestrian when recognizing that the pedestrian is separated from a road boundary (eg, a guardrail, a center line, a curb, etc.).

[0011] The technical problems to be solved by the present disclosure are not limited to the above-mentioned problems, and any other technical problems not mentioned herein will be clearly understood by those skilled in the art to which the present disclosure belongs from the following description.

[0012] According to one or more exemplary embodiments of the present disclosure, a vehicle control device may include: a sensor and a processor. The processor may be configured to: convert a first virtual frame associated with a first object and obtained via a sensor at a first time into a second virtual frame associated with a second time later than the first time; determine a plurality of third virtual frames associated with the second virtual frame based on a plurality of data points obtained via the sensor at the second time; merge at least a portion of the plurality of third virtual frames into a merged frame based on at least one of the following: the distance between two or more of the plurality of third virtual frames at the second time or whether at least one of the plurality of third virtual frames at the second time corresponds to a second object separated from a road boundary; maintain or cancel the merging of at least a portion of the plurality of third virtual frames based on at least one of the following: the distance between two or more of the data points included in the merged frame or the type of the first object; and output a signal indicating the result of maintaining or canceling the merge.

[0013] The processor may be configured to determine the plurality of third virtual frames by determining the plurality of third virtual frames based on at least one of the following: the second virtual frame overlaps each of the plurality of third virtual frames at the second time by a ratio greater than a first threshold, or a distance between a plurality of first data points included in the second virtual frame and a plurality of second data points included in the plurality of third virtual frames at the second time is within a second threshold.

[0014] The processor may be further configured to determine whether the first object is stationary or moving based on at least one of the following: the first object is not blocked, the type of the first object is a pedestrian, or the number of the plurality of third virtual frames at the second time is less than a threshold.

[0015] At least a portion of the plurality of third virtual frames may include a fourth virtual frame and a fifth virtual frame. The processor may be configured to maintain or cancel the merging by canceling the merging based on: the first object is stationary, at least one of a width or a length of the second virtual frame is greater than or equal to a first threshold, and a distance between a first straight line perpendicular to a line segment forming the fourth virtual frame and passing through a center of the fourth virtual frame at the second time and a center of the fifth virtual frame at the second time is greater than or equal to a second threshold.

[0016] At least a portion of the plurality of third virtual frames may include a fourth virtual frame and a fifth virtual frame. The processor may be configured to maintain or cancel the merge by canceling the merge based on the following items: the first object is moving; the width and length of each of the plurality of third virtual frames at the second time are less than or equal to a first threshold; and the distance between the center of the fourth virtual frame and the center of the fifth virtual frame is greater than or equal to a second threshold, and the second threshold is greater than the first threshold.

[0017] The processor may be further configured to determine a fourth virtual frame closest to the vehicle associated with the vehicle control device from among the plurality of third virtual frames at the second time. The width and length of the fourth virtual frame may be greater than or equal to a threshold. The processor may also be configured to: determine a fifth virtual frame associated with the second object from among the plurality of third virtual frames at the second time; and cancel the merging of the fourth virtual frame and the fifth virtual frame.

[0018] The processor may be further configured to: determine the fourth virtual frame and the fifth virtual frame that are farthest from each other among the multiple third virtual frames at the second time based on the following items: the first object is not obscured; the type of the first object is not a special vehicle; and one of the width or length of the merged frame is greater than or equal to the first threshold.

[0019] The processor may also be configured to cancel the merging of the fourth virtual frame and the fifth virtual frame based on the distance between a first data point in the fourth virtual frame that is closest to the fifth virtual frame and a second data point in the fifth virtual frame that is closest to the fourth virtual frame being greater than a second threshold, where the second threshold is less than the first threshold.

[0020] The processor may be further configured to determine the object type associated with the merged box based on the following: the first object is moving; the width of the merged box is less than or equal to a first threshold; the length of the merged box is less than or equal to a second threshold, the second threshold is greater than the first threshold; and the merged box is associated with an area that is at least a threshold distance from a boundary of the sensor's field of view.

[0021] The processor may be configured to maintain or cancel the merge by canceling the merge based on: the type of the first object is a pedestrian, and one of a width of the merged box or a length of the merged box is greater than or equal to a third threshold that is less than the first threshold.

[0022] According to one or more exemplary embodiments of the present disclosure, a vehicle control method may include: converting, by one or more processors of the vehicle, a first virtual frame associated with a first object and obtained via a sensor at a first time into a second virtual frame associated with a second time later than the first time; determining a plurality of third virtual frames associated with the second virtual frame based on a plurality of data points obtained via the sensor at the second time; merging at least a portion of the plurality of third virtual frames into a merged frame based on at least one of the following: a distance between two or more of the plurality of third virtual frames at the second time, or whether at least one of the plurality of third virtual frames at the second time corresponds to a second object separated from a road boundary; and maintaining or canceling the merging of at least a portion of the plurality of third virtual frames based on at least one of the following: a distance between two or more data points included in the merged frame, or a type of the first object; and outputting a signal indicating the result of maintaining or canceling the merging.

[0023] Determining the plurality of third virtual frames may include determining the plurality of third virtual frames based on at least one of the following: the second virtual frame overlaps with each of the plurality of third virtual frames at the second time by a ratio greater than a first threshold, or a distance between a plurality of first data points included in the second virtual frame and a plurality of second data points included in the plurality of third virtual frames at the second time is within a second threshold.

[0024] The method may further include determining whether the first object is stationary or moving based on at least one of the following: the first object is not blocked; the type of the first object is a pedestrian; or the number of the plurality of third virtual frames at the second time is less than a threshold.

[0025] At least a portion of the plurality of third virtual frames may include a fourth virtual frame and a fifth virtual frame. Maintaining or cancelling the merge may include cancelling the merge based on: the first object is stationary; at least one of a width or a length of the second virtual frame is greater than or equal to a first threshold; and a distance between a first straight line and a center of the fifth virtual frame at the second time is greater than or equal to a second threshold, the first straight line being perpendicular to a line segment forming the fourth virtual frame and passing through the center of the fourth virtual frame.

[0026] At least a portion of the plurality of third virtual frames may include a fourth virtual frame and a fifth virtual frame. Maintaining or canceling the merge may include canceling the merge based on the following items: the first object is moving; the width and length of the plurality of third virtual frames at the second time are less than or equal to a first threshold; and the distance between the center of the fourth virtual frame and the center of the fifth virtual frame is greater than or equal to a second threshold greater than the first threshold.

[0027] The method may further include determining a fourth virtual frame closest to the vehicle from among the plurality of third virtual frames at the second time. The width and length of the fourth virtual frame may be greater than or equal to a threshold. The method may also include: determining a fifth virtual frame associated with the second object from among the plurality of third virtual frames at the second time; and canceling the merging of the fourth virtual frame and the fifth virtual frame.

[0028] The method may further include: determining a fourth virtual frame and a fifth virtual frame that are farthest from each other among multiple third virtual frames at the second time based on the following items: the first object is not blocked; the type of the first object is not a specific vehicle; and one of the width or length of the merged frame is greater than or equal to the first threshold.

[0029] The method may further include: canceling the merging of the fourth virtual frame and the fifth virtual frame based on the distance between the first data point in the fourth virtual frame that is closest to the fifth virtual frame and the second data point in the fifth virtual frame that is closest to the fourth virtual frame is greater than a second threshold, and the second threshold is less than the first threshold.

[0030] The method may further include determining the object type associated with the merged box based on the following items: the first object is moving; the width of the merged box is less than or equal to a first threshold; the length of the merged box is less than or equal to a second threshold, the second threshold is greater than the first threshold; and the merged box is associated with an area that is at least a threshold distance from a boundary of the sensor's field of view.

[0031] Maintaining or cancelling the merging may include cancelling the merging based on: the type of the first object is a pedestrian; and one of a width of the merging box or a length of the merging box is greater than or equal to a third threshold value that is smaller than the first threshold value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The above and other objects, features and advantages of the present disclosure will become more apparent from the following detailed description in conjunction with the accompanying drawings:

[0033] Figure 1 An example of a block diagram of a vehicle control device according to an embodiment of the present disclosure is shown;

[0034] Figure 2 An example of generating a merged frame by using a virtual frame in the embodiment of the present disclosure is shown;

[0035] Figure 3 An embodiment of identifying distances between virtual frames is shown;

[0036] Figure 4 An embodiment in which an object separated from a road edge at a second time point and a virtual frame are not merged is shown;

[0037] Figure 5 An embodiment of separating a merged frame based on a plurality of points included in a virtual frame is shown;

[0038] Figure 6 An embodiment of the result of separating the merge boxes is shown.

[0039] Figure 7 An embodiment of a flow chart illustrating a vehicle control method; and

[0040] Figure 8 A computing system associated with a vehicle control device is shown. DETAILED DESCRIPTION

[0041] Hereinafter, one or more exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. When adding reference numerals to components of the respective drawings, it should be noted that the same components have the same reference numerals even though they are shown in another drawing. In addition, when describing one or more exemplary embodiments of the present disclosure, detailed descriptions associated with well-known functions or configurations will be omitted when they may unnecessarily obscure the subject matter of the present disclosure.

[0042] When describing the elements of the embodiment of the present disclosure, the terms first, second, A, B, (a), (b) etc. can be used in this article. These terms are only used to distinguish one element from another element, but do not limit the corresponding elements, and are independent of the properties, order or priority of the corresponding elements. In addition, unless otherwise defined, all terms including technical terms and scientific terms used herein will be interpreted as the customary terms of the field to which the present disclosure belongs. It should be understood that the terms used in this article should be interpreted as having the same meaning as their meaning in the context of the present disclosure and the related art, and should not be interpreted in an idealized or overly formal sense, unless clearly defined in this article.

[0043] In the following, reference will be made to Figures 1 to 8 One or more exemplary embodiments of the present disclosure are described in detail.

[0044] Figure 1 An embodiment of a block diagram of a vehicle control device is shown.

[0045] Reference Figure 1 The vehicle control device 100 may be implemented inside or outside the vehicle, and some components included in the vehicle control device 100 may be implemented inside or outside the vehicle. In this case, the vehicle control device 100 may be integrated with the internal control unit of the vehicle, and may be implemented by a separate device so as to be connected to the control unit of the vehicle by means of a separate connection device. For example, the vehicle control device 100 may further include Figure 1 Components not shown.

[0046] Reference Figure 1 , the vehicle control device 100 may include a processor 110 and a LiDAR 120. The processor 110 or the LiDAR 120 may be electrically or operably connected to each other through electronic components including a communication bus.

[0047] Hereinafter, the fact that hardware is operably coupled may include the fact that a direct or indirect connection between hardware is established wired or wirelessly so that the second hardware is controlled by the first hardware in the hardware. Although different blocks are shown, the present disclosure is not limited thereto.

[0048] Figure 1 Some of the hardware in the vehicle control device 100 may be included in a single integrated circuit including a system on a chip (SoC). The type or number of hardware included in the vehicle control device 100 is not limited to Figure 1 For example, the vehicle control device 100 may include Figure 1 Only some of the hardware is shown.

[0049] The vehicle control device 100 may include hardware for processing data based on one or more instructions. The hardware for processing data may include a processor 110 .

[0050] For example, the hardware for processing data may include an arithmetic and logic unit (ALU), a floating point unit (FPU), a field programmable gate array (FPGA), a central processing unit (CPU), and / or an application processor (AP). The processor 110 may have a structure of a single-core processor, or may have a structure of a multi-core processor including a dual-core, quad-core, hexa-core, or octa-core.

[0051] The LiDAR 120 included in the vehicle control device 100 may obtain a data set from identifying objects around the vehicle control device 100. For example, the LiDAR 120 may identify at least one of the position of the surrounding object, the moving direction of the surrounding object, the speed of the surrounding object, or any combination thereof based on a pulse laser signal emitted from the LiDAR 120, reflected by the surrounding object, and returned.

[0052] The processor 110 included in the vehicle control device 100 may obtain a virtual frame corresponding to an object specified at the first time point through the LiDAR 120. For example, the processor 110 may obtain the virtual frame at the first time point by using a plurality of points obtained at the first time point through the LiDAR 120.

[0053] The processor 110 may generate a tracking frame obtained by converting a virtual frame at a first time point into a virtual frame at a second time point after the first time point.

[0054] For example, at least one of the first time point, the second time point, or any combination thereof may include a time point for obtaining a data set by LiDAR 120. For example, the second time point may be referred to as the current time point (e.g., time point "t"). For example, the second time point may include a time point at which a current frame is obtained. For example, the first time point may include a time point immediately before the current frame is obtained. For example, the first time point may include a time point "t-1". The expression time point is at least one of the first time point, the second time point, or any combination thereof. However, at least one of the first time point, the second time point, or any combination thereof may mean a frame number. For example, when the second time point means the tth frame, the first time point may mean the (t-1)th frame.

[0055] The processor 110 may obtain a plurality of points at the second time point through the LiDAR 120. For example, the processor 110 may obtain a virtual frame at the second time point, the virtual frame being formed by the plurality of points obtained at the second time point through the LiDAR 120. For example, the virtual frame at the second time point may be referred to as a “meta frame”. For example, the virtual frame at the second time point may include a frame formed based on points obtained as light emitted from the LiDAR 120 is reflected from an external object.

[0056] The processor 110 may identify a virtual frame associated with the tracking frame among the virtual frames at the second time point based on obtaining the virtual frame at the second time point.

[0057] For example, the processor 110 may identify a ratio (e.g., a proportion) at which the tracking frame overlaps with each virtual frame in the virtual frame obtained at the second time point. For example, the processor 110 may identify that the tracking frame overlaps with each virtual frame in the virtual frame obtained at the second time point at a specified ratio (e.g., about 25%) or more.

[0058] For example, the processor 110 may identify the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point. For example, the processor 110 may determine whether the distance between the plurality of first points included in the tracking frame and the plurality of second points included in the virtual frame at the second time point is within the relevant distance. For example, the relevant distance may be referred to as a "Mahalanobis distance".

[0059] The processor 110 can identify a virtual box associated with the tracking box among the virtual boxes at the second time point based on the fact that the tracking box overlaps with each virtual box obtained at the second time point at a specified ratio (e.g., a proportion) or more, or the distances between multiple first points included in the tracking box and multiple second points included in the virtual box at the second time point are identified as being within the relevant distances.

[0060] For example, the processor 110 may identify a virtual frame associated with the tracking frame by identifying that the tracking frame overlaps with each virtual frame in the virtual frame obtained at the second time point at a specified ratio (e.g., a proportion) or more, or that the distances between multiple first points included in the tracking frame and multiple second points included in the virtual frame at the second time point are within a relevant distance.

[0061] The processor 110 may merge all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from a road boundary (e.g., an edge), or any combination thereof.

[0062] For example, when the distance between the virtual frames at the second time point is long, the processor 110 may not merge the virtual frames at the second time point. The operation of not merging the virtual frames at the second time point will be described later.

[0063] The processor 110 may determine whether to merge the virtual frames at the second time point based on at least one of whether the designated object is blocked, the type of the designated object, the number of virtual frames at the second time point, or any combination thereof.

[0064] For example, the processor 110 may determine whether the designated object is in a moving state based on the following facts: the designated object is not blocked, the type of the designated object is identified as a first designated type including pedestrians, and the number of virtual frames at the second time point is less than a designated number (e.g., 3). For example, the processor 110 may identify the stationary or moving state of the designated object based on the following facts: the designated object is not blocked, the type of the designated object is identified as a first designated type including pedestrians, and the number of virtual frames at the second time point is less than a designated number.

[0065] The processor 110 may recognize that the designated object is in a stationary state. When the designated object is in a stationary state, the processor 110 may perform the following operations.

[0066] For example, the processor 110 may identify the width or length of the tracking frame. The processor 110 may identify that at least one of the width or length of the tracking frame is greater than or equal to a first reference length (eg, approximately 30 m).

[0067] Based on the fact that the designated object is in a stationary state and at least one of the width or the length of the tracking frame is greater than or equal to the first reference length, the processor 110 can identify, from the virtual frame at the second time point, a first straight line that is perpendicular to the first center point of the first virtual frame and the line segments forming the first virtual frame.

[0068] For example, by projecting multiple points obtained by LiDAR 120 onto a plane formed by the x-axis and the y-axis among the x-axis, the y-axis, and the z-axis, the processor 110 can identify a first straight line that is perpendicular to the first center point of the first virtual frame and the line segment forming the first virtual frame from the virtual frame at the second time point.

[0069] The processor 110 may identify a distance between the first straight line and a second center point of the second virtual frame in the virtual frame at the second time point. The processor 110 may not merge the associated virtual frame based on the fact that the distance between the first straight line and the second center point of the second virtual frame in the virtual frame at the second time point is greater than or equal to a second reference length (e.g., approximately 20 m). For example, the processor 110 may exclude such a virtual frame from the merge candidate, that is, the virtual frame has a distance between the first straight line and the second center point of the second virtual frame in the virtual frame at the second time point greater than or equal to the second reference length (e.g., approximately 20 m). Hereinafter, the operation of not merging the associated virtual frame may include an operation of excluding the associated virtual frame from the merge candidate.

[0070] The processor 110 may recognize that the designated object is in a moving state. For example, when the designated object is in a moving state, the processor 110 may perform the following operations.

[0071] The processor 110 may identify at least one of the width of each virtual frame in the virtual frame at the second time point, the length of each virtual frame in the virtual frame at the second time point, or any combination thereof. For example, the processor 110 may identify a first center point of a first virtual frame in the virtual frame at the second time point and a second center point of a second virtual frame in the virtual frame at the second time point based on the specified object being in a moving state and the width and length of each virtual frame at the second time point being less than or equal to a third reference length (e.g., approximately 1 m).

[0072] The processor 110 may identify a distance between a first center point of a first virtual frame among virtual frames at a second time point and a second center point of a second virtual frame among virtual frames at the second time point. The processor 110 may not merge the associated virtual frames based on the fact that a distance between a first center point of a first virtual frame among virtual frames at the second time point and a second center point of a second virtual frame among virtual frames at the second time point is greater than or equal to a fourth reference length (e.g., approximately 1.5 m).

[0073] Hereinafter, an embodiment of merging or not merging a virtual frame at a second time point regardless of whether a designated object is in a moving state will be described.

[0074] The processor 110 may identify, from the virtual frame at the second time point, a virtual frame that is closest to the host vehicle including the vehicle control device 100 and has a width and length greater than or equal to a fourth reference length (e.g., approximately 1 m). The processor 110 may identify, from the virtual frame at the second time point, a virtual frame having an assigned first identifier indicating that the virtual frame is closest to the host vehicle including the vehicle control device 100 and has a width and length greater than or equal to the fourth reference length (e.g., approximately 1 m). For example, the first identifier may be referred to as a primary index.

[0075] The processor 110 may identify a virtual frame having a second identifier assigned thereto from among the virtual frames at the second time point, the second identifier indicating an object separated from the road boundary. For example, the second identifier may be called "ContourSplitFlag".

[0076] The processor 110 may not merge the virtual frame to which the first identifier is assigned and the virtual frame to which the second identifier is assigned. For example, when attempting to merge the virtual frame to which the first identifier is assigned and the virtual frame to which the second identifier is assigned, the processor 110 may not merge the virtual frame to which the first identifier is assigned and the virtual frame to which the second identifier is assigned. For example, the processor 110 may exclude the virtual frame to which the second identifier is assigned from the merge candidates.

[0077] Hereinafter, an embodiment of unmerging a merged frame generated by merging virtual frames at a second time point will be described.

[0078] The processor 110 may merge all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether at least one of the virtual frames at the second time point corresponds to an object separated from the road boundary, or any combination thereof. The processor 110 may generate a merged frame by merging. The processor 110 may maintain or cancel the merge based on at least one of the distance between the multiple points included in the merged frame generated by merging, the type of the specified object, or any combination thereof.

[0079] The processor 110 may identify whether the designated object is occluded. The processor 110 may identify the type of the designated object. The processor 110 may identify at least one of the width of the merged box, the length of the merged box, or any combination thereof.

[0080] The processor 110 may identify that the designated object is not obscured, the type of the designated object is identified as a type different from a second designated type indicating a specific vehicle, and a width or length of the merged frame is greater than or equal to a fifth reference length (eg, approximately 30 m).

[0081] For example, the specific vehicle may include at least one of a 1-ton truck, a container truck, a heavy truck, or any combination thereof. For example, the specific vehicle may include a vehicle having a size larger than a passenger vehicle.

[0082] The processor 110 can identify the distance between the virtual frames at the second time point included in the merged frame based on the following facts: the specified object is not obscured, the type of the specified object is identified as a type different from the second specified type indicating a specific vehicle, and the width or length of the merged frame is greater than or equal to the fifth reference length.

[0083] For example, the processor 110 may identify two virtual frames each having the farthest distance from the virtual frame at the second time point included in the merged frame. The processor 110 may identify the distance between the virtual frames at the second time point included in the merged frame based on the points included in the two virtual frames each having the farthest distance in the virtual frame at the second time point included in the merged frame. For example, the two virtual frames having the farthest distance in the virtual frame at the second time point included in the merged frame may be respectively referred to as a "third virtual frame" and a "fourth virtual frame".

[0084] For example, the processor 110 may identify a point included in the third virtual frame. For example, the processor 110 may identify a point included in the fourth virtual frame. The processor 110 may identify two closest points among the points included in the third virtual frame and the points included in the fourth virtual frame. For example, the two closest points among the points included in the third virtual frame and the points included in the fourth virtual frame may be referred to as a “first point” and a “second point”, respectively.

[0085] The processor 110 may identify the distance between the first point and the second point. The processor 110 may identify that the distance between the first point and the second point exceeds a sixth reference length (e.g., approximately 20 m). The processor 110 may cancel the merge based on the distance between the first point and the second point exceeding the sixth reference length. For example, the sixth reference length may be less than the fifth reference length.

[0086] The processor 110 may cancel the merging based on at least one of whether the specified object is in a stationary state or a moving state, the type of the specified object, or any combination thereof.

[0087] For example, the processor 110 may identify that the designated object is in a mobile state among a stationary state and a mobile state. The processor 110 may identify the type of the object corresponding to the merged frame based on the fact that the designated object is identified as being in a mobile state among a stationary state and a mobile state, the width of the merged frame is less than or equal to a seventh reference length (e.g., about 4.5 m), the length of the merged frame is an eighth reference length (e.g., about 20 m) exceeding the seventh reference length, and the merged frame is identified in an area exceeding a specified distance from a boundary of a field of view (FOV) of the LiDAR 120. The area within a specified distance from the boundary of the FOV of the LiDAR 120 may be referred to as a "FOV boundary area".

[0088] The processor 110 may identify the width and length of the merged frame based on identifying that the type of the designated object is the first designated type including a pedestrian.

[0089] The processor 110 may unmerge the merged frame based on recognizing that at least one of the width of the merged frame, the length of the merged frame, or any combination thereof is greater than or equal to a ninth reference length (e.g., about 1.5 m) that is less than the seventh reference length. In detail, the processor 110 may unmerge the merged frame based on recognizing that one of the width of the merged frame or the length of the merged frame is greater than or equal to a ninth reference length that is less than the seventh reference length.

[0090] The processor 110 may obtain a merged frame based on all or part of the associated virtual frame. The processor 110 may map the merged frame or the virtual frame at the second time point to the external object. The processor 110 may identify at least one of the forward direction of the external object, the speed of the external object, the type of the external object, or any combination thereof based on the mapping of the merged frame or the virtual frame at the second time point to the external object.

[0091] As described above, the processor 110 of the vehicle control device 100 can accurately map the virtual frame to the external object by merging the virtual frame at the second time point or unmerging the merged frame. The processor 110 can assist the operation of the host vehicle including the vehicle control device 100 by accurately mapping the virtual frame to the external object.

[0092] Figure 2 An embodiment is shown in which a merged frame is generated by using a virtual frame.

[0093] Figure 2 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 2 The operations can be included in Figure 1 The processor 110 in the vehicle control device 100 executes.

[0094] refer to Figure 2In the first embodiment 201, the processor included in the vehicle control device can obtain a virtual frame 211 corresponding to a specified object at the first time point through LiDAR at the first time point. For example, the processor can obtain the virtual frame 211 by using a plurality of points obtained at the time point "t-1".

[0095] refer to Figure 2 In the second embodiment 203, the processor may generate a tracking frame 213 obtained by converting a virtual frame 211 at a first time point into a virtual frame at a second time point after the first time point. For example, the second time point may be referred to as a current frame.

[0096] refer to Figure 2 In the third embodiment 205 , the processor may obtain virtual frames 221 , 223 , 225 , and 227 at the second time point formed by a plurality of points obtained by LiDAR at the second time point.

[0097] The processor may identify the tracking frame 213 and an associated virtual frame among the virtual frames 221 , 223 , 225 , and 227 at the second time point based on obtaining the virtual frames 221 , 223 , 225 , and 227 at the second time point formed by the plurality of points obtained at the second time point.

[0098] For example, the processor may determine whether the tracking box 213 is associated with the virtual boxes 221, 223, 225, and 227 at the second time point based on at least one of the tracking point, the output point, the center point, the ratio (e.g., proportion) at which the tracking box 213 overlaps with the virtual boxes 221, 223, 225, and 227 at the second time point, or any combination thereof.

[0099] For example, the processor may determine whether the tracking frame 213 is associated with the virtual frames 221 , 223 , 225 , and 227 at the second time point based on the correlation distances between the tracking frame 213 and the virtual frames 221 , 223 , 225 , and 227 at the second time point.

[0100] For example, the processor may determine whether the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on the correlation distance obtained according to the Mahalanobis distance. For example, when the Mahalanobis distance is less than or equal to the reference distance (e.g., about 0.5 m), the processor may determine that the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point.

[0101] For example, the processor may determine that the tracking frame 213 is associated with the virtual frames 221, 223, 225, and 227 at the second time point based on the fact that the tracking frame 213 overlaps the virtual frames 221, 223, 225, and 227 at the second time point by more than a reference ratio (e.g., approximately 25%).

[0102] refer to Figure 2 In the fourth embodiment 207 , the processor may generate a merged frame 230 based on the associated virtual frames 221 , 223 , 225 , and 227 .

[0103] For example, the processor may merge the tracking box 213 and the associated virtual boxes 221, 223, 225, and 227. For example, the processor may merge the tracking box 213 and the associated virtual boxes 221, 223, 225, and 227 in whole or in part.

[0104] The processor may generate a merged frame 230 based on merging all or part of the associated virtual frames 221 , 223 , 225 , and 227 .

[0105] Hereinafter, a vehicle control device for generating the merging frame 230 will be described later.

[0106] Figure 3 An embodiment of identifying distances between virtual frames is shown.

[0107] Figure 3 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 3 The operations can be included in Figure 1 The processor 110 in the vehicle control device 100 executes.

[0108] Figure 3 Each of the first embodiment 301 and the second embodiment 303 in FIG. 3 shows an embodiment of merging all or part of a virtual frame associated with a tracking frame based on a distance between virtual frames at a second time point.

[0109] Referring to the first embodiment 301, the processor included in the vehicle control device may identify virtual frames 311 and 313 associated with the tracking frame 310 in the virtual frame at the second time point. For example, when the designated object corresponding to the tracking frame 310 is identified as being in a stationary state, the processor may perform the operation described in the first embodiment 301.

[0110] The processor 110 may determine whether the designated object is in a stationary state or a moving state based on whether the designated object corresponding to the tracking frame 310 is blocked, the type of the designated object, the number of virtual frames at the second time point, or any combination thereof.

[0111] The processor may identify the stationary or moving state of the specified object based on the following facts: the specified object is not obscured, the type of the specified object is identified as a first specified type including pedestrians, and the number of virtual frames at the second time point is less than a specified number (e.g., approximately 3).

[0112] The processor may recognize that the designated object is stationary, and may recognize that at least one of the width 319 of the tracking frame 310 , the length of the tracking frame 310 , or any combination thereof is greater than or equal to a first reference length (eg, approximately 30 m).

[0113] For example, when the designated object corresponding to the tracking frame 310 is stationary and the width 319 of the tracking frame 310 is greater than or equal to 30 m, and when the distance between the associated virtual frames is greater than or equal to 20 m, the processor may not merge the associated virtual frames.

[0114] For example, the processor may identify a first straight line 315 perpendicular to the line segments forming the first virtual frame 311 from the center point of the first virtual frame 311 based on the fact that the first virtual frame 311 is associated with the second virtual frame 313 .

[0115] The processor may identify a distance 317 between the first straight line 315 and the center point of the second virtual frame 313. The processor may not merge the first virtual frame 311 and the second virtual frame 313 based on identifying that the distance between the first straight line 315 and the center point of the second virtual frame 313 is greater than or equal to a reference distance (e.g., about 20 m).

[0116] The operation of the processor not merging the first virtual frame 311 and the second virtual frame 313 may include an operation of the processor excluding at least one of the first virtual frame 311 , the second virtual frame 313 , or any combination thereof from the merging candidates.

[0117] Referring to the second embodiment 303, the processor can identify that the specified object is not obscured, the type of the specified object is identified as a first specified type including pedestrians, the number of virtual frames at the second time point is less than a specified number (e.g., 3), and the specified object is in a moving state.

[0118] The processor may identify virtual frames 321 and 323 associated with the tracking frame 320. For example, the processor may identify a first virtual frame 321 associated with the tracking frame 320 and a second virtual frame 323 associated with the tracking frame 320.

[0119] For example, the processor may identify the width and length of the first virtual frame 321. The processor may identify the width and length of the second virtual frame 323.

[0120] The processor may recognize that each of the width and length of the first virtual frame 321 is less than or equal to about 1 m. The processor may recognize that each of the width and length of the second virtual frame 323 is less than or equal to about 1 m.

[0121] The processor may recognize the distance between the first virtual frame 321 and the second virtual frame 323 based on the fact that each of the width and the length of the first virtual frame 321 and the second virtual frame 323 is less than or equal to about 1 m.

[0122] For example, the processor may identify a first center point of the first virtual frame 321 and a second center point of the second virtual frame 323. For example, at least one of the first center point of the first virtual frame 321, the second center point of the second virtual frame 323, or any combination thereof may be obtained based on a coordinate system centered on the host vehicle, and using the position of the host vehicle including the vehicle control device as the origin.

[0123] The processor may identify a distance 325 between the first center point of the first virtual frame 321 and the second center point of the second virtual frame 323 based on identifying the first center point of the first virtual frame 321 and the second center point of the second virtual frame 323 .

[0124] The processor may not merge the first virtual frame 321 and the second virtual frame 323 based on recognizing that the distance 325 between the first center point of the first virtual frame 321 and the second center point of the second virtual frame 323 is greater than or equal to about 1.5 m.

[0125] The operation of the processor not merging the first virtual frame 321 and the second virtual frame 323 may include an operation of the processor excluding at least one of the first virtual frame 321 , the second virtual frame 323 , or any combination thereof from the merging candidates.

[0126] Figure 4 An embodiment is shown in which an object separated from a road boundary and a virtual frame at a second time point are not merged.

[0127] Figure 4 The operation can be done by Figure 1 The vehicle control device 100 executes. For example, Figure 4 The operation can be included in Figure 1 The processor 110 in the vehicle control device 100 executes.

[0128] Reference Figure 4 , a processor included in the vehicle control device may generate a tracking frame 410 obtained by converting a virtual frame at a first time point into a virtual frame at a second time point after the first time point.

[0129] The processor may obtain virtual frames 411 , 413 , 415 , 417 , 420 , and 430 at a second point in time, the virtual frames being formed by a plurality of points obtained by LiDAR at the second point in time.

[0130] The processor may identify the virtual frames 411 , 413 , 415 , 417 , and 430 associated with the tracking frame 410 among the virtual frames 411 , 413 , 415 , 417 , 420 , and 430 at the second time point.

[0131] The processor may identify a virtual frame associated with the tracking frame 410 based on at least one of: a ratio (e.g., a proportion) at which the tracking frame 410 overlaps the virtual frame, relative distances between points included in the tracking frame and points included in the virtual frame, or any combination thereof.

[0132] The processor may identify virtual frame 411 from virtual frames 411, 413, 415, 417, and 430 associated with tracking frame 410, the virtual frame having a first identifier indicating that virtual frame 411 is closest to the host vehicle including the vehicle control device and having a width and a length each greater than or equal to a fourth reference length (e.g., approximately 1 m). For example, the first identifier may be referred to as a primary index.

[0133] At least one of the position reference point of the merged frame, the feature of the merged frame, or any combination thereof may be determined by using the virtual frame 411 to which the first identifier is assigned. For example, the processor may determine at least one of the position reference point of the merged frame, the feature of the merged frame, or any combination thereof based on the position reference point of the virtual frame 411 to which the first identifier is assigned and the feature of the virtual frame 411.

[0134] The processor may identify a virtual frame 430 having a second identifier indicating an object separated from the road boundary from the virtual frame at the second time point. For example, the second identifier may be called "ContourSplitFlag".

[0135] For example, in Figure 4 In the example, the virtual frame 420 may include a virtual frame corresponding to a road boundary. Figure 4 , the virtual frame 430 may include a virtual frame corresponding to an object separated from the virtual frame 420 corresponding to the road boundary.

[0136] When attempting to merge the virtual frame 430 to which the second identifier is assigned and the virtual frame 411 to which the first identifier is assigned, the processor may exclude the virtual frame 430 to which the second identifier is assigned from the merge candidates. For example, the processor may not merge the virtual frame 430 to which the second identifier is assigned and the virtual frame 411 to which the first identifier is assigned.

[0137] The processor may not merge the virtual frame 430 to which the second identifier is assigned with the virtual frame 411 to which the first identifier is assigned, and thus may obtain a merged frame based on the virtual frames 411, 413, 415, and 417 associated with the tracking frame. For example, the processor may perform merging while excluding the virtual frame 430 to which the second identifier is assigned from the virtual frames 411, 413, 415, 417, and 430 associated with the tracking frame 410.

[0138] As described above, the processor of the vehicle control device can perform merging while excluding the virtual frame 430 to which the second identifier is assigned from the virtual frames 411, 413, 415, 417 and 430 associated with the tracking frame 410, thereby preventing the shape of the virtual frame 420 corresponding to the road boundary from expanding abnormally.

[0139] Figure 5 An embodiment of separating a merged frame based on a plurality of points included in a virtual frame is shown.

[0140] Figure 5 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 5 The operation can be included in Figure 1 The processor 110 in the vehicle control device 100 executes.

[0141] refer to Figure 5 The processor included in the vehicle control device may merge all or part of the virtual frame associated with the tracking frame. The processor may generate the merged frame 510 based on merging all or part of the virtual frames 520 and 530 associated with the tracking frame.

[0142] The processor may identify virtual frames 520 and 530 which are farthest from each other from among the virtual frames at the second time point included in the merged frame 510 .

[0143] The processor may identify a first virtual frame 520 located relatively close to the host vehicle based on identifying virtual frames that are farthest from each other in the virtual frames at the second time point. The processor may identify a second virtual frame 530 different from the first virtual frame 520 located relatively close to the host vehicle based on identifying virtual frames that are farthest from each other in the virtual frames at the second time point.

[0144] The processor may identify a point included in the first virtual frame 520. The processor may identify a first point 521 closest to the second virtual frame 530 from among the points included in the first virtual frame 520.

[0145] The processor may identify a point included in the second virtual frame 530. The processor may identify a second point 531 closest to the second virtual frame 520 from among the points included in the second virtual frame 530.

[0146] The processor may identify a distance between the first point 521 and the second point 531. The processor may identify a distance 540 between the first point 521 and the second point 531 along the x-axis direction.

[0147] For example, the processor may identify the coordinates of the first point 521. For example, the processor may identify the coordinates of the second point 531.

[0148] The processor may identify a distance 540 between the first point 521 and the second point 531 in the x-axis direction by using an x ​​coordinate among the coordinates of the first point 521 and an x ​​coordinate among the coordinates of the second point 531 .

[0149] The processor may recognize that the distance between the first point 521 and the second point 531 exceeds a sixth reference length (eg, about 20 m) that is less than the fifth reference length. The processor may cancel the merge based on the distance between the first point 521 and the second point 531 exceeding the sixth reference length.

[0150] For example, the processor may unmerge the merging frame 510 based on the distance between the first point 521 and the second point 531 exceeding the sixth reference length.

[0151] The processor may cancel the merging based on the fact that the distance 540 between the first point 521 and the second point 531 in the x-axis direction exceeds the sixth reference length.

[0152] The processor may map the virtual frames 520 and 530 to external objects based on the de-merging.

[0153] Figure 6 An embodiment of the result of separating the merge boxes is shown.

[0154] Figure 6 The operation can be performed by Figure 1 The vehicle control device 100 executes. For example, Figure 6 The operations can be included in Figure 1 The processor 110 in the vehicle control device 100 executes.

[0155] Figure 6 A first embodiment 600 in may include an embodiment of an image representing the surroundings of a host vehicle.

[0156] Figure 6 The second embodiment 601 in may include an embodiment where the output merge box is not applied with the present technology.

[0157] Figure 6 The third embodiment 602 in may include an embodiment in which the merged boxes are unmerged and output by applying the present technology.

[0158] Referring to the second embodiment 601, the processor may generate a merged frame 620 and may output the generated merged frame 620. Even when virtual frames included in the merged frame 620 are positioned away from each other, the processor may output the merged frame 620 without unmerging the merged frame 620.

[0159] The pedestrian 610 and the guardrail may be output as one object by outputting the merged frame 620 without unmerging the merged frame 620 .

[0160] Referring to the third embodiment 602, the processor may unmerge the merged frame 620. The processor may output the first virtual frame 630 and the second virtual frame 635 based on unmerging the merged frame 620.

[0161] Alternatively, the processor may output a plurality of virtual frames at the second point in time, wherein the virtual frames are not merged.

[0162] For example, the first virtual frame 630 may support a portion of a guardrail. For example, the second virtual frame 635 may correspond to the pedestrian 610.

[0163] The processor included in the vehicle control device may accurately map the virtual frames 630 and 635 to the external object by not merging the first virtual frame 630 and the second virtual frame 635 or unmerging the merged frames.

[0164] Figure 7 An embodiment of a flow chart of a vehicle control method is shown.

[0165] In the following, reference will be made to Figure 7 A vehicle control method is described in detail. Figure 7 is a flowchart for describing a vehicle control method.

[0166] In the following, it is assumed Figure 1 The vehicle control device 100 executes Figure 7 In addition, Figure 7 In the description, it can be understood that the operations described as being performed by the device are controlled by the processor 110 of the vehicle control device 100.

[0167] Figure 7 At least one of the operations may be Figure 1 The vehicle control device 100 is executed. Figure 7 Each operation in can be performed sequentially, but does not necessarily have to be performed sequentially. For example, the order of the operations can be changed, and at least two operations can be performed in parallel.

[0168] refer to Figure 7In operation S701, the vehicle control method may include an operation of generating a tracking frame based on a virtual frame corresponding to a designated object at a first time point obtained at a first time point through LiDAR, wherein the tracking frame is obtained by converting the virtual frame at the first time point into a virtual frame at a second time point after the first time point.

[0169] For example, the vehicle control method may include an operation of generating a tracking frame based on merging virtual frames at a first time point.

[0170] In operation S703 , the vehicle control method may include an operation of identifying a virtual frame associated with the tracking frame in the virtual frame at the second time point based on obtaining a virtual frame at the second time point composed of a plurality of points obtained by LiDAR at the second time point.

[0171] For example, the vehicle control method may include an operation of determining whether the virtual frame at the second time point is associated with the tracking frame based on at least one of the following: a ratio (e.g., a proportion) of overlap between the virtual frame at the second time point and the tracking frame, a relative distance between the virtual frame at the second time point and the tracking frame, or any combination thereof.

[0172] In operation S705, the vehicle control method may include the following operations: merging all or part of the associated virtual frames based on at least one of the distance between the virtual frames at the second time point, whether at least one virtual frame in the virtual frames at the second time point corresponds to an object separated from the road boundary, or any combination thereof.

[0173] For example, the vehicle control method may include an operation of generating a merged frame based on merging all or part of associated virtual frames.

[0174] In operation S707 , the vehicle control method may include an operation of maintaining or canceling the merging based on at least one of a distance between a plurality of points included in a merged frame generated by the merging, a type of a designated object, or any combination thereof.

[0175] The vehicle control method may include an operation of mapping the merge box to an external object or outputting the merge box while maintaining the merge.

[0176] The vehicle control method may include the following operations: when canceling the merging, mapping the virtual frame obtained at the second time point and whose merging is canceled to the external object, or separating and outputting the merged frame as the virtual frame at the second time point.

[0177] Figure 8 A computing system associated with a vehicle control device is shown.

[0178] refer to Figure 8, the computing system 1000 may include at least one processor 1100 , a memory 1300 , a user interface input device 1400 , a user interface output device 1500 , a storage device 1600 , and a network interface 1700 connected to each other via a bus 1200 .

[0179] The processor 1100 may be a central processing unit (CPU) or a semiconductor device that processes instructions stored in the memory 1300 and / or the storage device 1600. Each of the memory 1300 and the storage device 1600 may include various types of volatile or non-volatile storage media. For example, the memory 1300 may include a read-only memory (ROM) and a random access memory (RAM).

[0180] Therefore, the operation of the method or algorithm described in conjunction with one or more exemplary embodiments disclosed in the specification may be directly implemented using a hardware module, a software module, or a combination of a hardware module and a software module, which is executed by the processor 1100. The software module may reside on a storage medium (i.e., the memory 1300 and / or the storage device 1600), such as a random access memory (RAM), a flash memory, a read-only memory (ROM), an erasable and programmable ROM (EPROM), an electrically EPROM (EEPROM), a register, a hard disk drive, a removable disk, or a compact disk ROM (CD-ROM).

[0181] The storage medium may be connected to the processor 1100. The processor 1100 may read information from the storage medium and may write information into the storage medium. Alternatively, the storage medium may be integrated with the processor 1100. The processor and the storage medium may be implemented with an application specific integrated circuit (ASIC). The ASIC may be provided in a user terminal. Alternatively, the processor and the storage medium may be implemented with separate components in a user terminal.

[0182] The above description is merely an embodiment of the technical concept of the present disclosure, and various modifications and changes may be made by those skilled in the art without departing from the essential features of the present disclosure.

[0183] Therefore, the present disclosure is not intended to limit but to explain the technical concept of the present disclosure, and the scope and spirit of the present disclosure are not limited by one or more exemplary embodiments described herein. The protection scope of the present disclosure should be interpreted by the attached claims, and all equivalents thereof should be interpreted as included in the scope of the present disclosure.

[0184] The vehicle control device may merge the virtual frame generated by using the LiDAR or may unmerge the merged frame.

[0185] Furthermore, the vehicle control device can reduce the load of the processor by determining whether to merge virtual frames before merging the virtual frames.

[0186] Furthermore, when a pedestrian is recognized to be separated from a road boundary such as a guardrail, a median, or a curb, the vehicle control device can accurately recognize the pedestrian.

[0187] Furthermore, various effects understood directly or indirectly through the specification can be provided.

[0188] Although the present disclosure has been described above with reference to one or more exemplary embodiments and the accompanying drawings, the present disclosure is not limited thereto, but various modifications and changes may be made by those skilled in the art to which the present disclosure belongs without departing from the spirit and scope of the present disclosure as claimed in the appended claims.

Claims

1. A vehicle control device, comprising: sensor; as well as The processor is configured as: converting a first virtual frame associated with a first object and obtained via the sensor at a first time into a second virtual frame associated with a second time later than the first time; determining a plurality of third virtual frames associated with the second virtual frame based on a plurality of data points obtained via the sensor at the second time; Merging at least a portion of the plurality of third virtual frames into a merged frame based on at least one of the following: a distance between two or more third virtual frames in the plurality of third virtual frames at the second time, or whether at least one third virtual frame among the plurality of third virtual frames at the second time corresponds to a second object separated from a road boundary; Maintain or cancel the merging of the at least a portion of the plurality of third virtual frames based on at least one of the following items: The distance between two or more data points included in the merged frame, or The type of the first object; and A signal indicating a result of maintaining or canceling the merge is output.

2. The vehicle control device according to claim 1, wherein: The processor is configured to determine the plurality of third virtual frames based on at least one of the following: The second virtual frame overlaps each of the third virtual frames in the plurality of third virtual frames at the second time by a ratio greater than a first threshold, or A distance between a plurality of first data points included in the second virtual frame and a plurality of second data points included in the plurality of third virtual frames at the second time is within a second threshold.

3. The vehicle control device according to claim 1, wherein: The processor is further configured to: Determining whether the first object is stationary or moving is based on at least one of the following: whether the first object is not blocked, whether the type of the first object is a pedestrian, or Whether the number of the plurality of third virtual frames at the second time is less than a threshold.

4. The vehicle control device according to claim 3, wherein: The at least a portion of the plurality of third virtual frames includes a fourth virtual frame and a fifth virtual frame, and wherein the processor is configured to maintain or cancel the merging by canceling the merging based on: the first object is stationary, At least one of a width or a length of the second virtual frame is greater than or equal to a first threshold, and A distance between a first straight line perpendicular to line segments forming the fourth virtual frame and passing through a center of the fourth virtual frame at the second time and a center of the fifth virtual frame at the second time is greater than or equal to a second threshold.

5. The vehicle control device according to claim 3, wherein: The at least a portion of the plurality of third virtual frames includes a fourth virtual frame and a fifth virtual frame, and wherein the processor is configured to maintain or cancel the merging by canceling the merging based on: the first object is moving, The width and length of each of the plurality of third virtual frames at the second time are less than or equal to a first threshold, and A distance between a center of the fourth virtual frame and a center of the fifth virtual frame is greater than or equal to a second threshold, and the second threshold is greater than the first threshold.

6. The vehicle control device according to claim 1, wherein: The processor is further configured to: determining a fourth virtual frame that is closest to the vehicle associated with the vehicle control device among the plurality of third virtual frames at the second time, wherein a width and a length of the fourth virtual frame are greater than or equal to a threshold; determining a fifth virtual frame associated with the second object among the plurality of third virtual frames at the second time; and The merging of the fourth virtual frame and the fifth virtual frame is cancelled.

7. The vehicle control device according to claim 1, wherein: The processor is further configured to: The fourth virtual frame and the fifth virtual frame, which are farthest from each other in the plurality of third virtual frames at the second time, are determined based on the following items: The first object is not occluded, The type of the first object is not a specific vehicle, and One of a width or a length of the merged frame is greater than or equal to a first threshold.

8. The vehicle control device according to claim 7, wherein: The processor is further configured to: Based on the fact that a distance between a first data point in the fourth virtual frame that is closest to the fifth virtual frame and a second data point in the fifth virtual frame that is closest to the fourth virtual frame is greater than a second threshold, the merging of the fourth virtual frame and the fifth virtual frame is canceled, and the second threshold is less than the first threshold.

9. The vehicle control device according to claim 1, wherein: The processor is further configured to determine an object type associated with the merge box based on: the first object is moving, The width of the merged frame is less than or equal to a first threshold, The length of the merged frame is less than or equal to a second threshold, the second threshold is greater than the first threshold, and The merge box is associated with an area that is at least a threshold distance from a boundary of the sensor's field of view.

10. The vehicle control device according to claim 9, wherein: The processor is configured to maintain or cancel the merge by canceling the merge based on: The type of the first object is a pedestrian, and One of a width of the merged frame or a length of the merged frame is greater than or equal to a third threshold value that is smaller than the first threshold value.

11. A vehicle control method, comprising: converting, by one or more processors of the vehicle, a first virtual frame associated with a first object and obtained via a sensor at a first time into a second virtual frame associated with a second time later than the first time; determining a plurality of third virtual frames associated with the second virtual frame based on a plurality of data points obtained via the sensor at the second time; Merging at least a portion of the plurality of third virtual frames into a merged frame based on at least one of the following: a distance between two or more third virtual frames in the plurality of third virtual frames at the second time, or whether at least one of the plurality of third virtual frames at the second time corresponds to a second object separated from a road boundary; as well as Maintaining or canceling the merging of the at least a portion of the plurality of third virtual frames based on at least one of the following: The distance between two or more data points included in the merged frame, or The type of the first object; and A signal indicating a result of maintaining or canceling the merge is output.

12. The vehicle control method according to claim 11, wherein: Determining the plurality of third virtual frames includes: determining the plurality of third virtual frames based on at least one of the following: The ratio of the second virtual frame overlapping each of the third virtual frames in the plurality of third virtual frames at the second time is greater than a first threshold, or A distance between a plurality of first data points included in the second virtual frame and a plurality of second data points included in the plurality of third virtual frames at the second time is within a second threshold.

13. The vehicle control method according to claim 11, further comprising: Determining whether the first object is stationary or moving is based on at least one of the following: whether the first object is not blocked, whether the type of the first object is a pedestrian, or Whether the number of the plurality of third virtual frames at the second time is less than a threshold.

14. The vehicle control method according to claim 13, wherein: The at least a portion of the plurality of third virtual frames includes a fourth virtual frame and a fifth virtual frame, and wherein the maintaining or canceling of the merger includes canceling the merger based on: the first object is stationary, At least one of a width or a length of the second virtual frame is greater than or equal to a first threshold, and A distance between a first straight line perpendicular to line segments forming the fourth virtual frame and passing through a center of the fourth virtual frame at the second time and a center of the fifth virtual frame at the second time is greater than or equal to a second threshold.

15. The vehicle control method according to claim 13, wherein: The at least a portion of the plurality of third virtual frames includes a fourth virtual frame and a fifth virtual frame, and wherein the maintaining or canceling of the merger includes canceling the merger based on: the first object is moving, The width and length of the plurality of third virtual frames at the second time are less than or equal to a first threshold, and A distance between a center of the fourth virtual frame and a center of the fifth virtual frame is greater than or equal to a second threshold, and the second threshold is greater than the first threshold.

16. The vehicle control method according to claim 11, further comprising: determining a fourth virtual frame closest to the vehicle among the plurality of third virtual frames at the second time, wherein a width and a length of the fourth virtual frame are greater than or equal to a threshold; determining a fifth virtual frame associated with the second object among the plurality of third virtual frames at the second time; and The merging of the fourth virtual frame and the fifth virtual frame is cancelled.

17. The vehicle control method according to claim 11, further comprising: The fourth virtual frame and the fifth virtual frame, which are farthest from each other in the plurality of third virtual frames at the second time, are determined based on the following items: The first object is not occluded, The type of the first object is not a specific vehicle, and One of a width or a length of the merged frame is greater than or equal to a first threshold.

18. The vehicle control method according to claim 17, further comprising: Based on the fact that a distance between a first data point in the fourth virtual frame that is closest to the fifth virtual frame and a second data point in the fifth virtual frame that is closest to the fourth virtual frame is greater than a second threshold, the merging of the fourth virtual frame and the fifth virtual frame is canceled, and the second threshold is less than the first threshold.

19. The vehicle control method according to claim 11, further comprising: The object type associated with the merge box is determined based on: the first object is moving, The width of the merged frame is less than or equal to a first threshold, The length of the merged frame is less than or equal to a second threshold, the second threshold is greater than the first threshold, and The merge box is associated with an area that is at least a threshold distance from a boundary of the sensor's field of view.

20. The vehicle control method according to claim 19, wherein: The maintenance or cancellation of the merger includes canceling the merger based on: The type of the first object is a pedestrian, and One of a width of the merged frame or a length of the merged frame is greater than or equal to a third threshold value that is smaller than the first threshold value.

Images