Method and system for tracking object in field of view of host vehicle
By creating pseudo-objects in the automotive system, the problem of failure of safety function when object tracking fails is solved, ensuring that the safe distance and speed of the target vehicle can be maintained while detecting and tracking system failures, and driving safety is improved.
Patent Information
- Application Number
- CN202411070409.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-11-30
- Filing Date
- 2024-08-06
- Publication Date
- 2025-05-30
AI Technical Summary
When an existing automotive system fails to track objects, it may lead to the failure of safety features such as automatic emergency braking (AEB) and adaptive cruise control (ACC), thereby increasing the risk of collision.
By creating a pseudo-object, using the grid representation of the last known position and forward path of the previously detected object, the unwired part is determined, thereby providing an alternative target object to ensure the proper operation of the security function when the object tracking fails.
It effectively avoids the failure of safety functions due to object tracker failure, ensuring that the safe distance and safe relative speed of the target vehicle can be maintained even in the case of detection and tracking system failure, thereby improving driving safety.
Smart Images
Figure CN120071292A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of automotive advanced safety and autonomous driving, and more particularly to methods, systems, and computer-readable storage media for tracking objects in the field of view (FOV) of a host vehicle. Background Art
[0002] Advanced driver assistance systems (ADAS) assist the driver to drive the host vehicle more safely and comfortably. For proper operation and for safety reasons, it is necessary to monitor the environment in front of the host vehicle, e.g., to determine a collision-free space in front of the host vehicle. More specifically, safety functions such as automatic emergency braking (AEB) and adaptive cruise control (ACC) can rely on selecting a target vehicle located in the forward path of the host vehicle, i.e., a vehicle to which such safety functions apply. Based on the respective selected target vehicle, the safety functions can control the host vehicle in a manner that maintains a safe distance and / or a safe relative speed to the selected target vehicle.
[0003] Therefore, the safety functions depend on the object tracker used upstream of the host vehicle robustly estimating the presence and state of the target object to be selected. However, if for some reason the object tracker fails to report a target vehicle exactly on the path of the host vehicle, safety functions such as AEB and ACC may similarly fail because no target object is detected, and thus the safety functions of the host vehicle cannot select to use such a target object. As a result, there are weaknesses in known safety systems where a poor object tracker of the host vehicle may indirectly lead to poor performance of the safety functions and, in the worst case, may even cause a collision of the host vehicle with another vehicle.
[0004] Therefore, there is a need for improved methods and systems for tracking objects in the field of view (FOV) of a host vehicle. Summary of the Invention
[0005] One aspect of the present invention relates to a computer-implemented method for tracking an object in the field of view (FOV) of a host vehicle. The computer-implemented method includes detecting and tracking an object in the FOV by a detection and tracking system mounted on the host vehicle. When a previously detected object fails to be tracked in the FOV, a pseudo-object representing the previously detected object is created based on the last known position of the previously detected object and for non-drivable portions of the FOV.
[0006] When tracking a previously detected object in the FOV fails, the present invention avoids the above problems caused by a poor object tracker employed by the host vehicle. In particular, the present invention ensures that safety functions such as automatic emergency braking (AEB) and adaptive cruise control (ACC) can still be applied even in the event of a failure of the object tracking system of the host vehicle. Generally, the present invention provides a more robust selection of target objects in front of the host vehicle for application to the safety functions of the host vehicle.
[0007] In one embodiment, the step of creating a pseudo object includes evaluating the FOV of the host vehicle to determine non-drivable portions of the FOV.
[0008] The step of evaluating the FOV of the host vehicle to determine non-drivable portions of the FOV allows compensation for failures of the detection and tracking system.
[0009] In another embodiment, the step of evaluating the FOV includes evaluating a grid representation of the FOV, wherein the grid representation includes a grid of cells, wherein each cell represents a portion of the FOV, and wherein each cell is classified according to a drivability class that includes at least drivable or non-drivable.
[0010] Using the grid representation of the FOV enables effective determination of non-drivable portions of the FOV.
[0011] In another embodiment, the last known position of the previously detected object is defined by a bounding box located in the grid representation and enclosing the previously detected object. The step of evaluating the FOV to determine non-drivable portions of the FOV includes creating a set of points in the grid representation in a region at the rear edge of the bounding box, and for each point in the set of points in an increasing longitudinal direction relative to the bounding box, determining whether the corresponding point is located within a cell classified as non-drivable, and if the corresponding point is located within a cell classified as non-drivable, selecting the corresponding point as a reference point in the grid representation for creating the pseudo object.
[0012] In this way, a reliable estimate of the current position of the target after a detection and tracking system failure is provided.
[0013] In another embodiment, each point in the set of points is created to be positioned in the grid representation at a respective offset in a longitudinal direction relative to the rear edge of the bounding box.
[0014] By applying an offset to the rear edge of the bounding box of the previously detected object, changes in the speed of the previously detected object can be compensated for, as the object can have an increasing or decreasing speed.
[0015] In another embodiment, the step of creating the set of points includes creating at least one point within each cell in the region that is included in the rear edge of the bounding box.
[0016] In this way, it is ensured that the created set of points is dense enough, and thus the region of interest (i.e., the possible current position of the previously detected object) is sufficiently covered by the set of points.
[0017] In another embodiment, the method further includes creating a pseudo bounding box representing a pseudo object at a reference point in the grid representation. The pseudo bounding box is created with a size corresponding to the size of the bounding box representing the previously detected object, where the rear edge of the pseudo bounding box is located at the reference point. Thus, the pseudo bounding box is shifted in the longitudinal direction relative to the bounding box of the previously detected object. In addition, the attributes of the previously detected object are assigned to the created pseudo object.
[0018] In this way, a pseudo object with the same attributes and dimensions as the previously detected object can be created at the possible current position of the previously detected object to ensure that, for example, safety functions that depend on target selection in the forward path of the host vehicle still apply to the host vehicle, regardless of the failure of the detection and tracking system.
[0019] In another embodiment, the set of points is created based on a set of parameters.
[0020] This allows for greater flexibility regarding the region of interest in which the previously detected object may be located after the loss of detection and tracking signals of the previously detected object.
[0021] In another embodiment, the set of parameters includes the lateral distance and / or the vertical distance between two adjacent points in the set of points.
[0022] In this way, the density of the created set of points can be adjusted.
[0023] In another embodiment, the set of parameters includes the number of points to be created.
[0024] This is another parameter that allows for adjusting the density of the created set of points.
[0025] In another embodiment, before creating the pseudo object, the last known position of the previously detected object is compensated for the movement of the host vehicle.
[0026] This allows for a more reliable determination of the current position of the previously detected object, and thus a more reliable determination of the position of the created pseudo object.
[0027] In another embodiment, the method further includes selecting the created pseudo object as the target object to which one or more security functions are applied.
[0028] Using the created pseudo object as the target object to which one or more security functions are applied enhances driving safety and can ensure a safe distance and a safe relative speed to the target vehicle even in the case of a detection and tracking system failure.
[0029] In another embodiment, one or more security functions at least include automatic emergency braking (AEB) and adaptive cruise control (ACC).
[0030] Another aspect of the present invention relates to a system including an apparatus for performing the above computer-implemented method.
[0031] Another aspect of the present invention relates to a computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the above method. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] To best describe the manner of implementing the above embodiments and to define other advantages and features of the present disclosure, a more specific description is provided below and is illustrated in the accompanying drawings. It should be understood that these drawings only depict exemplary embodiments of the present invention and should not be considered as limiting the scope. These examples will be described and explained with additional specificity and detail by using the drawings, in which:
[0033] Figure 1 A typical road scene is shown;
[0034] Figure 2 An object tracking scene is shown;
[0035] Figure 3 A target tracking scene after losing the tracked target is shown;
[0036] Figure 4 A computer-implemented method for tracking an object in the field of view (FOV) of the host vehicle is shown; and
[0037] Figure 5 A system for tracking an object in the field of view (FOV) of the host vehicle is shown. DETAILED DESCRIPTION
[0038] The various embodiments of the disclosed method and apparatus are discussed in detail below. Although specific embodiments are discussed, it should be understood that this is for illustrative purposes only. Those skilled in the relevant art will recognize that other components, configurations, and steps can be used without departing from the scope of the present disclosure.
[0039] Figure 1Shows a typical road scenario 100 with three vehicles. In front of the host vehicle 110, another vehicle 120 is traveling in the forward path (i.e., the field of view FOV) of the host vehicle 110 (in the direction indicated by the dashed arrow). A detection and tracking system 115 is installed on the host vehicle 110. In the shown scenario, the detection and tracking system 115 has detected another vehicle 120 in the forward path of the host vehicle 110 and is tracking the other vehicle 120 as a target object. The target object can be used for applications such as safety functions, such as AEB (Automatic Emergency Braking) and ACC (Adaptive Cruise Control). As the detection and tracking system 115, any distance sensor can be employed to obtain distance detection, such as a lidar sensor or a radar sensor.
[0040] Figure 2 Shows Figure 1 A different view of the shown situation. As Figure 2 shown, the vehicle 120 in the forward path of the host vehicle 110 is detected as a target object via the detection and tracking system 115 and is being tracked. As shown, the detected vehicle 120 can be represented by a bounding box surrounding the detected vehicle 120. The dimensions of the corresponding bounding box are considered to correspond to the true object dimensions, i.e., the width and length of the corresponding detected and tracked vehicle 120.
[0041] As Figure 3 shown, at a certain moment, the tracking of the previously detected object 120 in the FOV of the host vehicle may fail (represented by the dashed bounding box indicating the loss of the previously detected object 120). In known systems, if the tracking of a previously detected object in the forward path of the host vehicle fails for some reason, features or safety functions such as AEB and ACC may also fail because a target object cannot be selected for these safety functions. Therefore, a poor object tracker can indirectly lead to poor feature performance and, in the worst case, even cause a collision between the host vehicle and an object in its forward path.
[0042] According to the present invention, the problem of the sudden loss of a previously detected object in the forward path of the host vehicle is addressed by evaluating a grid representation of the surroundings (i.e., the FOV of the host vehicle) near the last known position of the previously detected object in front of the host vehicle to create a pseudo-object representing the previously detected object 120, which can then be used as a target object for the safety functions of the host vehicle.
[0043] Figure 3 Shows the corresponding scenario of the sudden loss of the previously detected object 120 represented by the dashed bounding box indicating the previously detected object 120. Figure 3A grid representation 130 of the FOV of the host vehicle 110 is also shown. The grid representation 130 includes a cell grid, where each cell represents a part of the FOV, and where each cell (and thus each part of the FOV of the host vehicle) is classified according to a drivability class, the drivability class including at least drivable or non-drivable.
[0044] The grid representation 130 can be created and updated using many different sources, such as visual lane markings and trajectories of tracked objects and radar and visual “freespace” algorithms. The “freespace” algorithm divides the FOV of one or several sensors (e.g., the detection and tracking system 115) into multiple parts and reports the distances of each part that is occupied (i.e., non-drivable). The algorithm converts the various sources into cells that make up the grid representation 130, where each cell is classified as drivable, non-drivable, or unknown. Thus, even when an upstream object tracker (such as the detection and tracking system 115) suddenly fails to report, the grid representation 130 can be used to detect that something is in front of the host vehicle 110.
[0045] The non-drivable part of the FOV can be a hint towards the possible current location of a previously detected object 120. To identify the non-drivable parts of the FOV (i.e., the cells of the grid representation 130 that are classified as non-drivable), the last known location of the previously detected object 120 is saved. In one embodiment, the last known location can optionally be compensated for the final movement of the host vehicle 110 relative to the previously detected object 120.
[0046] As Figure 3 shown, since the travel speed of the previously detected object 120 may have changed since the tracking failure, a set of points 140 is created in the area of the rear edge of the bounding box of the previously detected object. More specifically, to compensate for the change in the travel speed of the previously detected object, each point 140 is created to be positioned in the grid representation 130 with a certain offset relative to the rear edge of the bounding box in the longitudinal direction. Then, the points 140 are evaluated in increasing longitudinal order in the grid representation 130. If any point 140 is determined to be located in a cell classified as non-drivable during the evaluation, the corresponding point 140 is selected as a reference point in the grid representation 130 for creating a pseudo-object. Then, a pseudo-bounding box representing the pseudo-object is created, where the pseudo-bounding box is created with a size corresponding to the size of the bounding box representing the previously detected object 120. As Figure 3 shown, the rear edge of the created pseudo-bounding box is located at the reference point 140 in the grid representation 130.
[0047] In other words, the pseudo-object is created using the attributes inherited from the previously detected object 120. That is, the created pseudo-object has the same attributes as the previously detected object 120, except that the longitudinal position of the rear edge of the bounding box enclosing the created pseudo-object is shifted to the reference point 140, which has been determined to be located within the first non-drivable cell encountered in the grid representation 130.
[0048] To ensure that the created set of points 140 is dense enough and thus the region of interest (i.e., the possible current position of the previously detected object) is sufficiently covered by the set of points 140, at least one point 140 is created within each cell included in the region including the rear edge of the bounding box enclosing the previously detected object 120.
[0049] To allow more flexibility regarding the region of interest where the previously detected object 120 may be located after the loss of its detection and tracking signals, the set of points 140 can be created based on a set of parameters. For example, to adjust the density of the created set of points 140, the set of parameters can include the lateral distance and / or the vertical distance between two adjacent points. Additionally or alternatively, the set of parameters can include the number of points 140 to be created.
[0050] Figure 4 A computer-implemented method for tracking an object in the field of view (FOV) of a host vehicle 110 is shown. The method includes step 410: detecting and tracking an object 120 in the FOV by a detection and tracking system 115 mounted on the host vehicle 110. When the previously detected object 120 cannot be tracked in the FOV, in step 420, a pseudo-object representing the previously detected object 120 is created based on the last known position of the previously detected object 120 and for the non-drivable portion of the FOV.
[0051] Figure 5 A system 500 for tracking an object in the field of view (FOV) of a host vehicle 110 is shown. System 500 includes a detection and tracking system 115 mounted on the host vehicle 110, wherein the detection and tracking system 115 is configured to detect and track an object 120 in the FOV. The system further includes means 510 configured to create a pseudo-object representing the previously detected object 120 based on the last known position of the previously detected object 120 and for the non-drivable portion of the FOV when tracking the previously detected object 120 in the FOV fails.
[0052] The present invention allows for more robust target selection even when an upstream object tracker fails and loses track of an existing object. In particular, the present invention can ensure the correct operation of safety functions in a host vehicle even in the case where the object tracker employed by the host vehicle fails, so as to avoid accidents such as a collision between the host vehicle and another vehicle. The present invention is useful in many applications of automotive advanced safety and autonomous driving, as it can maintain the characteristic functions that require a target object in the forward path of the host vehicle. For example, even when the upstream object tracker fails to report an existing object in the forward path of the host vehicle for some reason, the AEB and ACC features can remain enabled.
[0053] The various embodiments described above are provided by way of illustration only and should not be construed as limiting the present invention. Those skilled in the art will readily recognize that various modifications and changes can be made to the present invention without following the exemplary embodiments and applications shown and described herein and without departing from the scope of the present disclosure.
Claims
1. A computer-implemented method for tracking an object (120) in a field of view (FOV) of a host vehicle (110), the computer-implemented method comprising the following steps: detecting and tracking (410) an object (120) in the FOV by a detection and tracking system (115) mounted on the host vehicle (110); When tracking of a previously detected object (120) in the FOV fails, a pseudo object representing the previously detected object (120) is created (420) based on the last known position of the previously detected object (120) and for a non-drivable portion of the FOV.
2. The computer-implemented method of claim 1 , wherein: The step of creating (420) the pseudo object includes evaluating the FOV of the host vehicle (110) to determine the non-drivable portion of the FOV.
3. The computer-implemented method of claim 2, wherein: The step of evaluating the FOV includes evaluating a grid representation (130) of the FOV, wherein the grid representation (130) includes a grid of cells, wherein each cell represents a portion of the FOV, and wherein each cell is classified according to a drivability category, wherein the drivability category includes at least drivable or non-drivable.
4. A computer-implemented method according to claim 2 or 3, wherein: The last known position of the previously detected object (120) is defined by a bounding box located in the grid representation (130) and surrounding the previously detected object (120), and The step of evaluating the FOV to determine the non-drivable portion of the FOV comprises: creating a set of points (140) in the grid representation (130) in the region of the back edge of the bounding box; In increasing longitudinal directions relative to the bounding box and for each point (140) in the set of points (140), determining whether the corresponding point (140) is located within a cell classified as non-drivable; and If the corresponding point (140) is located within a cell classified as non-drivable, the corresponding point (140) is selected as a reference point in the grid representation (130) for creating the pseudo object.
5. The computer-implemented method of claim 4, wherein: Each point (140) in the set of points (140) is created to be positioned in the grid representation (130) with respect to the back edge of the bounding box in a longitudinal direction relative to the bounding box at a corresponding offset.
6. A computer-implemented method according to claim 4 or 5, wherein: The step of creating the set of points (140) includes creating at least one point (140) within each cell in the area included in the rear edge of the bounding box.
7. The computer-implemented method according to any one of claims 4 to 6, further comprising the steps of: creating a pseudo bounding box representing the pseudo object at the reference point in the grid representation (130), wherein the pseudo bounding box is created with a size corresponding to a size of the bounding box representing the previously detected object (120), and wherein a trailing edge of the pseudo bounding box is located at the reference point; and The properties of the previously detected object (120) are assigned to the created pseudo object.
8. A computer-implemented method according to any one of claims 4 to 7, wherein: The step of creating the set of points (140) is based on a set of parameters.
9. The computer-implemented method of claim 8, wherein: The set of parameters includes a lateral distance and / or a vertical distance between two adjacent points (140) in the set of points (140).
10. The computer-implemented method of claim 8 or 9, wherein: The set of parameters includes the number of points (140) to be created.
11. A computer-implemented method according to any one of claims 1 to 10, wherein: Prior to creating (420) the pseudo object, the last known position of the previously detected object (120) is compensated with respect to the motion of the host vehicle (110).
12. The computer-implemented method of any one of claims 1 to 11, further comprising selecting the created pseudo object as a target object for applying one or more security functions.
13. The computer-implemented method of claim 12, wherein: The one or more safety functions include at least automatic emergency braking AEB and adaptive cruise control ACC.
14. A system (500) comprising means (110, 1150, 510) for performing the computer-implemented method according to any one of claims 1 to 13.
15. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the computer-implemented method according to any one of claims 1 to 13.