Collision time estimation in computer vision systems

Abstract

The invention relates to collision time estimation in a computer vision system. The invention provides a method for estimating a time-to-collision (TTC) of a detected object in a computer vision system, the method comprising: determining (200) a three-dimensional (3D) position of a camera in the computer vision system; determining (202) a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image; calculating (204) a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object; and calculating (206) the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

Description

[0001] This application is a divisional application of the invention patent application filed on October 18, 2017, with application number 201710971461.7 and invention title "Collision Time Estimation in Computer Vision System". Technical Field

[0002] Embodiments of the present invention generally relate to computer vision systems, and more specifically to collision time estimation in computer vision systems. Background Technology

[0003] Estimated Time-of-Collision (TTC) is crucial information in Advanced Driver Assistance Systems (ADAS) and autonomous driving because it is used in safety-critical applications such as Forward Collision Warning (FCW), Automatic Emergency Braking (AEB), and Adaptive Cruise Control (ACC). TTC can be defined as the time it takes for an object to collide with an obstacle in its path, or, in the case of a moving vehicle, as the estimated time until the vehicle collides with an object in its trajectory. Many current methods for calculating TTC are based on sensor types such as radar and lidar. LiDAR can be prohibitively expensive for consumer vehicles, and radar's angular resolution may be suboptimal. Summary of the Invention

[0004] Embodiments of the present invention relate to estimating collision time in a computer vision system. In one aspect, a method for estimating the collision time (TTC) of a detected object in a computer vision system is provided, the method comprising: determining a three-dimensional (3D) position of a camera in the computer vision system; determining a 3D position of the detected object based on a 2D position of the detected object in an image captured by the camera and an estimated ground plane corresponding to the image; calculating a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object; and calculating the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

[0005] In one aspect, a computer vision system is provided, the computer vision system comprising: a monocular camera configured to capture a two-dimensional (2D) image of a scene; an object detection component configured to detect objects in the 2D image; and a time-of-collision (TTC) component configured to estimate the TTC of the objects detected by the object detection component, wherein the TTC component is configured to determine a three-dimensional (3D) position of the camera corresponding to the 2D image; determining a 3D position of the detected object based on the 2D position of the detected object in the image and an estimated ground plane corresponding to the image; calculating a relative 3D position of the camera, a velocity of the relative 3D position, and an acceleration of the relative 3D position based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object; and calculating the TTC of the detected object based on the relative 3D position, the velocity, and the acceleration.

[0006] In one aspect, a method is provided for estimating the time-to-collision (TTC) of a detected object in a computer vision system, the method comprising: calculating a distance of the detected object from a camera included in the computer vision system, wherein the detected object is located within the field of view of the camera; calculating a presumed velocity of the camera's relative three-dimensional (3D) position along a direction of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object; calculating a presumed acceleration of the camera's relative 3D position along the direction of the detected object; and calculating the TTC based on the distance, the presumed velocity, and the presumed acceleration.

[0007] In one aspect, a method for determining whether a detected object is located in the trajectory of a vehicle, the method comprising: determining an asymptotic angle of the detected object, wherein the asymptotic angle is an angle between a velocity vector of a camera included in the vehicle and a camera-object vector, wherein the detected object is located in the field of view of the camera; and comparing the asymptotic angle with an angle threshold to determine whether the detected object is located in the trajectory. Attached Figure Description

[0008] Specific embodiments will now be described by way of example and with reference to the accompanying drawings:

[0009] Figure 1 This is a block diagram of a system used to estimate the time of collision (TTC).

[0010] Figure 2 It is used for calculation Figure 1 A flowchart of the method for estimating TTC in the system;

[0011] Figure 3 This is a flowchart of a method for estimating the TTC of an object;

[0012] Figure 4 It is a graphical illustration of an example of estimating the trajectory of an object; and

[0013] Figure 5 This is a high-level block diagram of an instanced multiprocessor single-chip system (SOC) configured for use in a monocular camera-based automotive safety application. Detailed Implementation

[0014] Specific embodiments of the invention will now be described in detail with reference to the accompanying drawings. For consistency, similar elements in the figures are indicated by similar reference numerals.

[0015] Embodiments of the present invention provide for estimating the time of collision (TTC) between a moving vehicle and an object in the vehicle's trajectory based on frames captured by a forward-facing monocular camera mounted on the vehicle. More specifically, the TTC is estimated using information such as the distance of the detected object from the vehicle, the relative acceleration and velocity of the detected object, and whether any of the detected objects is in the collision path (i.e., in the vehicle's trajectory).

[0016] Figure 1 This is a block diagram of a system 100 for estimating Time-of-Collision (TTC). Images captured by a forward-facing monocular camera (not shown) mounted in the vehicle are input to system 100. System 100 includes an object detection component 102, a structure-of-motion (SfM) component 104, a ground plane estimation component 106, and a time-of-collision (TTC) component 108.

[0017] Object detection component 102 is coupled to a camera to receive images. Object detection component 102 is configured to detect objects in the camera's field of view (FOV) in each input image, and for each detected object, output a bounding box and a unique identifier indicating its two-dimensional (2D) location in the image. Object detection component 102 can use any suitable real-time object detection technique. In some embodiments, object detection component 102 is configured to detect vehicles and pedestrians in the FOV.

[0018] The SfM component 104 is coupled to a camera to receive images. The SfM component 104 is configured to generate a three-dimensional (3D) point cloud, which represents the 3D environment corresponding to each input image in the camera's field of view (FOV). The SfM component 104 is further configured to generate a camera pose matrix as part of the 3D point cloud generation. As is known, the camera pose matrix is ​​a 4x4 matrix containing: a 3x3 rotation matrix indicating how much the camera has rotated in 6 degrees of freedom; and a translation vector indicating how much the camera has moved along the x, y, and z directions. The camera pose matrix generated by the SfM component 104 is a relative pose matrix, indicating any changes in camera translation and rotation from previous images. The SfM component 104 can use any suitable real-time SfM technique. For example, a suitable example of SfM technology can be found in U.S. Patent Application No. 15 / 197,749, filed June 20, 2016, entitled “Method and System for Real Time Structure from Motion in a Computer Vision System,” which is incorporated herein by reference.

[0019] Ground plane estimation component 106 is coupled to SfM component 104 to receive a 3D point cloud. Ground plane estimation component 106 is configured to estimate the ground plane based on the 3D point cloud. Ground plane estimation component 106 can use any suitable real-time ground plane estimation technique. For example, examples of suitable ground plane estimation techniques can be found in U.S. Patent Application No. 15 / 255,832, filed September 2, 2016, entitled "Ground Plane Estimation in a Computer Vision System," which is incorporated herein by reference.

[0020] The TTC component 108 is coupled to the object detection component 102 to receive the bounding box and identifier of the detected object, coupled to the SfM component 104 to receive the camera pose matrix, and coupled to the ground plane estimation component 106 to receive the estimated ground plane. The TTC component 108 is configured to use the received information to estimate the TTC of objects in the vehicle's trajectory. (Reference) Figure 2 and Figure 3 The method explains the operation of TTC component 108.

[0021] Figure 2 This is a flowchart of a method for estimating TTC, which can be derived from... Figure 1The TTC component 108 in the system performs this operation. First, the 3D position of camera 200 is determined based on the camera pose matrix received from SfM component 104. To determine the 3D position of the camera, translation vectors are extracted from the camera pose matrix. As previously described, the translation vectors contain the changes in the X, Y, and Z coordinates of the camera relative to the previous image. The absolute camera position [X] is obtained using the previous translation vector and the current translation vector. c ,Y c Z c The camera position is fed into a Kalman filter, which is configured to track the camera position. However, since the camera pose matrix is ​​calculated based on camera images, there may be errors in the camera pose. The Kalman filter is a constant acceleration model that predicts the output measurement based on constant acceleration. The output of the Kalman filter is the filtered camera position [X]. c ,Y c Z c ]', the velocity vector [Vx] that indicates the rate of change (velocity) of the camera's position. c Vy c ,Vz c ] and the acceleration vector [Ax] indicating the rate of change (acceleration) of the camera's velocity. c Ay c Az c ].

[0022] The 3D position of the detected object 202 is also determined for each of the detected objects received from the object detection component 102. As previously described, the object detection component 102 outputs a unique identifier and bounding box for each detected object. Each bounding box is represented by the 2D coordinates of the corners of the bounding box. For the purpose of determining the 3D position of the detected object, the 2D coordinates of the midpoint of the bottom of the corresponding bounding box are used to obtain the 2D position of the object, since the bottom of the bounding box should be on the ground plane.

[0023] The 3D position of the detected object on the ground is calculated using the 2D position at the bottom of the corresponding bounding box described above and the ground plane equation provided by the ground plane estimation component 106. The ground plane equation for the image is of the form AX + BY + CZ = 1, where A, B, and C are coefficients of the ground plane determined by the ground plane estimation component 106, and X, Y, and Z are 3D coordinates. The 3D position of the detected object with 2D coordinates x and y is calculated by solving the following equation to determine the X, Y, and Z of the object:

[0024] AX + BY + CZ = 1

[0025] Xf / Z=x

[0026] Yf / Z=y

[0027] Where f is a given focal length (pinhole camera model).

[0028] Because the bounding boxes of the detected objects may jump between images and / or fail to cover the bottom of the object when it intersects with the ground, a Kalman filter is used to track the position of each 3D object [X]. o ,Y o Z o To reduce noise. Each of the Kalman filters is a constant acceleration model that predicts the output measurement based on constant acceleration. The output in each Kalman filter is the filtered 3D position of the object [X]. o ,Y o Z o ]', the velocity vector [Vx] that indicates the rate of change (velocity) of an object's position. o Vy o ,Vz o ] and the acceleration vector [Ax] which indicates the rate of change (acceleration) of an object's velocity. o Ay o Az o ].

[0029] Since the estimated ground plane coefficients [A,B,C] may also have errors, a Kalman filter is used to track these coefficients. The Kalman filter is a constant velocity model that predicts the output measurement based on a constant velocity. The output of the Kalman filter is a velocity vector [V] whose rate of change (velocity) is measured by the filtered coefficients [A,B,C]' and the indicator coefficients. A V B V C A Kalman filter can be applied by the ground plane estimation component 106 or the TTC component 108.

[0030] Next, the 3D camera position of 204 relative to the position of each 3D object is calculated. That is, for each detected object, the 3D camera position is calculated from the filtered 3D camera position [X]. c ,Y c Z c Subtract the filtered 3D object position [X] o ,Y o Z o [X]' to determine the position of the 3D camera relative to the position of the 3D object r ,Y r Z r Using a Kalman filter to track relative position [X] r ,Y r Z rEach of the Kalman filters is a constant acceleration model that predicts the output measurement based on constant acceleration. The output of each Kalman filter is the filtered 3D relative position [X]. r ,Y r Z r ]', A velocity vector [Vx] that indicates the rate of change (velocity) of 3D relative position. r Vy r ,Vz r ] and the acceleration vector [Ax] which indicates the rate of change (acceleration) of the velocity relative to the 3D position. r Ay r Az r ].

[0031] Next, based on the filtered 3D relative position [X] corresponding to each detected object... r ,Y r Z r ]'、 Velocity vector [Vx r Vy r ,Vz r ] and acceleration vector [Ax r Ay r Az r The collision time (TTC) of the object is then calculated. See below for reference. Figure 3 The method describes how to calculate the TTC of an object.

[0032] The method also checks the trajectory of each of the 208 detected objects to determine if any object is within the vehicle's trajectory. If none of the detected objects are within the vehicle's trajectory, the method terminates. If any object is determined to be within the vehicle's trajectory, the TTC of that object is output (210) and the method terminates. If the object's asymptotic angle θ is less than a predetermined angle threshold, then the detected object is determined to be within the vehicle's trajectory. The asymptotic angle of the detected object can be determined by calculating the angle between the camera velocity vector and the camera-object vector, as given by the following equation.

[0033] θ=absolute(acos((u*v) / (mod(u)*mod(v))))

[0034] Where u and v are two vectors. Any suitable value for the angle threshold can be used and can be determined empirically. In some embodiments, the angle threshold may be between 0 and 10 degrees. The angle threshold may be, for example, based on the position of the camera relative to the expected lane width.

[0035] Because objects closer to the camera in the same lane have a greater asymptotic angle variability θ than objects farther away in the same lane, a distance-based angle thresholding method can be used in some embodiments. The distance-based angle thresholding method is based on a predetermined angle threshold, such that the angle threshold gradually decreases as the object's distance from the camera increases. (Reference) Figure 3 The method explains how to calculate the distance between an object and the camera.

[0036] For example, suppose the predetermined angle threshold is 8 degrees. Distance-based thresholding can be applied as follows: If the distance to the object is less than 3 meters, then the predetermined angle threshold across the entire span (i.e., 8°) is used for trajectory checking. If the distance to the object is between 3 and 6 meters, then 75% of the predetermined angle threshold (i.e., 6°) is used for trajectory checking. If the distance to the object is between 6 and 12 meters, then 50% of the predetermined angle threshold (i.e., 4°) is used for trajectory checking. If the distance to the object is greater than 12 meters, then 25% of the predetermined angle threshold (i.e., 2°) is used for trajectory checking.

[0037] Any suitable distance range, number of distance ranges, and / or predetermined angle threshold reduction for each distance range can be used. The distance range, number of distance ranges, and / or threshold reduction can be determined empirically.

[0038] Figure 4 This is a diagram illustrating an example of distance-based thresholding. In this example, as indicated by the bounding box, five objects have been detected on the ground. The camera-object vector for each object is shown by the arrows between the camera position and the objects. The tracked camera velocity vector V is also shown. c As previously described, the asymptotic angle of each object is determined based on the camera velocity vector and the camera-object vector. In this example, since only θ1 < the angle threshold, only objects corresponding to this asymptotic angle are located in the vehicle's trajectory.

[0039] Figure 3 This is a flowchart of a method for calculating the collision time of objects. The input to this method is the filtered 3D relative position [X] corresponding to the object. r ,Y r Z r ]'、 Velocity vector [Vx r Vy r ,Vz r ] and acceleration vector [Ax r Ay r Az r First, calculate the distance d from the 300 object to the camera. The distance d can be calculated using the following equation.

[0040] d = norm([Xr ,Y r Z r ]').

[0041] Estimate the relative velocity of camera 302 along the direction of the object. The estimated velocity v can be calculated using the following equation.

[0042] v = [Vx] r Vy r ,Vz r ]x([X r ,Y r Z r ]' / norm([X r ,Y r Z r ]')).

[0043] Furthermore, the relative acceleration of the 304 camera along the object's direction is inferred. The inferred acceleration 'a' can be calculated using the following equation.

[0044] a = [Ax] r Ay r Az r ]x([X r ,Y r Z r ]' / norm([X r ,Y r Z r ]')).

[0045] Next, the collision time TTC is calculated according to the following equation.

[0046] If a = 0, then TTC = d / v

[0047] Otherwise, TTC = min[(-v(+-)sqrt(v)] 2 +2ad)) / a).

[0048] Figure 5This is a high-level block diagram of an instanced multiprocessor system-on-a-chip (SoC) 500 configured for use in a monocular camera-based ADAS. Specifically, the instanced SoC 500 is an embodiment of the TDA3X SoC available from Texas Instruments, Inc. A high-level description of the components of the SoC 500 is provided herein. More detailed descriptions of the instanced components can be found in: “High Performance Front Camera ADAS Applications on TI’s TDA3X Platform” by M. Mody et al., and Proceedings of the 22nd International Conference on High Performance Computing, IEEE 2015, Bangalore, India, December 16-19, 2015. nd Pages 456-463 of the International Conference on High Performance Computing and pages 1-6 of the SPRT704A "TDA3x SOC Processors for Advanced Driver Assist Systems (ADAS) Technical Brief" issued by Texas Instruments in October 2014 are incorporated herein by reference.

[0049] The SOC 500 includes dual general-purpose processors (GPPs) 502, dual digital signal processors (DSPs) 504, and a vision processor 506 coupled via high-speed interconnects 522. The SOC 500 further includes a direct memory access (DMA) component 508, a camera capture component 510 coupled to a monocular camera 524, a display management component 514, on-chip random access memory (RAM) 516 (e.g., computer-readable media), and various input / output (I / O) peripherals 520, all of which are coupled to the processors via interconnects 522. Additionally, the SOC 500 includes a safety component 518, which includes safety-related functionalities that meet automotive safety requirements. These functionalities may include support for CRC (cyclic redundancy check) of data, a clock comparator for drift detection, error signaling, a windowed watchdog timer, and SOC self-tests for damage and malfunction. Software implementing the collision time estimation described herein may be stored in memory 516 and may be executed on one or more programmable processors of the SOC 500.

[0050] Other embodiments

[0051] Although the invention has been described with reference to a limited number of embodiments, those skilled in the art to which this invention pertains will understand that other embodiments may be conceived without departing from the scope of the invention as disclosed herein.

[0052] For example, an embodiment in which the TTC of all detected objects is calculated has been described herein. Those skilled in the art will understand an embodiment in which the TTC of only detected objects in the trajectory of the vehicle is calculated.

[0053] In another example, an embodiment described herein in which the camera pose matrix is ​​a relative pose matrix that indicates any changes in camera translation and rotation from a previous image will be understood by those skilled in the art.

[0054] In another example, an embodiment in which a camera pose matrix is ​​calculated as part of determining a 3D point cloud has been described herein. Those skilled in the art will understand that embodiments in which an inertial measurement unit or a vehicle's odometer is used, for example, to determine the camera pose.

[0055] In another example, an embodiment using a Kalman filter has been described herein. Those skilled in the art will understand embodiments using other filter types (e.g., particle filters or average-mean-average filters).

[0056] In another example, an embodiment has been described herein in which the midpoint of the bottom of the bounding box of the detected object is used as the 2D position of the detected object. Those skilled in the art will understand that other suitable positions along the bottom of the bounding box can be used in embodiments. In these embodiments, the angle threshold used for trajectory checking may differ from the angle threshold used when using the midpoint position.

[0057] In another example, embodiments described herein assume that the detected object is another vehicle. Those skilled in the art will understand that embodiments in which the detected object may include, for example, pedestrians, bicycles, and obstacles (such as utility poles, road debris, etc.).

[0058] In another example, embodiments described herein have been referenced to automotive safety systems. Those skilled in the art will understand embodiments for other computer vision applications (e.g., industrial applications, robotics, and consumer applications such as vacuum cleaners). For example, TTC can be used to determine if an object moving on a conveyor belt is problematic. The same computing components described herein can be used to calculate the relative velocity of an object moving on a conveyor belt, where the reference plane is the conveyor belt rather than the ground plane.

[0059] As used in this article, the term "vehicle" can also be used to refer to other types of devices, such as robots, industrial devices, consumer devices, etc., in which it is beneficial to calculate TTC.

[0060] In another example, an embodiment in which the TTC estimation is implemented as software instructions executed on a processor in a multiprocessor SoC has been described herein. Those skilled in the art will understand that ground plane estimation can be implemented as any suitable combination of software, firmware, and hardware. For example, some of the functionality can be implemented using one or more hardware accelerometers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc.

[0061] Software instructions for implementing all or part of the methods described herein may first be stored in a computer-readable medium and loaded and executed by one or more processors. In some cases, the software instructions may be distributed via a removable computer-readable medium, via an emission path from a computer-readable medium on another digital system, etc. Examples of computer-readable media include non-writable storage media (e.g., read-only memory devices), writable storage media (e.g., disks, flash memory, memory), or combinations thereof.

[0062] Although method steps may be presented and described sequentially herein, one or more of the steps shown in the figures and described herein may be performed simultaneously, in combination, and / or in a different order than that shown in the figures and / or described herein. Therefore, embodiments should not be considered as limited to the specific order shown in the figures and / or described herein.

[0063] Specific terms are used throughout the description and claims to refer to specific system components. As those skilled in the art will appreciate, components in the system may be referred to by different names and / or combined in ways not shown herein without departing from the described functionality. This document is not intended to distinguish between components with different names rather than different functions. In the description and claims, the terms “comprising” and “including” are used in an open-ended manner and should therefore be interpreted as meaning “including but not limited to…”. Furthermore, the term “coupled” and its derivatives are intended to mean indirect connection, direct connection, optical connection, and / or radio connection. Thus, if a first device is coupled to a second device, the connection (for example) may be via a direct electrical connection, via an indirect electrical connection via other devices and connections, via an optical electrical connection, and / or via a wire electrical connection.

[0064] Therefore, it is believed that the appended claims will cover these modifications to the embodiments that fall within the scope of the invention.

Claims

1. A method for estimating the collision time of a detected object in a computer vision system, the method comprising: Determine the three-dimensional (3D) position of the camera in the computer vision system; The 3D position of the detected object is determined based on the 2D position of the detected object in the image captured by the camera and the estimated ground plane corresponding to the image; The relative 3D position of the camera, the velocity of the relative 3D position, and the acceleration of the relative 3D position are calculated based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object. The distance between the detected object and the camera is calculated based on the image; The collision time of the detected object is calculated based on the distance of the detected object from the camera, the velocity relative to the 3D position, and the acceleration relative to the 3D position. Determine the asymptotic angle of the detected object; An angle threshold is determined based on the distance; The progressive angle of the detected object is compared with the angle threshold to determine the location of the detected object on the trajectory of the vehicle including the camera; as well as The collision time is output in response to determining that the detected object is located on the trajectory of the vehicle.

2. The method according to claim 1, wherein the angle threshold is a percentage of a predetermined angle threshold, wherein the percentage decreases as the distance increases.

3. The method of claim 2, wherein comparing the asymptotic angle with the angle threshold comprises: If the distance is less than a first predetermined distance, then the asymptotic angle is compared with the predetermined angle threshold. If the distance is between the first predetermined distance and the second predetermined distance, then the asymptotic angle is compared with a first percentage of the predetermined angle threshold, wherein the second predetermined distance is greater than the first predetermined distance; If the distance is between the second predetermined distance and the third predetermined distance, then the asymptotic angle is compared with a second percentage of the predetermined angle threshold, wherein the third predetermined distance is greater than the second predetermined distance and the second percentage is less than the first percentage; and If the distance is greater than the third predetermined distance, then the asymptotic angle is compared with a third percentage of the predetermined angle threshold, wherein the third percentage is less than the second percentage.

4. The method of claim 3, wherein the first percentage is 75%, the second percentage is 50%, and the third percentage is 25%.

5. The method of claim 1, wherein the 2D position of the detected object is the midpoint between the bottom two corners of the bounding box of the detected object.

6. The method of claim 1, wherein the 3D position of the camera is determined based on a camera pose matrix corresponding to the image.

7. The method of claim 1, wherein calculating the collision time further comprises: The collision time is calculated based on the estimation of the velocity of the camera relative to the 3D position along the direction of the detected object and the estimation of the acceleration of the camera relative to the 3D position along the direction of the detected object.

8. A computer vision system comprising a vision processor; and a memory coupled to the vision processor, the memory including instructions executable by the vision processor, which, when executing the instructions, operate to: Capture 2D images of a scene using a single-lens reflex camera; Determine the three-dimensional 3D position of the monocular camera corresponding to the 2D image; The 3D position of the detected object is determined based on the 2D position of the detected object in the 2D image and the estimated ground plane corresponding to the 2D image. The relative 3D position of the camera, the velocity of the relative 3D position, and the acceleration of the relative 3D position are calculated based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object. The distance between the detected object and the camera is calculated based on the image; The collision time of the detected object is calculated based on the distance of the detected object from the camera, the velocity relative to the 3D position, and the acceleration relative to the 3D position. Determine the asymptotic angle of the detected object; An angle threshold is determined based on the distance; The progressive angle of the detected object is compared with the angle threshold to determine the location of the detected object on the trajectory of the vehicle including the camera; as well as The collision time is output in response to determining that the detected object is located on the trajectory of the vehicle.

9. The computer vision system of claim 8, wherein the 2D position of the detected object is the midpoint between the bottom two corners of the bounding box of the detected object.

10. The computer vision system of claim 8, wherein when the vision processor executes the instructions, it further operates to generate a motion recovery structure component corresponding to a 3D point cloud of the 2D image.

11. The computer vision system of claim 10, wherein when the vision processor executes the instructions, it further operates to generate a ground plane estimation component based on the estimated ground plane using the 3D point cloud.

12. The computer vision system of claim 8, wherein the vision processor, when executing the instructions, further operates to calculate the collision time based on a prediction of the velocity of the relative 3D position of the camera along the direction of the detected object and a prediction of the acceleration of the relative 3D position of the camera along the direction of the detected object.

13. A method for estimating the collision time of a detected object in a computer vision system, the method comprising: Calculate the distance between the detected object and the camera in the computer vision system, wherein the detected object is located in the field of view of the camera; Calculate the estimated velocity of the camera's relative 3D position along the direction of the detected object, wherein the camera's relative 3D position is relative to the 3D position of the detected object; Calculate the estimated acceleration of the camera's relative 3D position along the direction of the detected object; and The collision time is calculated based on the distance to the detected object, the estimated velocity relative to the 3D position, and the estimated acceleration relative to the 3D position. Determine the asymptotic angle of the detected object; An angle threshold is determined based on the distance; The progressive angle of the detected object is compared with the angle threshold to determine the location of the detected object on the trajectory of the vehicle including the camera; as well as The collision time is output in response to determining that the detected object is located on the trajectory of the vehicle.

14. The method of claim 13, wherein calculating the collision time further comprises: When the estimated acceleration is zero, the collision time is calculated as the distance divided by the estimated velocity; and When the inferred acceleration is non-zero, the collision time is calculated as min[(-v(+-)sqrt(v)]. 2 +2ad)) / a], where v is the estimated velocity, a is the estimated acceleration, and d is the distance.

15. The method of claim 13, further comprising: Determine the 3D position of the camera; The 3D position of the detected object is determined based on the 2D position of the detected object in a 2D image captured by the camera and the estimated ground plane corresponding to the 2D image; and The relative 3D position of the camera, the velocity of the relative 3D position, and the acceleration of the relative 3D position are calculated based on the 3D position of the camera and the 3D position of the detected object, wherein the relative 3D position of the camera is relative to the 3D position of the detected object.

16. The method of claim 15, wherein calculating the estimated velocity comprises calculating the estimated velocity as: the relative 3D position of the camera multiplied by the velocity at the relative 3D position and then divided by the distance, and calculating the estimated acceleration comprises calculating the estimated acceleration as: the relative 3D position of the camera multiplied by the acceleration at the relative 3D position and then divided by the distance.

Images