Motor vehicle system with motion detection capability

Abstract

The present invention relates to automotive systems with motion detection capabilities. A vehicle can be equipped with a camera for detecting movement of objects in the vicinity of the vehicle. The camera can be used to capture a series of images. A number of target sample points can be defined in each image. The locations of the target sample points can correspond to a driver's blind spot or other region(s) of interest. The camera can also include separate filters for generating long-term and short-term pixel intensity outputs at each of the target sample points. These long-term and short-term average outputs can be compared to determine whether a large change in intensity has occurred over a short period of time. The order of target activation can then be used to determine the direction of vehicle motion while filtering out noise or other road markings.

Description

[0001] This application is a divisional application. The parent application is entitled "Motor Vehicle System with Motion Detection Capability", filed on December 5, 2017, with application number 201711266837.0. Background Technology

[0002] Automobiles typically include standard optical rearview mirrors attached near the flaps of the driver's and passenger's doors. These conventional rearview mirrors extend from the vehicle body and are prone to collisions, especially in critical situations. Traditional rearview mirrors also have blind spots, where the driver cannot see oncoming vehicles due to the limited field of vision of the mirror.

[0003] The embodiments described herein arose within this context. Summary of the Invention

[0004] This generally relates to motor vehicle systems, and more particularly to automobiles equipped with cameras for sensing motion in their vicinity. According to an embodiment, an automobile may be provided with a camera that captures video frames. The camera may be a rear-view camera, a front-view camera, or a side-view camera. An array of target sampling points may be selected in each of the video frames. The camera may measure the brightness level and / or color at each of the target sampling points in the array.

[0005] The camera may further include a long-term averaging filter circuit and a short-term averaging filter circuit. The long-term averaging filter circuit filters the measured brightness level over a first number of frames to produce a long-term average, while the short-term averaging filter circuit filters the measured brightness level over a second number of frames to produce a short-term average, the second number of frames being at most one-tenth or one-hundredth of the first number of frames. The long-term average is used as a baseline measurement for detecting noise and road / weather conditions, while the short-term average is compared to the baseline measurement to determine whether an external object is indeed approaching the vehicle.

[0006] The camera can also be configured to analyze the sequence of arrays of target sampling points it activates. For example, the camera can determine the order in which rows in the array of target points are being activated / triggered. Depending on the sequence being triggered, the camera can help detect the presence of another vehicle in an adjacent lane, the presence of another vehicle immediately behind the vehicle, and can also sense the direction of another vehicle in front of the vehicle. In response to determining that another vehicle is moving toward the camera in a potentially dangerous manner, the camera can send an alert to the driver or guide the vehicle to take other appropriate preventative actions to avoid a collision.

[0007] Further features, properties and various advantages of the invention will become more apparent from the accompanying drawings and the following detailed description. Attached Figure Description

[0008] Figure 1 This is an illustration of an illustrative motor vehicle system with motion sensors according to an embodiment.

[0009] Figure 2 This is a perspective view of a car with an illustrative rearview camera according to an embodiment.

[0010] Figure 3 This is an illustration of an embodiment showing how a rear-view camera can include multiple target points for detecting oncoming vehicles in adjacent lanes.

[0011] Figure 4 This is an illustration of how a rear-view camera can include multiple target points for rear impact detection, according to an embodiment.

[0012] Figure 5 This is an illustration of how a forward-looking camera can include multiple target points for use in cross-traffic alerts, according to an embodiment.

[0013] Figure 6 This is a flowchart illustrating the steps for operating a motion sensor on a vehicle according to an embodiment. Detailed Implementation

[0014] Embodiments of the present invention relate to motion sensing, and more particularly to detecting motion near a motor vehicle system. Specifically, the vehicle may be equipped with a camera including a long-average filter and a short-average filter for filtering brightness measurements at selected target sampling points. The long-average filter is used to output the long-term pixel intensity at the target point, while the short-average filter is used to output the short-term pixel intensity at the target point. By comparing the short-term pixel intensity with the long-term pixel intensity, the camera will be able to filter out background noise and detect the direction of an approaching vehicle.

[0015] Those skilled in the art will recognize that this exemplary embodiment can be practiced without some or all of these specific details. In other instances, well-known operations have not been described in detail so as not to unnecessarily obscure this embodiment.

[0016] Figure 1 These are illustrations of motor vehicle systems, such as System 100. Motor vehicle system 100 can be a car, motor vehicle, electric vehicle, motorboat, machine, or other means of transportation powered by an engine. Figure 1As shown, the vehicle system 100 may include a motion sensor, such as a motion sensor 102 (e.g., an ultrasonic sensor), for detecting moving objects near the system 100. For example, the motion sensor 102 may be a camera that captures video frames. By analyzing changes in the position of objects in the video frames, the camera can be used to detect whether an external object is moving toward or away from the system 100.

[0017] According to an embodiment, system 100 may include filtering circuitry, such as a long averaging filter 104 and a short averaging filter 106. The long averaging filter 104 and the short averaging filter 106 may be infinite impulse response (IIR) filters, respectively, for measuring long-term pixel intensity and short-term pixel intensity. In other words, the long averaging filter 104 may output a brightness level averaged over a first (longer) duration, while the short averaging filter 106 may output a brightness level averaged over a second duration shorter than the first duration. As an example, the long averaging filter may produce a brightness value averaged over hundreds of frames. On the other hand, the short averaging filter may produce a brightness value averaged over only two to three frames (as an example). Generally, the duration of the long-term averaging may be at least ten times or at least one hundred times that of the short-term averaging.

[0018] Configured in this way, the long averaging filter 104 can be used to filter out "slower" changes or noise in the background, such as changes in road signs, changes in road conditions, changes in weather, and / or other non-critical environmental changes. On the other hand, the short averaging filter 106 can be used to isolate "faster" changes near the vehicle, such as when another vehicle is rapidly approaching from behind, when another vehicle is rapidly approaching from the side, when another vehicle is rapidly approaching from the front, when another vehicle is rapidly approaching the vehicle from any suitable direction, or to detect any motion relative to the vehicle.

[0019] Figure 2 This is a perspective view of a car with an illustrative rearview camera according to an embodiment. The car 200 has car doors such as doors 202, and as shown in the close-up view, a rearview camera 210 may be mounted on the door 202. Figure 2 The camera 210, mounted on the car door 202 and facing the rear of the car 200 in the example, is merely illustrative and not intended to limit the scope of this embodiment. The camera 210 may be mounted on any other car door, in the trunk, on the front hood, on the dashboard, on the front or rear bumper, on the door handle, on the car body, or on any other suitable part of the car 200 facing any desired direction, if desired.

[0020] Figure 3This is an illustration of an embodiment showing how a rear-view camera can include multiple target sampling points for detecting an approaching vehicle in an adjacent lane. Figure 3 As shown, the rearview camera is capable of capturing objects in at least the current lane 302 over which the car 200 is currently traveling, as well as objects in the adjacent lane 304. Specifically, each video frame 300 may include at least an array 308 of target points for detecting cars in the adjacent lane 304. The positions of the target points 308 may be strategically selected such that they correspond to the driver's "blind spot" in the context of a conventional optical rearview mirror, or may be dynamically adjusted over time to adapt to changing conditions (e.g., to account for lane width changes, the car's current speed, or other factors).

[0021] Array 308 may include a first row of points 310-1, a second row of points 310-2, a third row of points 310-3, and a fourth row of points 310-4. Brightness measurements can be obtained at these target points. These measurements can then be filtered using a long-average filter to identify road conditions, and also using a short-average filter to identify objects, such as cars, moving "fast" through adjacent lanes 304.

[0022] For example, a car such as car 306 can be detected in adjacent lane 304. In one scenario, row 310-1 can sense a car entering the array, row 310-2 can then sense a car moving through the array, then row 310-3 can be triggered, and then row 310-4 can be triggered. When this particular activation sequence is detected, the camera will know that car 306 is rapidly approaching from behind. In another scenario, row 310-4 can sense a car entering the array, row 310-3 can then sense a car moving through the array, then row 310-2 can be triggered, and then row 310-1 can be triggered. When this particular activation sequence is detected, the camera will know that car 200 (i.e., the car on which the camera is mounted) is actually overtaking car 306 in the adjacent lane. In yet another scenario, car 306 may be moving around within array 308 by chance (i.e., randomly triggering row 310-1 or row 310-4), which could indicate that car 306 is moving at approximately the same speed as car 200 and is hovering in the driver's blind spot.

[0023] Still referencing Figure 3Video frame 300 may also include an additional set of points 314 for detecting vehicles (such as car 312) further behind in adjacent lanes 304. Points 314 may optionally be organized into multiple rows as described above to help determine the directionality of car 312. This can be helpful in cases where car 312 is moving very rapidly toward car 200, giving array 308 an even better chance of predicting the movement of upcoming vehicles in adjacent lanes.

[0024] In some embodiments, it may also be desirable to detect the movement of a vehicle (such as vehicle 320) immediately following vehicle 200. Figure 4 This illustration shows how a video frame 400 captured by a rearview camera can include multiple target points for detecting a rear-end collision. This particular camera can be mounted on the trunk, rear bumper, or any other part of the vehicle's body. Figure 4 As shown, frame 400 may include target point array 412. In particular, array 412 may be divided into a first row 410-1 of points, a second row 410-2 of points, and a third row 410-3 of points.

[0025] Brightness or light intensity measurements can be taken at each of these target points. A long-average filter can then be used to filter the intensity measurements to identify road conditions, and a short-average filter can also be used to filter the intensity measurements to identify “fast-moving” objects, such as a car rapidly approaching from behind.

[0026] For example, a car such as car 402 can be detected from behind in the same lane. In one scenario, row 410-1 can sense a car entering the array, row 410-2 can then sense a car moving through the array, and then trigger row 410-3. When this particular activation sequence is detected, the camera will know that car 402 is rapidly approaching from behind, potentially posing a rear-end collision risk. In another scenario, array 412 can sense a car leaving array 402, indicating that car 402 is decelerating relative to the camera or changing lanes. In yet another scenario, car 402 may be accidentally moving around within array 412 (i.e., randomly triggering row 410-1 or row 410-3), which could indicate that car 402 is moving at approximately the same speed as car 200 and is following closely from behind (and potentially dangerously close).

[0027] In other suitable arrangements, it may also be desirable to detect the movement of vehicles moving across the lane from the front. Figure 5This is an illustration showing how video frames 500 captured by a rearview camera can include multiple target points for use in cross-traffic alerts. The particular camera can be mounted on the hood, front bumper, or any other part of the vehicle's body. Figure 5 As shown, frame 500 may include an array of target points 512. Specifically, array 512 may be divided into a first column 510-1 of points, a second column 510-2 of points, and a third column 510-3 of points. Luminance or light intensity measurements can be obtained at each of these target points. A long averaging filter can then be used to filter the luminance measurements to identify road conditions, and a short averaging filter can also be used to filter the luminance measurements to identify “fast-moving” objects, such as a car moving quickly from left to right (or vice versa) from the driver's perspective.

[0028] For example, a car (such as car 502) moving from left to right in the direction of arrow 503 can be detected. In this scenario, column 510-1 can sense a car entering the array, column 510-2 can then sense a car moving through the array, and then trigger column 510-3. When this particular activation sequence is detected, the camera will know that car 502 is moving rapidly from left to right. A similar array of target points (not shown) can also be included on the right side of frame 500 to help detect cars moving rapidly from right to left.

[0029] Figure 3-5 The exemplary configuration of the target points shown is merely illustrative and is not intended to limit the scope of this embodiment. Generally, the motor vehicle system 100 ( Figure 1 The system may include a camera 102 for capturing video frames with any desired number of target points arrays, wherein each array may include any appropriate number of rows or columns for sensing target points moving across frames in any desired direction relative to the system 100.

[0030] Figure 6 This is a flowchart illustrating the steps for operating a motion sensor, such as camera 102 on car 100. At step 600, a desired region in the video frame can be selected for motion detection (e.g., several points can be selected for measuring brightness levels in the camera). For example, a region of a target point can be selected for detecting an approaching car in an adjacent lane (see example...). Figure 3 As another example, a region of the target point can be selected for back-end collision detection (see example...). Figure 4 As yet another example, a region of the target point can be selected for detecting intersecting traffic (see example...). Figure 5 ).

[0031] At step 602, the long averaging filter 104 can be used to obtain a baseline average measurement as the vehicle 100 is moving. This baseline average measurement is used to detect slowly changing road or weather conditions (as an example).

[0032] At step 604, the short-average filter 106 can be used to obtain a short-term average measurement as the vehicle 100 is moving. This short-term average is used to detect rapidly changing and potentially unsafe conditions, such as when the vehicle or other external obstacles are moving toward the vehicle 100 at a rapid speed (relative to the baseline measurement).

[0033] At step 606, the signal processor associated with the camera (e.g., it may be part of the camera unit or a signal processor of a separate component) can calculate the difference between the short-term average measurement and the baseline average measurement. If the calculated difference exceeds a predetermined threshold level, the camera can determine whether the target point has been activated in a specific sequence (step 608).

[0034] If a potentially dangerous sequence has been detected, the camera can issue an alert to the driver and take other preventative actions (step 610). For example, if the camera detects that a car is rapidly approaching from an adjacent lane, the camera can issue an alert to the driver so that the driver can decide not to change lanes, or the car can actually prevent the driver from changing lanes (e.g., by mechanically locking the steering wheel to prevent a potential collision).

[0035] As another example, if a camera detects that a car is rapidly approaching from behind, the camera can issue an alert to the driver so that the driver can avoid a rear-end collision in some way, or if a collision is unavoidable, the car can take other appropriate actions to minimize the impact (e.g., by applying the brakes in a smart way so that the car does not spin out of control or cause the car to come to an abrupt stop).

[0036] As yet another example, if a camera detects a car rapidly approaching from an intersecting lane (such as in...) Figure 5 In the intersection scenario shown, the camera can issue another warning to the driver so that the driver can avoid entering the intersection in an inappropriate way, or if the driver is unaware, the car can automatically apply the brakes to prevent the car from entering the intersection.

[0037] These steps are illustrative only. Existing steps may be modified or omitted; some of the steps may be performed in parallel; additional steps may be added; and the order of some steps may be reversed or changed.

[0038] The embodiments described herein, which use only two different filters to detect motion, are illustrative only and are not intended to limit the scope of the invention. More than two filters may be used if desired. In other suitable embodiments, at least three filters of different sensitivities may be used, at least four filters of different sensitivities may be used, at least five filters of different sensitivities may be used, and so on.

[0039] The foregoing description only illustrates the principles of the present invention, and various modifications can be made by those skilled in the art. The embodiments described above can be implemented individually or in any combination.

Claims

1. A motor vehicle system capable of operating in a first lane, comprising: A camera, operable to capture video frames of objects outside the vehicle system; as well as Circuit, which is used for: Based on video frames from the camera, the presence of vehicles in the second lane adjacent to the first lane is detected; The predicted movement of the vehicle is determined based on video frames from the camera; as well as An alarm is issued in response to the predicted movement of the vehicle. The circuit is configured to determine the predicted movement of the vehicle by applying a long-averaging filter and a short-averaging filter to an array of points in each video frame, and by comparing a baseline measurement from the long-averaging filter with a short-term measurement from the short-averaging filter, wherein the long-averaging filter outputs an average brightness level over a first duration, and wherein the short-averaging filter outputs an average brightness level over a second duration, which is shorter than the first duration. The camera is configured to capture frames with multiple target points arranged in an array of rows and columns, and the camera analyzes the sequence of activation of the target points in the array to determine whether other vehicles are approaching the motor vehicle system.

2. The automotive system of claim 1, wherein, The vehicle is detected behind the motor vehicle system in the second lane, and the predicted movement of the vehicle includes the movement of the vehicle toward the motor vehicle system.

3. The automotive system of claim 1, wherein, The vehicle is detected in front of the motor vehicle system, and the predicted movement of the vehicle includes movement of the vehicle toward the first lane.

4. The automotive system of claim 1, wherein, The circuit is also used for: The presence of a second vehicle in the first lane is detected based on video frames from the camera. Based on video frames from the camera, determine the second predicted movement of the second vehicle; and An alarm is issued in response to a second predicted movement of the second vehicle.

5. The automotive system of claim 4, wherein, The second vehicle is detected behind the motor vehicle system, and the second predicted movement of the second vehicle includes movement of the second vehicle toward the motor vehicle system.

6. The automotive system of any one of claims 1-5, wherein, The camera is installed in at least one of the following: a vehicle door, trunk, hood, dashboard, front or rear bumper, door handle, and housing of a motor vehicle system.

7. The automotive system of any one of claims 1-5, wherein, The circuit further enables the motor vehicle system to take preventative actions in response to predicted movement of the vehicle.

8. The automotive system of claim 7, wherein, The preventative action includes applying the brakes.

9. An apparatus installed on a motor vehicle system, the motor vehicle system having a camera and being operable in a first lane, the apparatus comprising: Circuit, which is used for: Based on video frames from the camera, the presence of vehicles in the second lane adjacent to the first lane is detected; The predicted movement of the vehicle is determined based on video frames from the camera; as well as An alarm is issued in response to the predicted movement of the vehicle. The circuit is configured to determine the predicted movement of the vehicle by applying a long-averaging filter and a short-averaging filter to an array of points in each video frame, and by comparing a baseline measurement from the long-averaging filter with a short-term measurement from the short-averaging filter, wherein the long-averaging filter outputs an average brightness level over a first duration, and wherein the short-averaging filter outputs an average brightness level over a second duration, which is shorter than the first duration. The camera is configured to capture frames with multiple target points arranged in an array of rows and columns, and the camera analyzes the sequence of activation of the target points in the array to determine whether other vehicles are approaching the motor vehicle system.

10. The apparatus of claim 9, wherein, The vehicle is detected behind the vehicle system in the second lane, and the predicted movement of the vehicle includes the movement of the vehicle toward the vehicle system.

11. The apparatus of claim 9, wherein, The vehicle is detected in front of the motor vehicle system, and the predicted movement of the vehicle includes movement of the vehicle toward the first lane.

12. The apparatus of claim 9, wherein, The circuit is also used for: The presence of a second vehicle in the first lane is detected based on video frames from the camera. Based on video frames from the camera, determine the second predicted movement of the second vehicle; and An alarm is issued in response to a second predicted movement of the second vehicle.

13. The apparatus of claim 12, wherein, The second vehicle is detected behind the motor vehicle system, and the second predicted movement of the second vehicle includes movement of the second vehicle toward the motor vehicle system.

14. The apparatus of any one of claims 9-13, wherein, The circuit further enables the motor vehicle system to take preventative actions in response to the predicted movement of the vehicle.

15. The apparatus of claim 14, wherein, The preventative action includes applying the brakes.

16. A method for a motor vehicle system, comprising: The presence of a vehicle in a second lane adjacent to the first lane in which the vehicle system is operating is detected by means of a circuit and based on video frames from a camera on the vehicle system. The predicted movement of the vehicle is determined by the circuit and based on video frames from the camera; as well as The circuit issues an alarm in response to a predicted movement of the vehicle. The determination of the predicted movement of the vehicle includes: Apply a long averaging filter and a short averaging filter to the array of points in each video frame, and The baseline measurement from the long averaging filter is compared with the short-term measurement from the short averaging filter, wherein the long averaging filter outputs the average brightness level over a first duration, and wherein the short averaging filter outputs the average brightness level over a second duration, which is shorter than the first duration. The camera is configured to capture frames with multiple target points arranged in an array of rows and columns, and the camera analyzes the sequence of activation of the target points in the array to determine whether other vehicles are approaching the motor vehicle system.

17. The method of claim 16, wherein, The vehicle is detected behind the motor vehicle system in the second lane, and the predicted movement of the vehicle includes the movement of the vehicle toward the motor vehicle system.

18. The method of claim 16, wherein, The vehicle is detected in front of the motor vehicle system, and the predicted movement of the vehicle includes movement of the vehicle toward the first lane.

19. The method of claim 16, further comprising: The presence of a second vehicle in the first lane is detected by the circuit and based on video frames from the camera. The second predicted movement of the second vehicle is determined by the circuit and based on video frames from the camera; as well as An alarm is issued by the circuit in response to a second predicted movement of the second vehicle.

20. The method of claim 19, wherein, The second vehicle is detected behind the motor vehicle system, and the second predicted movement of the second vehicle includes movement of the second vehicle toward the motor vehicle system.

21. The method according to any one of claims 16-20, further comprising: The circuit enables the motor vehicle system to take preventative action in response to a predicted movement of the vehicle.

22. The method of claim 21, wherein the preventive action includes applying a brake.

23. An apparatus for a motor vehicle system, comprising means for performing the method of any one of claims 16 to 22.

Images