Camera system for vehicle, application of camera system and vehicle

Abstract

The invention relates to a camera system (10) for three-dimensional object detection of a vehicle (F) having a front and / or rear windshield (S), comprising at least two, preferably four cameras (1), which are arranged on the inside of the front or rear windshield (S) of the vehicle (F), the front or rear windshield (S) has edge regions on both sides, and at least one camera (1) is arranged in each of the two edge regions. The invention also relates to the use of the camera system (10) according to the invention and to a vehicle (F).

Description

Technical Field

[0001] This invention relates to a camera system for vehicles. The invention also relates to applications of this camera system and vehicles having the camera system according to the invention. Background Technology

[0002] Driver assistance and autonomous driving systems in vehicles include optical systems, typically camera systems, whose recorded data undergoes digital processing. Computer-aided object recognition holds particular significance in digital processing, enabling the identification of other vehicles, pedestrians, and other objects within recorded camera images. The design of the camera systems used for this purpose involves compromises between various conflicting design goals, including: the largest possible field of view (FoV), the widest possible operating range and / or spectral range, sensitivity, and temporal resolution.

[0003] In the field of camera systems, stereo camera systems (i.e., systems with two or more cameras) play a particularly important role in safety-critical tasks, such as automated emergency braking. The main advantage of stereo camera systems compared to monocular camera systems lies in the possibility of performing 3D object detection and scene tracking. This provides the possibility of direct 3D measurements using fundamental geometric relationships, as can be utilized in a single—notably calibrated—stereo camera system.

[0004] However, due to the large space requirements, high cost, and extremely high requirements for the calibration quality of different cameras, such camera systems have not yet been widely adopted. Furthermore, there is currently no suitable alternative that meets the complex safety requirements of autonomous driving.

[0005] A major problem with using stereo camera systems in the field of autonomous driving is the limited distance at which 3D object detection and scene tracking can be achieved. For a typical stereo camera system with 1.2 megapixel resolution, a 50-degree horizontal field of view, and a camera spacing of about 12 cm, this distance is approximately 55 meters.

[0006] Therefore, the objective of this invention is to provide a camera system that increases the distance for reliable 3D object detection and scene tracking. To address this objective, a camera system is proposed according to this invention. Further preferred embodiments are described below. Furthermore, applications and vehicles based on the camera system according to this invention are also provided. Summary of the Invention

[0007] A camera system for 3D object detection of a vehicle with a windshield and / or rear windshield is proposed, comprising at least two, preferably four, cameras arranged inwards on the windshield or rear windshield of the vehicle. The windshield or rear windshield has edge regions on both sides, and at least one camera is arranged in each of the two edge regions. This arrangement ensures the largest possible spacing between the cameras, enabling the detection of parallax displacement of objects at greater distances in space. In this way, 3D object detection and 3D scene tracking can be achieved at large distances via cameras. Preferably, four cameras are used, arranged in pairs on the left and right edge regions of the windshield or rear windshield. Preferably, one camera is arranged in the upper glass region and one camera is arranged in the lower glass region on the left and right edge regions of the windshield or rear windshield (viewed along the forward direction of the vehicle). In this case, six independent parallax measurement possibilities are provided—all possible combinations of paired cameras. Preferably, one side of the edge region is defined by the edge of the windshield or rear windshield that intersects with the body components.

[0008] Furthermore, it is proposed that the cameras are respectively arranged in the overlapping area of ​​the corresponding edge region and the cleaning area (R) of at least one windshield wiper. With this preferred embodiment, the camera's field of view is not limited by rainwater. The camera's field of view also benefits from the cleaning mechanisms of the commonly provided front or rear windshield, which are typically achieved through the wiper assembly.

[0009] Furthermore, each camera is mounted on a separate holding device. Compared to using a common, rigid holding device (as is typically used to hold sensitive camera calibration), this preferred implementation enables a wider range of camera arrangements, thereby sensing parallax displacement at greater distances, and correspondingly enabling 3D object detection and scene tracking at greater distances.

[0010] Abandoning a common, rigid holding mechanism negatively impacts camera calibration. In particular, everyday thermal and mechanical forces can cause minute displacements between cameras, making one-off calibrations unreliable. Therefore, this proposed camera system incorporates a computing unit configured to execute a dedicated hardware algorithm for feature matching. Utilizing this preferred implementation, compensation effects are achieved based on a single image, typically through calibration. Here, the algorithm overlays contours and object edges identified as identical on a single image to generate reference points required for parallax measurement. The term "dedicated hardware algorithm for feature matching" describes a specific algorithm executed on a hardware-optimized computer architecture to efficiently and in real-time identify and match features between image data.

[0011] It is also proposed that at least one camera has a 15-degree horizontal field of view. In pursuing the ability to achieve 3D object detection and scene tracking at long distances, this physically sacrifices near-field resolution for higher clarity at greater distances. A 15-degree horizontal field of view provides the highest possible resolution at long distances while covering a medium distance equivalent to approximately three highway lanes, which is sufficient for typical applications in vehicle sensing.

[0012] Furthermore, it is proposed that at least one camera has a telephoto lens and an image sensor with a resolution of at least two megapixels. Using a telephoto lens increases the distance at which objects can be resolved and identified and tracked within a 3D object detection framework. This also applies to using an image sensor with sufficient resolution. For an exemplary camera arrangement (with a 1.3-meter spacing between cameras, a 15-degree horizontal field of view, a 16:9 image aspect ratio, a 2.1-micron pixel pitch, and an f-number of 1.6), the theoretical maximum distances for effective 3D object detection and scene tracking for different image sensor resolutions are listed below. Here, resolution is in megapixels, focal length f is in pixels, and theoretical distance d is in meters.

[0013] The listed distances can only be achieved in practice when image stabilization and motion blur compensation measures are taken.

[0014] Therefore, in an extended embodiment of the invention, it is proposed that at least one camera has a controllable lens and / or a controllable image sensor. This preferred embodiment stabilizes images generated in moving scenes and / or when the camera is moving. In this way, sufficiently clear imaging is achieved even at large distances, allowing the results to be used for 3D object detection and scene tracking. The controllable lens or controllable image sensor is a lens or image sensor equipped with adjustable electromechanical devices, enabling it to perform motion compensation.

[0015] It is also proposed that at least one camera is gimbaled. This preferred implementation stabilizes the images recorded by each camera in the camera system, which reduces motion blur when the camera system and / or scene are in motion and thus improves object tracking and 3D scene capture over long distances, even in low ambient light conditions (and therefore requiring longer exposure times).

[0016] Furthermore, the gimbal suspension device is implemented as a gimbal, wherein the gimbal has at least one servo motor for performing rotational movements about the roll axis and pitch axis, respectively. Using this preferred embodiment, camera motion and / or scene motion can be actively compensated, resulting in less motion blur over longer exposure times and consequently, images with more robust object recognition. This, in turn, enhances 3D object detection and scene tracking. The gimbal is a rotatably suspended mechanical arm on at least two axes, suitable for compensating for the motion of optical equipment.

[0017] Furthermore, it is proposed that the gimbal has an inertial measurement unit (IMU) for measuring spatial acceleration. Using the IMU (preferably mounted on the camera), the motion of the camera system and / or the camera itself can be measured. Based on this measurement data, compensating motion can be determined for the occurring motion and executed via servo motors, thereby stabilizing the image.

[0018] Furthermore, it is proposed that the gimbal can be adjusted via a closed-loop control circuit. Using this preferred embodiment, motion blur is significantly reduced when the camera system moves and / or the scene moves. In particular, when using the closed-loop control circuit, specific maximum permissible motion, motion blur, or the required exposure time under given motion blur conditions can be set. This improves image stability, reduces motion blur, and enables sharper images at greater distances.

[0019] It is also proposed that the intersection of the roll axis and the pitch axis coincides with the perspective center of the camera. This preferred embodiment ensures that any rotational movement of the camera caused by the motorized arm lies within the image plane, thereby avoiding parallax shift effects.

[0020] Furthermore, the gimbal also features at least one servo motor for performing rotational motion about a yaw axis, which intersects the intersection of the roll and pitch axes. Adding a yaw axis enables compensated motion in a third spatial dimension. Therefore, the image can be fully stabilized for each spatial motion direction.

[0021] In an extended embodiment of the invention, it is proposed that the gimbal has an active control device, by which the camera can rotate beyond the camera system and / or the camera's motion compensation. Using this preferred embodiment, an artificial expansion of the field of view can be achieved by actively rotating the camera about an axis and sequentially capturing a wide scene. This rotation is preferably performed about a yaw axis.

[0022] Furthermore, applications are proposed for using the camera system according to the invention as a data source for driving assistance systems and / or systems for autonomous driving. Such applications provide the aforementioned advantages to driving assistance systems and / or systems for autonomous driving.

[0023] Furthermore, a vehicle including a camera system according to the present invention is proposed. This vehicle has the advantages described above. Attached Figure Description

[0024] The invention will now be described in more detail with reference to the accompanying drawings. The drawings show: Figure 1 : A schematic diagram of the camera system according to the present invention; Figure 2 : A schematic diagram of a camera in the camera system according to the present invention. Detailed Implementation

[0025] Figure 1 A camera system 10 according to the invention is shown, comprising four cameras 1. In the present example, they are arranged on the inner side of the windshield S of a vehicle F. The windshield S has left and right edge regions, and the cameras 1 are arranged in the left and right edge regions, respectively. Here, the cameras 1 are arranged such that they are located within the cleaning area R of the vehicle's windshield wipers. The maximum spacing between the cameras 1 is denoted here by the width b, and the vertical spacing between the cameras 1 is denoted by the height h. The larger the width b and height h, the greater the distance at which 3D object detection and scene tracking can be reliably achieved. The presented configuration provides six independent parallax measurement possibilities by comparing the images of every two cameras: below the driver's side and below the passenger's side, below the driver's side and above the passenger's side, above the driver's side and below the passenger's side, above the driver's side and above the passenger's side, above the driver's side and above the passenger's side, above the driver's side and below the passenger's side, and above the passenger's side and below the passenger's side.

[0026] Figure 2A camera 1 of a camera system 10 according to the present invention is shown. The camera 1 includes a telephoto lens 2 and an image sensor 3, suspended on a gimbal 4. The gimbal 4 has two servo motors 6 for transmitting rotational motion to the camera 1 on two axes (7a, 7b). One servo motor 6 is designed to perform rotational motion about the roll axis 7a of the camera 1, and the second servo motor is designed to perform rotational motion about the pitch axis 7b of the camera 1. The intersection of the roll axis 7a and the pitch axis 7b coincides with the perspective center 9 of the camera 1. Furthermore, the camera 1 has an inertial measurement unit 8. For mounting to an external support, the motor arm 4 also has a base 5. When the camera system 10 and / or the camera 1 moves, the inertial measurement unit 8 captures the direction and magnitude of motion. Based on the measurements, the servo motors 6 initiate corresponding compensating rotational motions about the roll axis 7a and the pitch axis 7b to stabilize the image despite the presence of motion.

Claims

1. A camera system (10) for three-dimensional object detection of a vehicle (F) having a windshield and / or a rear windshield (S), comprising at least two, preferably four, cameras (1) arranged on the inner side of the windshield or rear windshield (S) of the vehicle (F), wherein, The front windshield or rear windshield (S) has edge regions on both sides, and at least one camera (1) is arranged in each of the two edge regions.

2. The camera system (10) according to claim 1, characterized in that, The cameras (1) are respectively arranged in the overlapping area of ​​the corresponding edge area and the cleaning area (R) of at least one windshield wiper.

3. The camera system (10) according to any one of the preceding claims, characterized in that, Each camera (1) is mounted on a separate holding device.

4. The camera system (10) according to any one of the preceding claims, characterized in that, The camera system (10) has a computing unit configured to execute a dedicated hardware algorithm for feature matching.

5. The camera system (10) according to any one of the preceding claims, characterized in that, At least one camera (1) has a horizontal field of view of 15 degrees.

6. The camera system (10) according to any one of the preceding claims, characterized in that, At least one camera (1) has a telephoto lens (2) and an image sensor (3) having a resolution of at least two million pixels.

7. The camera system (10) according to any one of the preceding claims, characterized in that, At least one camera (1) has a controllable lens and / or a controllable image sensor (3).

8. The camera system (10) according to any one of the preceding claims, characterized in that, At least one camera (1) is mounted by a universal joint.

9. The camera system (10) according to claim 8, characterized in that, The universal joint suspension device is implemented as a gimbal (4), wherein the gimbal (4) has at least one servo motor (6) for performing rotational movements about the roll axis and the pitch axis (7a, 7b) respectively.

10. The camera system (10) according to claim 9, characterized in that, The gimbal (4) has an inertial measurement unit (8) for measuring spatial acceleration.

11. The camera system (10) according to any one of claims 9 or 10, characterized in that, The gimbal (4) can be adjusted through a closed-loop control circuit.

12. The camera system (10) according to any one of claims 9 to 11, characterized in that, The intersection of the roll axis (7b) and the pitch axis (7a) coincides with the perspective center (9) of the camera (1).

13. The camera system (10) according to any one of claims 9 to 12, characterized in that, The gimbal (4) also has at least one servo motor (6) for performing rotational motion about a yaw axis, wherein the yaw axis intersects the intersection of the roll axis and the pitch axis (7a, 7b).

14. The camera system (10) according to any one of claims 9 to 13, characterized in that, The gimbal (4) has an active control device, which enables the camera (1) to rotate beyond the camera system (10) and / or the motion compensation of the camera (1).

15. The application of the camera system (10) according to any one of the preceding claims as a data source for a driving assistance system and / or a system for autonomous driving.

16. A vehicle (F) comprising a front windshield and / or a rear windshield (S) and a camera system (10) according to any one of claims 1 to 14.

Images