Device and method for detecting at least one object

Abstract

Device for detecting at least one object, with a camera comprising a first image acquisition sensor (4), with a stereo camera comprising a second image acquisition sensor (2) and a third image acquisition sensor (3), and with an evaluation unit, wherein the evaluation unit detects the image of the object (21) in an image captured by the camera and determines the position of the image of the object (21), the evaluation unit defines an image area of ​​an image pair captured with the aid of the stereo camera depending on the determined position of the object (21) in the image captured with the aid of the camera, and the evaluation unit processes the defined image area to detect an image of the object (21), characterized in that that the evaluation unit detects the image of the object (21) in each image of a sequence of images captured by the camera and determines the positions of the images of the object (21) in the images, wherein a tracking function is stored in the evaluation unit, with the help of which the image of the object (21) is determined in the images of the sequence and the image of the object (21) is determined via the images of the sequence, and that the evaluation unit defines an image area of ​​an image pair captured with the aid of the stereo camera depending on at least two determined positions of the object (21) in at least two images of the image sequence captured with the aid of the camera.

Description

[0001] The invention relates to a device for detecting at least one object. The device comprises a camera including a first image acquisition sensor, a stereo camera including a second and a third image acquisition sensor, and an evaluation unit. The evaluation unit detects the image of the object in an image captured by the camera and determines the position of the object's image within the image. The invention further relates to a method for detecting at least one object.

[0002] The device and method are used particularly in driver assistance systems. In these systems, stereo or mono cameras are used to detect objects, capturing images of a detection area. Objects are detected within these images, and the detected objects are classified into object classes. Predefined detection algorithms are executed for this purpose. To achieve sufficiently high detection and classification accuracy, high-quality detection algorithms must be used. Such high-quality detection algorithms require a relatively long time for object detection and classification, as well as significant memory. While using lower-quality detection algorithms reduces both the required detection speed and memory usage, it also reduces the overall accuracy.However, there is a risk that objects may not be detected and / or may be misclassified, potentially causing the driver assistance system to provide incorrect information to the driver or other units controlled by the system. This could potentially compromise road safety.

[0003] Document DE 100 33 355 A1 discloses a stereo camera with two image sensors which are read out pixel by pixel in a manner controlled by a readout clock signal generator.

[0004] Document DE 10 2004 061 998 A1 discloses a device for a motor vehicle with a stereo camera system comprising a first camera and a second camera, wherein the first camera and the second camera differ in at least one camera characteristic. In particular, the light sensitivity of the first camera and the light sensitivity of the second camera differ.

[0005] Document DE 42 19 851 A1 discloses a stereo camera comprising a single object lens, a single imaging lens, and a twin prism between these two lenses. Preferably, at least the imaging lens is a compound lens.

[0006] Document DE 44 10 358 A1 discloses a stereo camera with electromechanically actuated shutters and electromechanically actuated apertures. It features a single control electronics unit with two outputs. Both shutter stepper motors are connected to one output, and the aperture stepper motors to the other.

[0007] Document DE 100 25 593 A1 discloses a stereo camera in which the distance between the optical axes of the recording objects is set narrower than the conventional distance between the optical axes.

[0008] DE 101 54 861 A1 discloses a method for providing image information in which a depth image of a monitored area is determined to ascertain the positional information of objects in different detection planes. This information can be used to identify the object detected with the aid of a depth image within an area of ​​an image recorded by a video system. The determined data from the depth image and the video image can then be correlated.

[0009] US patent 20050232463 A1 discloses a method and device for detecting a target in an image, primarily for collision avoidance in vehicles. This involves generating multiple depth images and comparing a multitude of target templates with at least one of these depth images. Based on this comparison, a "scores" image is generated to classify a target.

[0010] US patent 2009043440 A1 discloses an autonomous vehicle and an associated control unit configured to regulate the vehicle's movement and detect obstacles. The control unit stores information on the temporary positional fluctuations of obstacles, enabling it to define a virtual obstacle area for the expected movement of the obstacle over a specific period. This allows the device to perform safe evasive maneuvers even in the face of unpredictable movements of people.

[0011] The object of the invention is to provide a device and a method for detecting at least one object, in which the object can be detected quickly and reliably with low storage requirements.

[0012] The problem is solved by a device having the features of claim 1 and by a method having the features of the independent method claim. Advantageous embodiments of the invention are specified in the dependent claims.

[0013] By processing only a defined area of ​​the stereo image pair, determined by the object's position within the camera's image, to detect an image of the object, significantly more resource-efficient image data processing is achieved compared to processing the entire image pair. This results in faster detection with reduced memory requirements. The detection rate, which indicates the number of objects detected per unit of time, is also increased. The faster object detection within the stereo image pair allows for quicker object classification and / or faster determination of one or more of the object's properties.In the further processing of the properties of the object and / or the classified object, for example in a driver assistance system, the vehicle's control units can be activated more quickly. The determination of the image area to be processed from the stereo camera's image pair, depending on the object's position within the image captured by the camera, is carried out in such a way that the image area is selected in which the object is most likely to be depicted, based on its position within the image captured by the camera.

[0014] It is advantageous if corresponding image data is generated from the images captured by the image acquisition sensors. This image data is then processed or further processed, particularly with the aid of the evaluation unit.

[0015] It is advantageous if the stereo camera is a second stereo camera and if the camera includes a fourth image acquisition sensor and is a first stereo camera. By using two stereo cameras, the distance of an object detected in the image to the device can be determined using each stereo camera in only one image pair captured by the stereo camera.

[0016] It is advantageous if the evaluation unit determines at least one property of the object, in particular an object class to which the object belongs, based on the object's representation in the defined image area. It is especially advantageous if the evaluation unit generates an output signal containing information about the object, depending on the object's property, and a control unit is provided for controlling at least one actuator based on the evaluation unit's output signal. This allows, for example, the simple control of the headlight output, a seatbelt pretensioner, a brake assist, a light assist, a speed assist, a lane change assist, and / or a headrest in a driver assistance system.Alternatively or additionally, the output signal or information corresponding to the output signal can be output to an operator of the device via a human-machine interface, in particular via a display unit.

[0017] In a preferred embodiment of the invention, the evaluation unit executes a first algorithm when detecting the image of the object in the image captured by the camera, and a second algorithm when detecting the image of the object in the defined image section of the image pair captured by the stereo camera. The second algorithm has a higher detection accuracy than the first algorithm. The first algorithm is preferably one that has low detection accuracy but is fast and requires relatively few resources, particularly relatively little memory. The second algorithm is preferably one with high detection accuracy that requires many resources compared to the first algorithm and is therefore relatively slow. In particular, the second algorithm requires more memory than the first algorithm.The first algorithm performs only a rough detection of objects in the image captured by the camera. The second algorithm detects objects within the image area where the images of the objects detected by the camera are highly likely to be represented. The objects are then classified based on their images detected within this defined image area. This ensures reliable classification of the detected objects. Because only a relatively small area of ​​the stereo image pair captured by the camera needs to be evaluated using the second algorithm, the required storage space is significantly reduced compared to evaluating the entire stereo image pair using the second algorithm.

[0018] The evaluation unit detects the object's representation in each image of a sequence captured by the camera and determines the object's position within those images. Based on at least two determined object positions in at least two images of the sequence, the evaluation unit defines a specific area within a pair of images captured by the second stereo camera. By capturing a sequence of images and detecting the object's representation in multiple images within that sequence, object tracking can be performed. This allows the search area within the stereo camera's image pair to be defined in such a way as to maximize the likelihood of the object being detected, and / or to select a smaller image area.

[0019] Alternatively or additionally, when the device is used in a driver assistance system, the vehicle's movement, particularly cornering, can be determined. This allows, for example, early detection of when an object will not be displayed in the image pair captured by the stereo camera due to the vehicle cornering, thus eliminating the need to define a specific image area to be searched.

[0020] In a preferred embodiment of the invention, the image acquisition sensors and / or the evaluation unit are mounted on a circuit board, in particular a printed circuit board made of epoxy resin. The image acquisition sensors and the evaluation unit are permanently connected to the circuit board, so that the relative positions of the image acquisition sensors are fixed and unchangeable. Furthermore, the image acquisition sensors and the evaluation unit are electrically connected to the circuit board. The fixed relative position of the image acquisition sensors reduces the calibration effort. Calibration is only necessary once during the installation of the image acquisition sensors; recalibration during operation of the device is not required.

[0021] The image acquisition sensors are preferably optical semiconductor sensors, CCD image acquisition sensors, and / or CMOS image acquisition sensors. Such sensors are easy and inexpensive to manufacture. Furthermore, the overall manufacturing costs of the device are reduced.

[0022] Furthermore, it is advantageous if a first and a second lens are arranged such that an image of a detection area of ​​the first stereo camera is projected and / or focused onto a corresponding image detection area of ​​the image detection sensors of the first stereo camera, and if a third and a fourth lens are arranged such that an image of a detection area of ​​the second stereo camera is projected and / or focused onto a corresponding image detection area of ​​the image detection sensors of the second stereo camera. The lenses can be permanently attached to the circuit board and / or arranged in a housing in which the device is received.

[0023] The first and second lenses are preferably identical. Likewise, the third and fourth lenses are preferably identical. It is particularly advantageous if the first and second lenses have a smaller angle of view than the third and fourth lenses. The angle of view of the first and second lenses is preferably in the range of 15° to 25°, and the angle of view of the third and fourth lenses in the range of 45° to 65°. This ensures that the first stereo camera has a longer focal length than the second stereo camera. Thus, the first stereo camera captures a distant area, while the second stereo camera captures a close-up area in front of the device.In this way, objects can be detected at a great distance from the device using the first stereo camera and later classified in the image pair captured by the second stereo camera as soon as the object is in close proximity. For this purpose, the image pair of the first stereo camera is acquired before the image pair of the second stereo camera.

[0024] Furthermore, it is advantageous to provide at least one infrared light source, in particular with at least one infrared LED, to illuminate the detection area of ​​the first stereo camera and / or the detection area of ​​the second stereo camera. In this way, objects can also be detected and classified at night.

[0025] In a preferred embodiment of the invention, the evaluation unit comprises a first processing module, a second processing module, and a central module. The first processing module detects the images of objects in an image captured by the camera and transmits a first object list containing information about the detected objects to the central module. This information is transmitted using data. Similarly, the second processing module detects the images of objects in a specific area of ​​the image pair captured by the stereo camera and transmits a second object list containing information about the detected objects to the central module. The central module then generates an output signal with output information based on the information in the first object list and the information in the second object list.The output signal is used in particular to control units operated by the driver assistance system. Using the information from the first and second object lists, relevant objects, especially lane and road markings, traffic signs, vehicles, and / or people, can be reliably detected.

[0026] Furthermore, the invention relates to a method for detecting at least one object, in which an image is captured using a camera, the object's representation in this image is detected, and the position of the object's representation is determined. Furthermore, a stereo camera captures a pair of images, and a region of the image pair is defined based on the object's determined position within the captured image. The defined image region is then searched for an image of the object.

[0027] The method specified by the independent method claim can be further developed in the same way as the device according to claim 1. In particular, the method can be further developed with the features specified in the dependent claims relating back to the device or with corresponding method features.

[0028] Further features and advantages of the invention will become apparent from the following description, which explains the invention in more detail in conjunction with the accompanying figures using exemplary embodiments.

[0029] They show: Fig. 1 a top view of a schematically represented device for detecting at least one object according to a first embodiment of the invention; Fig. 2 a schematic perspective representation of a device for detecting at least one object according to a second embodiment of the invention; Fig. 3 a schematic representation of a vehicle with a device for detecting at least one object according to Fig. 2 in a first traffic situation; Fig. 4 a schematic representation of the vehicle according to Fig. 3 in a second traffic situation; and Fig. 5 a schematic representation of a device for detecting at least one object according to a third embodiment of the invention.

[0030] In Fig. Figure 1 shows a top view of a schematic representation of a device 10 for detecting at least one object. The device comprises a circuit board 1 and further components 2 to 7 connected to the circuit board 1. The circuit board 1, in turn, comprises a carrier made of insulating material, which has electrically conductive connections that connect the electrical terminals of the components 2 to 7 connected to the circuit board 1. The electrically conductive connections are preferably implemented as conductive traces in the form of a printed circuit board. The circuit board 1 is preferably made of epoxy resin with copper conductive traces and is also referred to as a printed circuit board.

[0031] On circuit board 1, a first image acquisition sensor 4, a second image acquisition sensor 5, a third image acquisition sensor 2, and a fourth image acquisition sensor 3 are arranged. The image acquisition sensors 2 to 5 are electrically connected to the conductor tracks of circuit board 1 (not shown) via electrical connections. The image acquisition sensors 2 to 5 can be either color image acquisition sensors or grayscale image acquisition sensors. Preferably, either all image acquisition sensors 2 to 5 are color image acquisition sensors or all image acquisition sensors 2 to 5 are grayscale image acquisition sensors. The first image acquisition sensor 4 and the second image acquisition sensor 5 form a first sensor group, and the third image acquisition sensor 2 and the fourth image acquisition sensor 3 form a second sensor group. Preferably, the image acquisition sensors 2 to 5 of each sensor group are of the same type. It is advantageous if all four image acquisition sensors 2 to 5 are of the same type.

[0032] Image acquisition sensors 2 to 5 are connected to circuit board 1 via soldered connections. Alternatively, image acquisition sensors 2 to 5 can each be inserted into a socket connected to circuit board 1 via soldered connections. Image acquisition sensors 2 to 5 are mounted in the form of optoarrays. The resulting fixed position of image acquisition sensors 2 to 5 relative to each other, particularly the fixed position of the first image acquisition sensor 4 relative to the second image acquisition sensor 5 and the fixed position of the third image acquisition sensor 2 relative to the fourth image acquisition sensor 3, reduces the calibration effort required for the device 10. Calibration is only necessary once during the installation of image acquisition sensors 2 to 5. Recalibration during operation of the device 10 is not required.

[0033] The image acquisition sensors 2 to 5 are preferably optical semiconductor sensors, in particular CCD image acquisition sensors and / or CMOS image acquisition sensors. The third image acquisition sensor 2 and the fourth image acquisition sensor 4 are preferably arranged such that the centers of their image acquisition areas lie on a straight line that runs parallel to one of the long side edges 102 of the circuit board 1. The first image acquisition sensor 4 and the second image acquisition sensor 5 are preferably arranged such that the centers of their image acquisition areas lie on a second straight line that intersects the first straight line orthogonally. Furthermore, the image acquisition sensors 2 to 5 are preferably arranged such that the distance of the intersection of the two straight lines to the center of the image acquisition area of ​​the first image acquisition sensor 4 and the distance of the intersection to the center of the image acquisition area of ​​the second image acquisition sensor 5 are identical.Likewise, the distance of the intersection point to the center of the image acquisition area of ​​the third image acquisition sensor 2 and the distance of the intersection point to the center of the image acquisition area of ​​the fourth image acquisition sensor 3 are identical. The image acquisition sensors 2 to 5 are arranged on the circuit board 1 such that at least the image acquisition areas of the first image acquisition sensor 4 and the second image acquisition sensor 5 are arranged in the first plane, apart from minor manufacturing tolerances, and the image acquisition areas of the third image acquisition sensor 2 and the fourth image acquisition sensor 5 are arranged in a second plane. Preferably, all image acquisition areas of the image acquisition sensors 2 to 5 are arranged in one plane.

[0034] Alternatively, other arrangements of the image acquisition sensors 2 to 5 are also possible. In particular, the image acquisition sensors 2 to 5 can be arranged such that the centers of their image acquisition areas lie on a straight line that runs parallel to edge 102. Furthermore, the image acquisition sensors 2 to 5 can be of different sizes.

[0035] Furthermore, the device 10 includes an evaluation unit. Fig. Figure 1 shows two components belonging to this evaluation unit, designated by reference numerals 6 and 7. Components 6 and 7 are, in particular, field-programmable gate arrays (FPGAs). Alternatively, components 6 and 7 can also be located on the back of the circuit board 1, opposite the front of the circuit board 1, on which the image acquisition sensors 2 to 5 are mounted. The image data generated by the image acquisition sensors 2 to 5 is transmitted to the evaluation unit via electrical connections provided by the circuit board 1. The image data is processed in the evaluation unit using image processing algorithms implemented in the unit. At least one image processing program can be stored in the evaluation unit for this purpose.

[0036] In Fig. Figure 2 shows a schematic perspective view of a device 20 for detecting at least one object according to a second embodiment of the invention. Elements with the same structure or function have the same reference numerals. The distances 80, 83 between the image acquisition sensors 2 to 5 are determined by the printed circuit board 1.

[0037] Furthermore, the device 20 comprises four lenses 60 to 63. The lenses 60 to 63 are arranged such that the optical axis of the first lens 63 passes through the center of the image acquisition area of ​​the first image acquisition sensor 4, the optical axis of the second lens 62 passes through the center of the image acquisition area of ​​the second image acquisition sensor 5, the optical axis of the third lens 60 passes through the center of the image acquisition area of ​​the third image acquisition sensor 2, and the optical axis of the fourth lens 61 passes through the center of the image acquisition area of ​​the fourth image acquisition sensor 3. The lenses 60 to 63 are arranged, in particular, such that the optical axes of the lenses 60 to 63 are orthogonal to the respective image acquisition areas of the respective image acquisition sensors 2 to 5.Alternatively, the lenses 60 to 63 can be arranged such that their optical axes intersect the image acquisition areas near their respective centers.

[0038] The lenses 60 to 63 are connected to the circuit board 1, preferably by gluing, screwing, or connecting them via suitable clamping or snap-fit ​​connections. Alternatively, the lenses 60 to 63 can also be integrated into a housing (not shown). The housing can, for example, serve to permanently protect the device 20 from environmental influences, especially water and / or vandalism. The lenses are advantageously attached to the circuit board 1 by means of holders, with each pair of pins of a holder snapping into holes in the circuit board 1.

[0039] The first image acquisition sensor 4, the second image acquisition sensor 5, the first lens 63, and the second lens 62 together form a first stereo camera. Likewise, the third image acquisition sensor 2, the fourth image acquisition sensor 3, the third lens 60, and the fourth lens 61 form a second stereo camera. In an alternative embodiment of the invention, a monocular camera comprising one image acquisition sensor and one lens can be provided instead of the first stereo camera.

[0040] The lenses 60 to 63 each have one or more lenses and / or other optical elements and serve in particular to focus images onto the image acquisition areas of the image acquisition sensors 2 to 5. The lenses 62, 63 of the first stereo camera and the lenses 60, 61 of the second stereo camera are preferably identical. The lenses 62, 63 of the first stereo camera have a longer focal length than the lenses 60, 61 of the second stereo camera. Specifically, lenses 62, 63 have an opening angle of 15° to 25°, and lenses 60, 61 have an opening angle of 45° to 65°. This choice of lenses 60 to 63 ensures that the first stereo camera captures images of a distant area, while the second stereo camera captures images of a near area. The area in front of the device 20 up to a distance of approximately 60 m is specifically defined as the near area.In the immediate vicinity, the edges of lane 16 are also recorded. Accordingly, the area beyond a distance of 60 m is referred to as the long-distance area.

[0041] Furthermore, components 70 of an infrared illumination device are arranged on circuit board 1. The components 70 are, in particular, infrared diodes. By using infrared light, the device 20 can also be used for image acquisition in darkness, since the detection areas of the stereo cameras are at least partially illuminated by the infrared illumination device.

[0042] The device 20 for detecting at least one object is used in particular in conjunction with driver assistance systems in vehicles, especially road vehicles. However, it can also be used in conjunction with traffic management systems. The device 20 is then preferably arranged in a fixed position, for example, mounted on a mast, such as a traffic light signal mast.

[0043] In Fig. Figure 3 is a schematic representation of a vehicle 15 with a device for detecting at least one object according to Fig. Figure 2 shows a first traffic situation. Vehicle 15 is traveling on lane 16 in the direction of P1.

[0044] The detection range of the first stereo camera is indicated by solid lines 17a and 17b and labelled with reference numeral 11. The detection range of the second stereo camera is indicated by solid lines 18a and 18b and labelled with reference numeral 12. An object 21, in particular a traffic sign, located next to the roadway 16, lies at least partially within the detection range 11 of the first stereo camera. The object 21 is far enough away from the vehicle 15 that it is not yet within the detection range 12 of the second stereo camera.

[0045] In Fig. Figure 4 is a schematic representation of vehicle 15 according to Fig. 3 in a second traffic situation. The second traffic situation occurs shortly after the one in Fig. The first traffic situation shown in Figure 3 has occurred. In the meantime, vehicle 15 has moved so far in the direction of travel P1 towards object 21 that object 21 is located outside the detection range 11 of the first stereo camera and within the detection range 12 of the second stereo camera.

[0046] When detecting an object 21, a first image pair depicting the detection area 11 is initially captured at a first time point using the first stereo camera. The evaluation unit detects an image of object 21 within this first image pair and determines the position of the object 21's image. For this purpose, the evaluation unit executes a first detection algorithm. At a second time point, sequentially arranged after the first, a second image pair depicting the detection area 12 of the second stereo camera is captured using the second stereo camera. Depending on the determined position of the object 21's image within the first image pair, a specific image area of ​​the second image pair is defined.The image area is defined such that the image of the detected object 21 in the second image pair is highly likely to be depicted within the defined image area at the second time point. The evaluation unit then processes the defined image area of ​​the second image pair to detect the image of object 21 in the second image pair. For this purpose, the evaluation unit executes a second detection algorithm.

[0047] The first and second detection algorithms differ in that the first has a significantly lower detection accuracy than the second. The higher the detection accuracy of a detection algorithm, the more reliably the objects 21 to be detected are actually detected. The second detection algorithm requires considerably more time and resources, especially more memory, to detect an object 21 than the first. The first detection algorithm is also known as feature stereo and the second as dense stereo.

[0048] This ensures that object 21 is already roughly detected using the first image pair captured by the first stereo camera, even when object 21 is still in the far range. Using the first detection algorithm to detect object 21 in the far range requires very little time and relatively little memory. Once object 21 has moved sufficiently close to device 20, or device 20 has moved sufficiently close to object 21, such that object 21 is in the near range and thus within the detection area 12 of the second stereo camera, it is reliably detected in the second image pair using the second detection algorithm. Since the position of object 21 has been determined and is therefore known in the first image pair, an image area of ​​the second image pair can be defined by mapping object 21 onto the second image pair.Therefore, the entire second image pair does not need to be searched for the image of object 21 using the second detection algorithm; only the defined image area needs to be searched. This means that, despite the high detection accuracy of the second detection algorithm, very little time and memory are required to detect the image of object 21 in the second image pair. Compared to a device for detecting objects with only one stereo camera, the detection range, detection rate, detection speed, and detection quality are all improved.

[0049] In a preferred embodiment of the invention, at least one property of object 21 is determined based on the image of object 21 in the second image pair. In particular, an object class to which object 21 can be assigned is determined. In driver assistance systems, this method is used to determine, in particular, whether object 21 is a traffic sign, a moving self-illuminated object, a static self-illuminated object, and / or a person or an animal. The object class is determined, in particular, by comparison with objects stored in a database. To ensure a reliable classification of objects 21, detection using only the first detection algorithm, with its lower detection accuracy, would be insufficient.Therefore, when using only one stereo camera system, the entire image pair would have to be searched using the time- and memory-intensive second detection algorithm.

[0050] In a further preferred embodiment of the invention, a sequence of images of the detection area 11 is captured using the first stereo camera. The images of the object 21 in the sequence are determined using a tracking function stored in the evaluation unit, thus identifying the object 21's position within the sequence. By tracking the position of the object 21's position, a more accurate prediction can be made of the image area of ​​the second stereo camera's image pair in which the object 21 will be depicted when it is in close proximity. In particular, this allows a driver assistance system to determine early on if an object 21, especially due to a vehicle 15 cornering, does not enter the close-range area at all and therefore does not need to be considered further.

[0051] In Fig.Figure 5 shows a schematic representation of a device 90 for detecting at least one object 21. The image pair captured by the first stereo camera is stored in a first storage element 29. Using a first processing unit 31, the image of the object 21 in the image pair captured by the first stereo camera is detected, and the position of the image of the object 21 in the image pair is determined. Furthermore, using the processing unit 31, a first object list containing information about the objects 21 detected in the first image pair is generated and transmitted to a fusion unit 40.

[0052] The second image pair, captured by the second stereo camera system, is stored in a second storage element 28. A second processing unit 30 generates a second object list containing information about the objects 21 detected in the second image pair and transmits this information to the fusion unit 40. Based on the information in the first object list and the information in the second object list, the fusion unit 40 generates an output signal, which is transmitted to a human-machine interface 50 and / or a control unit 52. The human-machine interface 50 is, in particular, a display unit that provides the operator with information about relevant objects 21 detected and classified by a device 90. The control unit 52 is used to control actuators 54 to 58.The actuators 54 to 58 are in particular a control module for controlling the front headlights of a vehicle 15, a brake assist of a vehicle 15, a unit for controlling a drive unit of a headrest of the vehicle 15 and / or a unit for controlling a belt tensioner.

[0053] The described method and the described device 10, 20, 90 can also be used for detecting, recording, evaluating, and counting static and moving objects, as well as for tracking and analyzing the movement behavior of persons and / or objects, and for measuring objects and / or persons. The device 10, 20, 90 can be used for object recognition in motor vehicles, for driver assistance systems, in road traffic, in manufacturing facilities, in event venues, in long-distance and local transport, in agricultural operations, in public facilities, for controlling mass flows, and in industrial processes. The device 10, 20, 90 is also referred to as a combination camera.

[0054] In a further embodiment of the invention, a microphone can additionally be provided for recording sounds. The sounds are stored and / or evaluated together with the corresponding images. Reference symbol list 1 circuit board 2 to 5 image capture sensors 6, 7 Component 10, 20, 90 Device 11, 12 Recording area 15 vehicles 16 lanes 17a, 17b, 18a, 18b Full line 21 objects 28, 29 Storage element 30, 31 processing unit 40 fusion units 50 Human-Machine Interface 52 Control unit 54, 56, 58 Actuator 60 to 63 lens 70 Infrared diode 80, 83 distance 102 edge P1 Direction of travel

Claims

[1] Device for detecting at least one object, with a camera comprising a first image acquisition sensor (4), with a stereo camera comprising a second image acquisition sensor (2) and a third image acquisition sensor (3), and with an evaluation unit, wherein the evaluation unit detects the image of the object (21) in an image captured by the camera and determines the position of the image of the object (21), the evaluation unit defines an image area of ​​an image pair captured with the aid of the stereo camera depending on the determined position of the object (21) in the image captured with the aid of the camera, and the evaluation unit processes the defined image area to detect an image of the object (21), characterized by , that the evaluation unit detects the image of the object (21) in each image of a sequence of images captured by the camera and determines the positions of the images of the object (21) in the images, wherein a tracking function is stored in the evaluation unit, with the help of which the image of the object (21) is determined in the images of the sequence and the image of the object (21) is determined via the images of the sequence, and that the evaluation unit defines an image area of ​​an image pair captured with the aid of the stereo camera depending on at least two determined positions of the object (21) in at least two images of the image sequence captured with the aid of the camera. [2] Device (10, 20, 90) according to claim 1, characterized by , that the stereo camera is a second stereo camera, and that the camera includes a fourth image capture sensor (5) and is a first stereo camera. [3] Device (10, 20, 90) according to any one of the preceding claims, characterized by , that the evaluation unit determines at least one property of the object (21), in particular an object class to which the object (21) can be assigned, based on the representation of the object (21) in the defined image area. [4] Device (10, 20, 90) according to claim 3, characterized by , that the evaluation unit generates an output signal with information about the object (21) depending on the property of the object (21). [5] Device (90) according to claim 4, characterized by , that a control unit (52) is provided for controlling at least one actuator (54, 56, 58) depending on the output signal of the evaluation unit. [6] Device (10, 20, 90) according to any one of the preceding claims, characterized by, that the evaluation unit, when detecting the image of the object (21) in the image captured by the camera, executes a first algorithm and, when detecting the image of the object (21) in the defined image area of ​​the image pair captured by the stereo camera, executes a second algorithm, the second algorithm having a higher detection accuracy than the first algorithm. [7] Device (10, 20, 90) according to any one of the preceding claims, characterized by , that the camera captures the image(s) of the image sequence before the stereo camera captures the image pair. [8] Device (10, 20, 90) according to any one of the preceding claims, characterized by , that the image acquisition sensors (2 to 5) and / or the evaluation unit are mounted on a circuit board (1), in particular a printed circuit board made of epoxy resin, and are electrically connected to it. [9] Device (10, 20, 90) according to any one of the preceding claims, characterized by that the image acquisition sensors (2 to 5) are optical semiconductor sensors, CCD image acquisition sensors and / or CMOS image acquisition sensors. [10] Device (20, 90) according to any one of claims 2 to 9, characterized by , that a first lens (63) and a second lens (62) are arranged such that an image of a detection area of ​​the first stereo camera is each projected and / or focused onto an image detection area of ​​the image detection sensors (4, 5) of the first stereo camera, and that a third lens (60) and a fourth lens (61) are arranged such that an image of a detection area of ​​the second stereo camera is each projected and / or focused onto an image detection area of ​​the image detection sensors (2, 3) of the second stereo camera. [11] Device (20, 90) according to claim 10, characterized by, that the first lens (63) and the second lens (62) have a smaller angle of opening than the third lens (60) and the fourth lens (61). [12] Device (20, 90) according to any one of claims 2 to 11, characterized by , that at least one infrared diode (70) is provided for illuminating part of the detection area of ​​the first stereo camera and / or part of the detection area of ​​the second stereo camera. [13] Device (90) according to any of the preceding claims, characterized by , that the evaluation unit comprises a first processing module (30), a second processing module (31) and a central module (40), the first processing module (30) detects the images of objects (21) in an image captured by the camera and transmits an initial object list with information about the detected objects (21) to the central module (40), the second processing module (32) detects the images of the objects (21) in a sub-area of ​​the image pair captured with the aid of the stereo camera and transmits a second object list with information about the detected objects (21) to the central module (40), and the central module (40) generates an output signal depending on the information in the first object list and the information in the second object list. [14] Method for detecting at least one object, in which an image is captured using a camera The image of the object (21) in this image is detected and the position of the image of the object (21) is determined. A stereo camera captures a pair of images. an image area of ​​the image pair is determined depending on the determined position of the object (21), and in which the specified image area is searched for an image of the object (21), characterized by , that the evaluation unit detects the image of the object (21) in each image of a sequence of images captured by the camera and determines the positions of the images of the object (21) in the images, whereby the images of the object (21) are determined in the images of the sequence using a tracking function stored in the evaluation unit and the image of the object (21) is determined via the images of the sequence, and that the evaluation unit defines an image area of ​​an image pair captured with the aid of the stereo camera depending on at least two determined positions of the object (21) in at least two images of the image sequence captured with the aid of the camera.

Images