Driver assistance system

Abstract

A driver assistance system comprising a camera device (K) with a data processing device (D) for detecting camera objects and a plurality of radar sensors (R1, R2, R3, R4) for detecting radar objects, wherein a fusion of the data characterizing the camera objects and the data characterizing the radar objects output by the camera device and the data characterizing the radar objects output by the radar sensors (R1, R2, R3, R4) takes place in the data processing device (D), is characterized in that the camera device has a control device (S) which controls the radar sensors (R1, R2, R3, R4) such that the radar sensors (R1, R2, R3, R4) output the data characterizing the radar objects only within predetermined time windows.

Description

Prior ArtThe present invention relates to a driver assistance system comprising a camera device having a data processing device for detecting camera objects and a plurality of radar sensors for detecting radar objects, wherein fusion of the data which characterize the camera objects and which are output by the radar sensors and which characterize the radar objects takes place in the data processing device.Driver assistance systems of this type are increasingly being used in modern vehicles. They serve to detect obstacles, which can be realized, for example, by other vehicles or pedestrians, in the environment of the vehicle and to actuate the vehicle accordingly, so that no collision with such obstacles takes place. In particular, so-called advanced driver assistance systems (ADAS systems) are increasingly also being used. Such systems should on the one hand have the greatest possible scope of functions, but on the other hand at the same time be very cost-effective. This has the consequence that camera-centric systems and radar sensors are combined and a fusion of the camera objects output by the camera and the radar objects output by the radar sensors must be fused. In the following, "camera objects" are data which characterize objects detected by the camera, and "radar objects" are data which characterize objects detected by the radar sensors.In present systems, the maximum number of radar objects with protocol data units of 64 bytes that cameras can read in is limited because of the CPU resources. For example, it is now possible to trigger seven protocol data units (PDUs) within 10 ms without limiting the operating system (OS limit) and connected thereto a camera reset. It is assumed here that the cycle time of the object transmission for an individual radar sensor is 50 ms. In order to now achieve sufficient performance of the camera and radar objects, at least six objects are required from each radar sensor. If it is now assumed, for example, that vehicles today usually have four radar sensors, then only 24 PDUs which characterize radar objects would have to be processed here. If all 24 PDUs arrive at the camera simultaneously, this would result in an OS limit as mentioned above, because in this case the CPU does not have enough time to process all PDUs and this would result in a camera reset.Disclosure of the InventionThe driver assistance system described here according to Claim 1 enables a merging of camera objects with radar objects of a plurality of radar sensors, for example four radar sensors. For this purpose, the invention provides that the camera device has a control device which controls the radar sensors in such a way that the radar sensors output data which characterize the radar objects only within predefined time windows.This targeted and controlled output of the data characterizing the radar objects prevents a simultaneous arrival of these data at the camera and as a result a proper fusion of the data characterizing the radar objects with the data characterizing the camera objects is possible and an aforementioned limitation of the operating system (OS limit) and thus a camera reset are largely ruled out.According to one aspect of the invention, it is provided that the camera device has a master function, whereas the radar sensors have slave functions. The master-slave arrangement allows large amounts of data to be processed in a controlled and synchronous manner without having to accept disadvantages on account of the runtime and / or the processing load.According to one aspect of the invention, it is furthermore provided that the control device of the camera device sends messages for sending the data characterizing the radar objects to the radar sensors via a CAN FD bus. In this way, processing of multiple radar sensors, for example four radar sensors, may be distributed evenly.Furthermore, according to a further aspect of the invention, it is provided that the time windows within which the radar sensors can output the data characterizing the radar objects are defined by the camera device as a function of the number of protocol data units (PDUs) which the data processing device of the camera device can read in per predefined time unit. This determination can be made in advance; however, it can be adapted purely in principle also during operation. It serves for the most optimum possible transmission of the data.DRAWINGSDetails of the driver assistance system are explained in more detail on the basis of an exemplary embodiment in the drawings. FIG. 1 schematically shows a driver assistance system concerned here. FIG. 2 shows the data fusion without the use of the driver assistance system according to the invention. FIG. 3 shows the data fusion using the driver assistance system according to the invention.Embodiments of the InventionFIG. 1 schematically illustrates a vehicle F which has four radar sensors R 1, R 2, R 3, R 4. The radar sensors are arranged here, by way of example, at the four corners of the vehicle F. It should be noted that this arrangement is not absolutely necessary. It is also possible, for example, for two radar sensors to be arranged centrally at the front and for two further radar sensors to be arranged centrally at the rear. In addition, radar sensors may also be arranged laterally. The following description is also not limited to four radar sensors. Rather, more than four radar sensors may also be provided.The vehicle also has a camera device K which, for its part, has a data processing device D and a control device S. Radar sensors R 1, R 2, R 3, R 4 detect objects O 1, O 2, O 3, O 4 in the surroundings of vehicle F, and objects O 1, O 2, O 3, O 4 detected by radar sensors R 1, R 2, R 3, R 4 are also referred to below as radar objects for short. The data characterizing the radar objects are transmitted to the data processing device D of the camera device K. This can be done in a wired or wireless manner.At the same time, the camera device K itself also captures the objects O 1, O 2, O 3, O 4. The data characterizing the objects O 1, O 2, O 3, O 4, which are captured by the camera device K, are referred to below as camera objects.In the data processing device D of the camera device K, the data characterizing the camera objects output by the camera device K and the data characterizing the radar objects output by the radar sensors R 1, R 2, R 3, R 4 now merge. The maximum number of radar objects with protocol data units (PDU protocol data units) of 64 bytes, which the camera device K can read in, is now very limited. Thus, in the case of camera devices known per se, up to 7 PDUs per 10 ms can be read in without reaching the limit of the operating system (OS limit, operation system limit) and thus triggering a camera reset. The cycle time of the object transmission for an individual radar sensor R 1, R 2, R 3, R 4 is 50 ms. For a desirable performance of the fusion of radar and camera objects, at least six objects are required by each radar sensor R 1, R 2, R 3, R 4. If now four radar sensors were to send these objects simultaneously to the data processing device D of the camera device K, the OS limit would be reached very often and the camera would be reset, which is unacceptable for the operation of the driver assistance system. This state is shown in Fig. 2, where "video" denotes the data concerning the camera objects and R1 to R4 denotes the data concerning the radar objects. The said very large number of objects to be transmitted is represented by the ellipse denoted 100.In order to solve this problem, it is provided according to the invention that radar sensors R 1, R 2, R 3, R 4 are controlled by control device S of camera device K in such a way that radar sensors R 1, R 2, R 3, R 4 output the data characterizing the radar objects only within predefined time windows. In other words, the transmission of the objects of each radar sensor R 1, R 2, R 3, R 4 is possible only in the time window provided for this purpose, and this time window is controlled by the camera device K and is configurable.The control device S of the camera device K itself sends the message that data can be transmitted to the radar sensors R 1, R 2, R 3, R 4 via a CAN FD bus. In other words, this transmission of the message is used as an event in order to transmit the data from the radar sensors R 1, R 2, R 3, R 4 on an event basis. Thus, the load of processing the four radar sensors R 1, R 2, R 3, R 4 can be distributed uniformly. This is schematically shown in FIG. 3, where again as in FIG. 2 "video" denotes the data relating to the camera objects and R1 to R4 denote the data relating to the radar objects.In other words, the control device S of the camera device K serves as a master and the four radar sensors R 1, R 2, R 3, R 4 serve as slaves. In other words, the control device S of the camera device K controls the radar sensors R 1, R 2, R 3, R 4 as slaves as the master. This leads to problem-free processing of the data and thus to problem-free operation of the driver assistance system.

Claims

Driver assistance system comprising a camera device (K) having a data processing device (D) for detecting camera objects and a plurality of radar sensors (R1, R2, R3, R4) for detecting radar objects, wherein fusion of the data which are output by the camera device and characterize the camera objects and of the data which are output by the radar sensors (R1, R2, R3, R4) and characterize the radar objects takes place in the data processing device (D), characterized in that the camera device has a control device (S) which controls the radar sensors (R1, R2, R3, R4) in such a way that the radar sensors (R1, R2, R3, R4) output the data which characterize the radar objects only within predefined time slots.Driver assistance system according to Claim 1, characterized in that the control device (S) of the camera device (K) has a master function and in that the radar sensors (R1, R2, R3, R4) have slave functions.Driver assistance system according to Claim 1 or 2, characterized in that the control device (S) of the camera device (K) sends messages for transmitting the data characterizing the radar objects to the radar sensors (R1, R2, R3, R4) via a CAN FD bus.Driver assistance system according to one of the preceding claims, characterized in that the time windows within which the radar sensors (R1, R2, R3, R4) can output the data characterizing the radar objects are defined by the control device (S) of the camera device (K) as a function of the number of protocol data units which the data processing device (D) of the camera device (K) can read in per predefined time unit.

Images