Method for improved detection of a vehicle's traffic sense

Abstract

The invention relates to a method for improved detection of a traffic sense of a vehicle (10), in which the vehicle (10), in addition to an existing front camera (12) and / or sensor system (14), has at least one further camera (20) on each side in order to detect road users (30, 32, 34), in which the locally existing traffic sense is deduced based on the evaluation of the detected road users (30, 32, 34). The invention also relates to a vehicle (10) that is configured to carry out the method according to the invention.

Description

[0001] The invention relates to a method for improved detection of a vehicle's traffic sense. In this method, the vehicle, in addition to an existing front camera and / or sensor system, has at least one additional camera on each side to detect road users. Based on the evaluation of the detected road users, the local traffic sense is deduced. The invention also relates to a vehicle configured to carry out the method according to the invention.

[0002] For many driver assistance systems, it is important to know whether a vehicle is driving on the left or right. One example is the centering of a Travel Assist system.

[0003] Systems for determining traffic sense are known in the prior art. These systems use a front camera and / or sensors (e.g., LIDAR, RADAR, ultrasound) to determine the sense of traffic in the vehicle's surroundings and can be adopted or applied to the vehicle. Reference is made, for example, to documents DE 10 2007 059 083 A1, DE 103 45 809 A1, and DE 10 2012 221 652 A1.

[0004] The known procedures are usually initiated at the beginning of the vehicle's journey and require a certain lead time to determine the direction of travel. During this time, the driver assistance systems cannot adjust and adapt to the prevailing local traffic patterns. Consequently, they cannot intervene quickly enough or correctly should a dangerous situation arise within the first few minutes of a journey.

[0005] The basis for determining the sense of direction is either location information from navigation systems and global positioning systems and / or inferences from the observations of other road users using the aforementioned front camera and / or the vehicle's sensors. The time required to reliably detect and determine the sense of direction depends on the number of other road users, especially oncoming vehicles, and their temporal separation.

[0006] In other situations, the sense of traffic may not be determined because the front camera is dirty or is being glared. The sensors may also be dirty or impaired for other reasons.

[0007] In addition, it may be desirable to determine the direction of traffic before the start of a journey, so that it is known in the vehicle at the start of the journey. This is not possible with current systems.

[0008] The object of the invention is therefore to provide an improved method for determining a traffic sense of a vehicle that eliminates the aforementioned disadvantages and problems.

[0009] The object of the invention is achieved by a method and a vehicle according to the independent claims. Further preferred embodiments of the invention emerge from the remaining features mentioned in the subclaims.

[0010] The method according to the invention for determining a traffic sense of a vehicle is initially based on the fact that the detection of other road users takes place by means of at least one front camera and / or by means of a sensor system of the vehicle, and is characterized in that the vehicle additionally has at least one camera on one side or both sides and by means of this camera or these cameras the position and / or movement of other road users is detected and evaluated with regard to the locally existing traffic sense together with the information from the at least one front camera and / or the sensor system of the vehicle.

[0011] The local traffic sense refers to the traffic sense at the vehicle's location, i.e., whether there is left-hand or right-hand traffic. The vehicle's traffic sense is its attitude toward the local traffic sense, enabling it to participate in it and adapt safety and assistance systems accordingly.

[0012] The vehicle is therefore already equipped with at least one front camera and / or sensors for detecting traffic conditions. These can be used for all environmental detection and monitoring needs of the vehicle, including determining the sense of traffic according to the state of the art. The vehicle's sensors can include, for example, LIDAR, radar, and / or ultrasound, and are therefore not limited to one type of sensor in terms of number or technology.

[0013] In addition to the existing systems, the vehicle now has at least one additional camera, either on one or both sides of the vehicle. This means that at least one additional camera can be installed on one side of the vehicle or on both sides. Therefore, the term "at least one side camera" is also used interchangeably below. This camera is designed to capture road users and the surroundings to the side of the vehicle. Depending on the field of view of the camera(s) used, this camera(s) can also capture areas of traffic in and against the direction of travel, as well as the road.

[0014] Two or more cameras can also be mounted on the side of a vehicle and aligned so that they can capture not only the traffic passing alongside and the surroundings to the side of the vehicle, but also areas to the front, rear and below.

[0015] The at least one camera mounted on the side of the vehicle then records the position and / or movement of other road users. The data thus recorded, preferably from other vehicles whose direction of movement allows a clear conclusion to be drawn about the prevailing local traffic situation, is subsequently evaluated. The information recorded by the at least one front camera and / or the vehicle's own sensors is also included. The evaluation is performed by a control device that is associated with or interacts with the at least one camera mounted on the side.

[0016] In the simplest case, the information recorded, especially that from the at least one front camera and the at least one side camera, is evaluated by means of image evaluation or image analysis. For this purpose, the image data is analyzed with regard to predetermined characteristics, i.e. these characteristics are identified and classified image by image. The predetermined characteristics include, among other things, the presence of other road users in a number of consecutive images, based on which the direction of movement of the respective road user can be determined. This is sufficiently known from the prior art and will therefore not be explained in more detail. From this analysis and its results, it can then be deduced whether the vehicle is driving on the left or the right, and thus the vehicle's own traffic sense, including its assistance systems, can be adapted to the local traffic sense.

[0017] In an exemplary application of the method, for a vehicle parked lengthways in the direction of travel with one camera on each side of the vehicle, provided the battery provides sufficient voltage, i.e. a voltage higher than a predetermined limit, the detection can be carried out and evaluated as described above. The images from the cameras on the right and left sides of the vehicle can be compared. If the vehicle is parked on the right-hand side of the vehicle and the side camera there only detects pedestrians, cyclists and the like, while the left-hand camera only detects objects approaching from behind and then detected in front of the vehicle, it can be concluded that the vehicle is driving on the right. The method according to the invention is therefore already carried out when the vehicle is stationary, i.e. when it is parked, provided sufficient energy is available.This means that the vehicle's sense of traffic can be verified (because it is known in advance) or adjusted as soon as the vehicle is started.

[0018] The method can also be successfully applied in special situations, for example when the parking space is located between two traffic lanes. In particular, in these situations, a comparison between at least one right-hand and at least one left-hand camera of the vehicle can contribute to a reliable determination of the direction of traffic. If the right-hand camera detects vehicles along the vehicle's own longitudinal axis and the left-hand camera detects vehicles moving against its own longitudinal axis - in other words, if an apparent counterclockwise rotation of the traffic around the vehicle is detected - this indicates right-hand traffic. If the right-hand camera detects vehicles against its own longitudinal axis and the left-hand camera detects vehicles along its own longitudinal axis - in other words, if an apparent clockwise rotation of the traffic around the vehicle is detected, this indicates left-hand traffic.

[0019] Traffic sense detection can also be improved while driving. Using at least one camera on the side, the detection time and the number of detected vehicles or objects can be significantly improved compared to a front camera alone.

[0020] As a result, the method according to the invention allows a vehicle's sense of traffic to be determined more quickly, reliably, and robustly, both while stationary and while driving. This allows the vehicle's assistance systems to be adjusted more quickly to the prevailing local sense of traffic, as well as the control system for autonomous driving, provided the vehicle is configured for this purpose.

[0021] According to a first embodiment of the method according to the invention, the vehicle should additionally have at least one rear camera which detects the position and / or movement of other road users in order to take these into account when evaluating the local traffic sense.

[0022] In addition to the existing cameras and / or sensors, a camera is also installed at the rear of the vehicle, primarily directed toward the rear and recording traffic behind it. This can be a dedicated camera or an existing camera that is part of the process. The images captured by this rear camera are also fed into the analysis and evaluated together with the other information.

[0023] This has the advantage of providing a larger detection area for surrounding traffic. Even when the vehicle is stationary, the surrounding traffic can be detected and evaluated with regard to its direction of travel. If a vehicle is parked perpendicular to the direction of travel on the right side of the road – in this example, facing forwards – the front camera only records pedestrians, cyclists, and the like, while the rear camera records vehicles or road users approaching from the right and departing on the left. This makes it possible to conclude that traffic is traveling on the right. The side cameras can confirm this with their own recordings if the viewing angle is sufficiently wide or they are positioned far back on the vehicle.

[0024] The method according to the invention can be further developed such that the infrastructure, road markings, and / or peripheral buildings surrounding the vehicle are recorded and taken into account when evaluating the local traffic sense. Thus, each of the cameras available on the vehicle—i.e., the front camera, side camera(s), and—if present—rear camera—with suitable alignment and a suitable field of view can record the vehicle's surroundings, in particular the infrastructure in the vicinity of the vehicle, such as road signs, bridges, tunnels, sign gantries, and the like, as well as road markings of all kinds and any peripheral buildings, such as guardrails, curbs, or grass verges. Their position and / or design also allows conclusions to be drawn about the local traffic sense.

[0025] For example, in right-hand traffic, the road markings on the right are solid, while on the left, they may be a dashed line, if the traffic situation allows. Using this additional information can provide further guidance for determining the local traffic sense and complement the analysis based on the position and / or movement of other road users.

[0026] At the start of a journey and / or during a journey, situations may arise in which the vehicle's front camera, sensors, and / or rear camera (collectively referred to below as other detection devices) cannot function as intended, for example, because they are dirty or damaged, or because they are dazzled by the low sun or other road users. In such cases, it may be useful to further develop and apply the method according to the invention in such a way that at least one camera on one or both sides of the vehicle replaces the function of a front camera, a sensor system of the vehicle, and / or the function of a rear camera.

[0027] If one or all of the other detection devices fail, the at least one side camera of the vehicle will be used to determine the local traffic sense. The missing information from one or all of the other detection devices will be replaced so that incorrect information from these other detection devices is not processed or at least cannot negatively influence the result, or so that its absence does not lead to error messages and / or incorrect determinations of the local traffic sense. However, this requires that a failure or impairment of one or more of the other detection devices is detected and that the detection range of the at least one side camera of the vehicle is sufficiently large to cover a sufficiently large area of the surroundings.

[0028] A further embodiment of the method according to the invention provides that in the event of failure of at least one camera on one side or both sides of the vehicle and / or an uncertain result with regard to the determined local existing traffic sense, a known method for determining the traffic sense, in particular a location-based method, is used.

[0029] A technical defect or glare can also affect at least one of the vehicle's side cameras, causing it to fail and / or provide no usable information. In this case, only state-of-the-art systems are available to determine the prevailing local traffic sense, such as, depending on the vehicle's equipment, at least a front camera and / or the vehicle's own sensors, and possibly a rear camera.

[0030] Likewise, a navigation system can be installed in the vehicle that can determine the vehicle's position. This can be used, for example, to determine the prevailing local traffic direction based on a vehicle's position: For example, if a vehicle is parked in Great Britain, the prevailing traffic direction is left-hand traffic, whereas if a vehicle is located or driving in Germany, the prevailing traffic direction can be determined based on the vehicle's position.

[0031] If no such navigation system is installed in the vehicle, but the other aforementioned systems are, then—as is known in the prior art—the sense of the road can be determined based on the information they record. However, this can also be done in addition to the use of a navigation system. Typically, this only occurs while driving and is time-consuming, as already explained in the introduction. However, in the event of a failure, the function of at least one side camera of the vehicle can be temporarily replaced, rather than leading to a total failure of the system according to the invention.

[0032] It has already been explained that the images from the front camera and the at least one side camera of the vehicle can be captured using image evaluation or image analysis. However, according to a further embodiment of the method according to the invention, this can also be done using a model based on a trained neural network. Such models can be configured and trained in such a way that other road users can be recognized, classified, and tracked across a sequence of images, and their direction of movement can be determined. This allows for more reliable detection of their movements and thus a more robust data basis for determining the local traffic sense.

[0033] The method according to the invention can be implemented in the form of a program, software, or algorithm containing commands and / or command sequences, the execution of which executes the steps of the method. The program, software, or algorithm can be stored on a storage medium that is assigned to a control device or which the control device can access in order to execute the method. Information from the front camera and / or the vehicle's sensors, the at least one side camera, and optionally the rear camera is received and evaluated in the manner described above. The result of the evaluation is a determination or at least an estimate of a very likely existing local traffic sense, which can be made available, i.e. transmitted, to other systems of the vehicle.

[0034] The invention also includes a vehicle that is equipped to determine its traffic sense using the method described above and has at least one control device, at least one front camera and / or a sensor system for detecting other road users and is characterized in that the vehicle also has at least one camera on one side or both sides of the vehicle.

[0035] The at least one control device is intended to carry out the method according to the invention, as already described. In doing so, it accesses and executes a program, software, or algorithm stored on a storage device associated with the control device or accessible by the control device. The images from the camera(s) and, if applicable, the sensors are evaluated, thereby determining the local traffic sense. This information can then be provided for adapting or adjusting vehicle systems, in particular assistance systems.

[0036] There are various configurations for the at least one control device. For example, a control device can be arranged directly together with one of the side cameras (and optionally also with the other cameras) of the vehicle as a functional unit, and then perform an individual evaluation of the local traffic sense. However, all evaluations must then be consolidated at a central location. Furthermore, such a configuration requires some additional installation space.

[0037] It is therefore advantageous if the evaluation of the acquired information takes place at a different location in the vehicle than where it was captured. This means that the at least one control device is not located on or in one of the side cameras and the evaluation takes place there, but rather in another area of the vehicle where sufficient space is available.

[0038] The at least one side-mounted camera of the vehicle then transmits the captured images to the at least one remote control unit of the vehicle. Accordingly, this can also be a common control unit that evaluates the information from all cameras participating in the method according to the invention, optionally using information from the sensors, jointly and with mutual consideration. This allows for considerable space savings and the method to be carried out more effectively.

[0039] An advantageous embodiment of the vehicle is when the at least one side camera of the vehicle is arranged on a railing of the vehicle or in a side mirror of the vehicle.

[0040] In principle, the positioning of at least one camera on one or both sides of a vehicle is possible at any location that offers sufficient installation space and enables communication with the vehicle or a control device. They can be surface-mounted or at least partially recessed. However, this can visually impair the overall appearance of the vehicle and / or pose the risk of damage. Therefore, a protected location and, if possible, a position with maximum visibility of the surrounding traffic would be desirable.

[0041] A placement in or on a side mirror of the vehicle provides such protection. Since the exterior mirrors are also the widest part of the vehicle, this prevents other vehicle components from obstructing the view along either side of the vehicle when the doors are closed. This ensures the desired overview.

[0042] Such an overview can also be achieved by arranging at least one side camera on a roof rail, whereby the elevated position in relation to the vehicle can result in the improved overview.

[0043] In both cases, at least one side camera can be mounted on or in the exterior mirror or roof rail in such a way that it can protect it against external influences, including deliberate damage, and, if possible, does not negatively affect the appearance of the vehicle.

[0044] Particularly in autumn and winter, when the weather is bad, at least one of the side cameras can become dirty, for example due to spray stirred up by oncoming traffic. It can therefore be expedient to provide a cleaning device on at least one of the cameras arranged on the side of the vehicle, especially if they are arranged close to the ground. Such a cleaning device can be connected to a reservoir for cleaning fluid and have a cleaning or wiping element which, when visibility is impaired and / or in response to an operating action by the user, applies the cleaning fluid to the dirty area and cleans the camera with the help of the cleaning or wiping element. This can therefore be done in a similar way to a windshield washer system or a device for cleaning a vehicle's headlights.

[0045] The method and vehicle according to the invention make it possible to determine the local traffic sense in the vicinity of a vehicle more reliably and quickly than with known methods. Furthermore, the determination of the local traffic sense is also possible when the vehicle is stationary, as the vehicle's surroundings can be observed more comprehensively. Consequently, on-board systems, such as assistance systems, can be adapted to the local traffic sense right from the start of the journey.

[0046] The various embodiments of the invention mentioned in this application can be advantageously combined with one another, unless otherwise stated in the individual case.

[0047] The invention is explained below in exemplary embodiments with reference to the accompanying drawings. They show: Fig. 1 a schematic diagram of a vehicle with side cameras and other detection devices, Fig. 2 a schematic diagram of a first traffic situation, Fig. 3 a schematic diagram of a second traffic situation and Fig. 4 a schematic diagram of another special traffic situation.

[0048] Fig. 1 shows a schematic diagram of a vehicle 10 from above. This vehicle 10 has a front camera 12 and a forward-facing sensor system 14 (shown in the center in a highly simplified manner). Their detection range is designated by reference numeral 24. Furthermore, four side cameras 20 are arranged on the vehicle, one in each side mirror of the vehicle 10 and another in the rear area of the vehicle, purely by way of example. Their detection ranges are also each designated by reference numeral 24. Finally, a rear camera 22 with a detection range 24 is provided on the vehicle 10.

[0049] Out of Fig. 1 it can be seen that the cameras 12, 20 and 22 as well as the sensors 14 can record almost the entire surroundings of the vehicle 10 and thus other road users in order to determine the local traffic sense on the basis of this information and to specify it for the vehicle 10 itself.

[0050] All cameras 12, 20 and 22 as well as the sensor system 14 of the vehicle 10 are connected to a control device 16, which receives the recorded images or sensor data and evaluates them with regard to the existing local traffic sense.

[0051] The process should be based on Fig. 2 will be explained.

[0052] The vehicle 10 is parked in a parking area 42 along a road 40. The vehicle 10 in Fig. The cameras 12, 20, and 22 shown in Figure 1, as well as the sensors 14, detect other road users such as pedestrians 32, cyclists 34, or other vehicles 30. Pedestrians 32 and cyclists 34 are primarily detected by the side cameras 20 on the right side of the vehicle. These also detect the grass strip 44 between the parking area 42 and the footpath.

[0053] The left-side cameras 20 and the other detection systems 12, 14 and 20 detect the traffic in front of, behind and to the left of the vehicle 10. Vehicles 30 on the road 40 approach from behind, pass the vehicle 10 on its left side along its own direction of travel and move away towards the front.

[0054] By detecting and evaluating other road users 30, 32, 34 as well as the peripheral buildings 44 using image recognition or even a trained model, their type can be classified and the objects tracked in such a way that their direction of movement can be determined. By additionally comparing the type of road users between the sides of the vehicle 10, differences can also be identified, such as the fact that no vehicles 30 are detected on the right-hand side. From the totality of this information, it is determined that right-hand traffic is, or should be, in effect here.

[0055] In a similar way, this also applies to the situation in Fig. 3. However, the vehicle 10 is parked here transversely to the direction of travel. Thus, the pedestrians 32 and cyclists 34 are primarily detected by the front camera 12 and, if applicable, the sensors 14, while the other vehicles 30 on the road 40 are detected by the side cameras 20 and the rear camera 22. It will be clear to the person skilled in the art that the evaluation is carried out in a similar manner to that in Fig. 2 can be done and will lead to the same result.

[0056] A special situation exists in Fig. 4. Here, parking spaces 42 are located between the lanes of a road 40 and on its sides. Other vehicles 30 can be parked there. Other road users can move on the road 40 or outside the side parking spaces (not shown). If the vehicle 10 is now parked in the central parking space 42, the front camera 12 and the rear camera 22 as well as the sensor system 14 detect passing vehicles both in front of and behind the vehicle 10. If the side cameras 20 of the vehicle 10 are as shown in Fig. 1, they can at least partially detect traffic in the surrounding area.

[0057] As from the Fig.4, other vehicles 30 pass behind the vehicle 10 in question, coming from the left (relative to the direction of travel of the vehicle 10) and then move away to the right. In front of the vehicle 10, the vehicles 30 approach from the right and move away to the left. This creates an apparent counterclockwise rotation of traffic around the vehicle 10, so it can be concluded that the traffic is traveling on the right.

[0058] The above statements were made for situations in which the vehicle 10 is parked and in the event that sufficient energy is available to operate the method. Of course, and as already explained, the method can be carried out in a similar manner while driving, whereby the local traffic sense can be determined more quickly and reliably than with known methods. List of reference symbols 10 vehicles 12 front camera 14 Sensor technology 16 Control device 20 side camera 22 rear camera 24 detection range 30 other road users (vehicle) 32 other road users (pedestrians) 34 other road users (cyclists) 40 Street 42 parking spaces 44 green strips QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2007 059 083 A1

[0003] DE 103 45 809 A1

[0003] DE 10 2012 221 652 A1

[0003]

Claims

[1] Method for determining a traffic sense of a vehicle (10), in which the detection of other road users (30, 32, 34) is carried out by means of at least one front camera (12) and / or by means of a sensor system (14) of the vehicle (10), characterized by that the vehicle (10) additionally has at least one camera (20) on one side or both sides and by means of this camera (20) or these cameras (20) the position and / or movement of other road users (30, 32, 34) is recorded and evaluated with regard to the locally existing traffic sense together with the information from the at least one front camera (12) and / or the sensor system (14) of the vehicle (10). [2] Method according to claim 1, characterized by that the vehicle (10) additionally has at least one rear camera (22) which detects the position and / or movement of other road users (30, 32, 34) in order to take these into account when evaluating the local traffic situation. [3] Method according to claim 1 or 2, characterized by that the infrastructure, road markings and / or peripheral buildings surrounding the vehicle (10) are recorded and taken into account when evaluating the local traffic sense. [4] Method according to one of the preceding claims, characterized by that the at least one camera (20) on one side or both sides of the vehicle (10) replaces the function of a front camera (12), a sensor system (14) of the vehicle (10) and / or the function of a rear camera (22). [5] Method according to one of the preceding claims, characterized by that in the event of failure of at least one camera (20) on one side or both sides of the vehicle (10) and / or an uncertain result with regard to the determined local existing traffic sense, a known method for determining the traffic sense, in particular a location-based method, is used. [6] Method according to one of the preceding claims, characterized by that the collected information is evaluated using a model based on a trained neural network. [7] Vehicle (10) equipped to determine its sense of traffic according to one of the preceding claims, with at least one control device (16), at least one front camera (12) and / or by means of a sensor system (14) for detecting other road users (30, 32, 34), characterized by that the vehicle (10) also has at least one camera (20) on one side or both sides of the vehicle (10). [8] Vehicle (10) according to claim 7, characterized by that the evaluation of the recorded information takes place at a different location in the vehicle (10) than its recording. [9] Vehicle (10) according to claim 7 or 8, characterized bythat the at least one camera (20) is arranged on a railing of the vehicle (10) or in a side mirror of the vehicle (10). [10] Vehicle (10) according to one of claims 7 to 9, characterized by that it has a cleaning device for cleaning at least one of the cameras (20) arranged on the side of the vehicle (10).

Images