Method and system for providing a rear surrounding area of a vehicle

Abstract

The invention relates to a method for providing image information of the rear surrounding area (5) of a vehicle (2) on at least one display unit (3) in the interior of the vehicle (2), the method comprising the following steps: - capturing image information with a camera system (4) of the vehicle (2), which comprises at least one camera (4.1, 4.2), wherein the camera system (4) is designed to capture an environmental area (5) behind the vehicle (2) and at least one camera (4.2) of the camera system (4) is adapted for homologation for the road-approved mirror replacement (S10); - receiving driving situation information characterising the current driving situation of the vehicle (2) (S11); - Adapting the captured image information depending on the received driving situation information (S12); - Providing the adapted image information to the display unit (3) of the vehicle (2) (S13).

Description

[0001] The invention relates generally to driver assistance systems of vehicles. In particular, the invention relates to a method and a system for providing a rear surrounding area of ​​a vehicle.

[0002] Driver assistance systems for motor vehicles are already known that allow the driver to view the area behind the vehicle. For example, so-called reversing camera systems are known that capture the area behind the vehicle when reversing and display this information on a display unit in the vehicle interior. Furthermore, mirror replacement systems, also referred to as "camera monitor systems," are known that comprise at least one camera and a display unit provided in the vehicle interior. A mirror replacement system can, for example, replace an interior mirror of a vehicle, which captures the area behind the vehicle, or other vehicle mirrors. A mirror replacement system is particularly advantageous when the vehicle has a closed body at the rear or a structure that obstructs the view to the rear.

[0003] Known rearview systems, also referred to as rearview systems, feature a wide-angle lens that captures a very wide-angle image, also known as a "fisheye image." Such rearview systems must be switched off when driving forward above a certain speed because the image they provide, in terms of image quality and reliability, as well as the field of view, is only suitable for observing the surroundings at low speeds, and thus does not meet the safety requirements of a mirror replacement system.

[0004] Mirror replacement systems, on the other hand, meet the technical requirements for providing suitable images even when the vehicle is moving forward at higher speeds, but do not cover the close area behind the vehicle, which is relevant when reversing, nor the wide field of vision to the side.

[0005] In the event that both a reversing system and a mirror replacement system are installed in the vehicle, the driver assistance provided by these systems is unsatisfactory from an ergonomic point of view, since the driver is overwhelmed by the multitude of different perspectives and displays. This creates the risk that safety-critical events in the vehicle's surroundings will not be recognized by the driver in a timely manner due to the different displays and the display formats interrupted by switching processes.

[0006] Based on this, it is the object of the invention to provide a method for providing a rear surrounding area of ​​a vehicle, which enables an improved and combined, direction-independent detection of the surrounding area behind the vehicle and thus increases vehicle safety.

[0007] This object is achieved by a method having the features of independent patent claim 1. Preferred embodiments are the subject of the dependent claims. A system for providing a rear surrounding area of ​​a vehicle is the subject of independent patent claim 15.

[0008] According to a first aspect, a method for providing image information of the rear surrounding area of ​​a vehicle on at least one display unit in the interior of the vehicle is disclosed. The method comprises the following steps: First, image information is captured using a camera system on the vehicle. The camera system comprises at least one camera. The camera system is designed to capture the surrounding area behind the vehicle. At least one camera in the camera system is designed for homologation for the road-legal mirror replacement. This means that the camera provides image information that complies with the regulations and standards for images from mirror replacement cameras.

[0009] Driving situation information is received that characterizes the current driving situation of the vehicle. This driving situation information can, in particular, be the planned or actual direction of travel of the vehicle, for example, indicated by the selected driving mode (e.g., D or R, adjustable via the driving mode switch or gear selection on the vehicle's transmission) or steering movements induced by the detected steering angle.

[0010] The captured image information is then adjusted depending on the received driving situation information. The adjustment may, in particular, involve creating an image section from the entire image information provided by the camera system.

[0011] Finally, the adjusted image information is displayed on the vehicle’s display unit.

[0012] The technical advantage of the proposed method is that the camera system provides image information that offers a comprehensive view of the surrounding area behind the vehicle. Depending on the driving situation, this image information can then be used to extract an image area that is to be displayed on the display unit in the respective driving situation. This allows a confidence-building, natural-looking view of the vehicle's rear area to be created, tailored to the specific application, thus increasing safety when driving the vehicle.

[0013] According to one embodiment, receiving the driving situation information includes information about the vehicle's direction of travel. This allows the image information to be adapted depending on whether the vehicle is moving forward or backward. For example, in a reverse driving mode, partial image information can be generated from the image information that corresponds to the rearview image of a rearview camera. In contrast, in a forward driving mode, partial image information can be extracted from the image information that corresponds to the image information of a mirror replacement camera.

[0014] According to one embodiment, receiving the driving situation information includes the steering angle of the vehicle. The steering angle of the vehicle, which is provided, for example, by a steering angle sensor, is an indicator of where the vehicle will move in the future. The steering angle of the vehicle can preferably be used—in compliance with legal regulations—to adapt the image information. In particular, partial image information can be extracted from the overall image information based on the direction of travel information and the steering angle, preferably in such a way that the extracted partial image information is focused on the rear environmental area into which the vehicle will move in the future.

[0015] According to one embodiment, receiving the driving situation information includes information about at least one environmental area classified as critical by a driver assistance system of the vehicle. This environmental area can, for example, have been detected by another sensor system of the vehicle, such as an ultrasonic sensor system, a radar sensor system, etc. The image information can be adapted based on the environmental area classified as critical in such a way that partial image information is extracted from the image information that focuses on the environmental area classified as critical.

[0016] According to one embodiment, receiving the driving situation information can include speed information of the vehicle. Based on the speed information, the image information can then be adapted, in particular in such a way that, depending on the speed of the vehicle, a smooth change occurs between two different representations of rear surrounding areas on the display unit. This can be done, for example, in such a way that in a reversing driving mode, the near-field surrounding area, also referred to as a rearview display, is initially displayed and, after switching to a forward driving mode and exceeding a speed threshold, the mirror replacement display is then selected. This adaptation of the image information can be adapted and optimized for the respective application. In particular, this adaptation also takes place in a predictive manner orpredictive based on vehicle data and / or driver data.

[0017] According to one embodiment, adapting the captured image information comprises cropping the image information provided by the camera system to an area that is relevant based on the received driving situation information. This allows those partial image information that are particularly relevant in the respective driving situation to be displayed on the display unit, depending on the driving situation. The driver thus receives image information that is adapted to the driving situation on the display unit. This cropping of the image information can be adapted and optimized for the respective application. In particular, this adaptation also takes place in a forward-looking or predictive manner based on vehicle data and / or driver data.

[0018] According to one embodiment, adapting the captured image information comprises dewarping image information provided by at least one camera. This allows, for example, image regions that are distorted due to the camera's optics to be dewarped.

[0019] According to one exemplary embodiment, when the vehicle is intended to or has actually reversed, a first piece of image information representing a first area with a first depth behind the vehicle is cut out of the image information and displayed on the display unit. This can be, for example, a rearview representation of the close-up area to the rear of the vehicle. When the vehicle is moving forward, a second piece of image information representing a second area with a second depth behind the vehicle is cut out of the image information and displayed. This can be, for example, a mirror-replacement representation of the rear surrounding area. The first depth is smaller than the second depth. The first area represents a rear surrounding area that is closer to the vehicle than the second area.This allows the partial image information that advantageously supports the driver in the respective driving situation to be extracted from the image information provided by the camera system, depending on the direction of travel.

[0020] According to one embodiment, a dynamic crossfade occurs between the display of the first partial image information and the second partial image information when the vehicle's direction of travel changes. This provides the driver with a seamless display of the rear area of ​​the vehicle, which, depending on the driving situation, is focused on the relevant area. This does not distract or overwhelm the driver, but rather supports their natural perception.

[0021] According to one embodiment, the camera system comprises at least two cameras. A first camera is designed to capture a first area, the near area behind the vehicle. A second camera is designed as a camera that replaces the interior rearview mirror and is homologated for road-legal mirror replacement. The second camera captures a second area behind the vehicle that is further away than the near area. The camera system can thus generate the respective adapted image information based on the image information from the first camera and the second camera and display it on the display unit.

[0022] According to one embodiment, the image information of the camera system is formed by combining images from the first and second cameras. This combination is known as stitching. Furthermore, the images from the first and second cameras can be at least partially modified before combination, for example, by warping / dewarping, etc. This allows the images from different cameras to be combined to form overall image information, from which the partial image information relevant to the driving situation can then be extracted.

[0023] According to one exemplary embodiment, a view of the rear area of ​​the vehicle is displayed on the vehicle's display unit while the vehicle is traveling forward. This view of the rear area of ​​the vehicle is preferably permanent and replaces the information that can be captured via an interior rearview mirror. This allows the camera system to capture the area surrounding the vehicle's rear, regardless of speed, even while traveling forward. This is particularly advantageous for vehicles where no interior rearview mirror is provided or required, for example, because the rear view is obstructed by the body or a vehicle structure.

[0024] According to one exemplary embodiment, the representation of the rear area of ​​the vehicle on the display unit is adjusted depending on the driving situation in such a way that the displayed image information is, at least temporarily, tailored to safety-critical areas. These safety-critical areas can be provided, for example, by a driver assistance system. This allows critical surrounding areas to be displayed in a targeted and focused manner, which significantly increases vehicle safety.

[0025] According to a further aspect, a system for providing image information of the rear surrounding area of ​​a vehicle on at least one display unit in the interior of the vehicle is disclosed. The system comprises a camera system with at least one camera and a processing unit. The at least one camera of the camera system is homologated for road-legal mirror replacement. The processing unit is configured to perform the following steps: - capturing image information with the camera system of the vehicle, wherein the camera system is designed to capture an area of ​​the surroundings behind the vehicle; - Receiving driving situation information that characterizes the current driving situation of the vehicle; - Adapting the captured image information depending on the received driving situation information; - Providing the adapted image information to the vehicle's display unit.

[0026] The terms “approximately”, “essentially” or “about” mean, in the sense of the invention, deviations from the exact value by + / - 10%, preferably by + / - 5% and / or deviations in the form of changes that are insignificant for the function.

[0027] Further developments, advantages, and possible applications of the invention will become apparent from the following description of exemplary embodiments and from the figures. All described and / or illustrated features, individually or in any combination, are fundamentally part of the invention, regardless of their summary in the claims or their reference back to them. The content of the claims is also incorporated into the description.

[0028] The invention is explained in more detail below with reference to exemplary embodiments and the figures. They show: Fig. 1 shows, by way of example, a schematic plan view of a vehicle equipped with a camera system with two cameras for capturing the rear surrounding area of ​​the vehicle; Fig. 2 shows, by way of example, a schematic plan view of a vehicle and the sketch of a first and second region of the rear surrounding area of ​​the vehicle; Fig. 3 a schematic side view of a vehicle with a single camera for capturing the rear surrounding area; and Fig. 4 shows, by way of example, a diagram illustrating the method steps of a method for providing image information of the rear surrounding area of ​​a vehicle on at least one display unit in the interior of the vehicle.

[0029] Fig. 1 and Fig. 2 show, by way of example and schematically, a vehicle 2 equipped with a system 1 for detecting the rear surrounding area 5 of the vehicle 2. The vehicle 2 is in particular a motor vehicle, particularly preferably a panel van, transporter, or van, whose body or structure restricts or completely blocks the driver's view to the rear.

[0030] In the exemplary embodiment shown, the system 4 comprises a first and a second camera 4.1, 4.2, a computing unit 4.3 and a display unit 3. The second camera 4.2 is arranged, for example, on the roof area of ​​the vehicle 2, the first camera 4.1 in the rear area of ​​the vehicle (2), for example on the tailgate of the vehicle 2. Alternatively, the system 4 can only have the second camera 4.2 for detecting the rear surrounding area 5, which is arranged, for example, on the roof area of ​​the vehicle 2, as shown in Fig. 3 is shown.

[0031] The computing unit 4.3 is designed to receive and further process the image information provided by at least one camera 4.1, 4.2 of the vehicle 2, resulting in customized image information, as described in more detail below. This customized image information is displayed on the display unit 3 of the vehicle 2 so that it can be perceived by the driver of the vehicle 2.

[0032] At least one camera 4.2 of the camera system 4 is designed as a mirror replacement camera. This means that this camera is homologated for road-legal mirror replacement. Mirror replacement is legally regulated, for example, in the UN R.46 or FMVSS111 regulations. Technical properties of such mirror replacement cameras are standardized, for example, in the ISO16505 or SAE J3155 standards. With the camera 4.2 designed as a mirror replacement camera, it must be able to capture a rear surrounding area 5 of the vehicle 2 that is equivalent to the area that can be captured by the rearview mirror of the vehicle 2 or at least contains this area. The camera 4.2 is designed in terms of resolution, latency, and frame rate to deliver image sequences or videos in video stream quality.

[0033] The camera system 4 is designed to provide a representation of the surrounding area to the rear of the vehicle 2 on the display unit 3 of the vehicle depending on the driving situation, for example when the vehicle 2 is reversing or has already been reversed (in particular indicated by the vehicle's reverse driving mode R), a legally prescribed rearview representation of the close range and when the vehicle is traveling forward, a representation of the surrounding area to the rear of the vehicle 2 that is further away from the vehicle 2, which is prescribed according to the mirror replacement camera homologation.

[0034] In Fig. 2 shows a first area B1, which indicates the area that must be legally displayed by a rearview camera. The second area B2, on the other hand, indicates the portion of the rear surrounding area of ​​vehicle 2 that must be captured by a mirror-replacement camera and displayed on the display unit 3 of vehicle 2.

[0035] Depending on the driving situation of the vehicle 2, a representation of the rear surrounding area 5 adapted to the respective driving situation is displayed on the display unit 3.

[0036] The driving situation of vehicle 2 can be characterized by one or more of the following variables: - Driving direction information of vehicle 2; - Steering angle of vehicle 2; - Area surrounding the vehicle classified as critical 2; - Speed ​​of vehicle 2;

[0037] For example, in a reverse driving mode of vehicle 2, which is activated by engaging the R gear or the reverse gear, the Fig. 2, the first area B1 of the rear surrounding area 5 is displayed, ie the close area behind the vehicle 2. This close area can, for example, have a first depth t1 of 20 m to 40 m, in particular 20 m to 30 m, viewed in the longitudinal direction of the vehicle, wherein the representation of the close area includes at least part of the rear bumper. In a direction perpendicular to the longitudinal axis of the vehicle, the first area B1 can have a first width b1 of at least 20 m. This allows the close area behind the vehicle 2 to be detected when the vehicle 2 is reversing.

[0038] However, if the forward driving mode of vehicle 2 is activated, for example by engaging the “D” gear or the forward gear, the Fig. 2, i.e. the area behind the vehicle 2 which is further away compared to the near area and which can also be detected, for example, by means of an interior rear-view mirror of the vehicle 2. This second area B2 can, for example, have a second depth t2, seen in the longitudinal direction of the vehicle, which is greater than the first depth t1 of the first area B1. In addition, the second area B2 does not begin at the bumper of the vehicle 2, but at a distance from it, for example 5 m or more from the bumper. In a direction perpendicular to the longitudinal axis of the vehicle 2, the second area B2 can have a second width b2 of at least 20 m. The driver of the vehicle 2 can therefore see the second area B2 on the display unit 2 when driving forward, so that this representation of the second area B2 replaces a rear-view mirror of the vehicle.

[0039] The image information can also be adapted depending on the vehicle speed and the partial image information obtained thereby can be displayed on the display unit 2, for example in such a way that when driving slowly forward in the city, partial image information for a second area with a greater width and smaller depth is displayed than when driving faster forward, for example on the motorway.

[0040] In addition, the area displayed on the display unit 3 can be changed depending on the steering angle of the vehicle 2. For example, when the vehicle 2 is reversing to the right, the first area B1 of the rear surrounding area 5 displayed on the display unit 3 may not be centrally aligned with respect to the longitudinal axis of the vehicle 2, but rather the first area B1 may be shifted to the right, so that a larger sub-area is displayed to the right of the vehicle 2 and a smaller sub-area is displayed to the left of the vehicle 2. This allows the focus of the displayed partial image information to be placed on the area into which the vehicle 2 is moving.

[0041] Furthermore, the image information can be adapted depending on information from the vehicle's driver assistance system or the associated additional sensors. If the driver assistance system or the sensors of vehicle 2 detect a critical area with respect to which the driver of vehicle 2 should be assisted, for example, a third-party vehicle, an object, etc., that must be taken into account when controlling the vehicle, the image information can be adapted and partial image information displayed in such a way that this critical area is specifically represented in the partial image information. This can be achieved, for example, by the partial image information being specifically focused on the rear surrounding area containing this critical area.

[0042] In a first embodiment, the vehicle 2 has only a single camera 4.2, which is arranged in the rear area of ​​the vehicle 2. This camera 4.2 is configured in such a way that it provides image information that covers both the area B1 and the area B2 according to Fig. 2. In other words, the camera 4.2 covers the detection range of a rear view camera (ie the area B1 according to Fig. 2) and the detection range of a mirror replacement camera (ie the area B2 according to Fig. 2) off.

[0043] In a second exemplary embodiment, the vehicle 2 has two cameras in the rear area, namely a first camera 4.1 and a second camera 4.2. The first camera 4.1 is a rear view camera that provides image information about the rear close range of the vehicle 2, for example the first area B1 according to Fig. 2, but not the second area B2 according to Fig. 2. The second camera 4.2, however, is designed as a mirror replacement camera and captures, for example, the second area B2 but not the first area B1 according to Fig. 2.

[0044] In this case, the images provided by the first and second cameras 4.1, 4.2 are first combined to form a common image information. This is known as stitching. The images can be separated from each other by a dividing line.

[0045] From the image information provided by one or more cameras 4.1, 4.2, an image section is then created depending on the driving situation, i.e. an image region is cut out of the overall image information and this image section is shown on the display unit. The image section can be created, for example, by what is known as cropping. In addition, what is known as dewarping of the image information can occur. This allows adapted image information to be generated from image information provided by the camera system 4. In particular, changes to the viewing direction, the aperture angle and / or dynamic distortion can be made to the image information provided by the camera system 4. This allows adaptive and jump-free or interruption-free image changes to be made that are permissible within the statutory provisions and that assist the driver in controlling the vehicle 2.

[0046] In particular, when switching between a driving mode for reversing and driving the vehicle 2 forward, a smooth transition can occur from a rear-view camera image displaying the first area B1 of the rear surrounding area 5 to a mirror replacement camera image displaying the second area B2 of the rear surrounding area 5. Thus, when changing the direction of travel from reversing to forward, the rear-view camera image is dynamically converted into a mirror replacement camera image, so that the driver receives a representation of the rear surrounding area 5 on the display unit 3 even when driving forward, preferably in a form similar to that shown in an interior rearview mirror of a vehicle 2. This is particularly advantageous for vehicles 2 with no or limited rear visibility, such as is the case with vans, panel vans, or trucks.This mirror replacement display can be permanent or essentially permanent on the display unit as long as it has not been deactivated by the vehicle user.

[0047] Fig. 4 shows a schematic flow diagram illustrating the method steps for providing a rear surrounding area 5 of a vehicle 2 on at least one display unit in the interior of the vehicle 2.

[0048] First, image information is captured using a camera system on the vehicle. The camera system comprises at least one camera designed to capture the surrounding area behind the vehicle. At least one camera in the camera system has specifications that allow it to be homologated for use as a road-legal mirror replacement (S10).

[0049] Driving situation information is received that characterizes the current driving situation of the vehicle (S11).

[0050] The captured image information is then adjusted depending on the received driving situation information (S12).

[0051] Finally, the adjusted image information is provided to the vehicle’s display unit (S13).

[0052] The invention has been described above using exemplary embodiments. It is understood that numerous changes and modifications are possible without departing from the scope of protection defined by the patent claims. List of reference symbols 1 system 2 vehicles 3 Display unit 4 camera system 4.1 first camera 4.2 second camera 4.3 Computing unit 5 rear surrounding area b1 first width b2 second width B1 first area B2 second area t1 first depth t2 second depth

Claims

[1] Method for providing image information of a rear surrounding area (5) of a vehicle (2) on at least one display unit (3) in the interior of the vehicle (2), the method comprising the following steps: - capturing image information with a camera system (4) of the vehicle (2), which comprises at least one camera (4.1, 4.2), wherein the camera system (4) is designed to capture an environmental area (5) behind the vehicle (2) and at least one camera (4.2) of the camera system (4) is adapted for homologation for the road-approved mirror replacement (S10); - receiving driving situation information characterising the current driving situation of the vehicle (2) (S11); - Adapting the captured image information depending on the received driving situation information (S12); - Providing the adapted image information to the display unit (3) of the vehicle (2) (S13). [2] Method according to claim 1, characterized by that the receiving of the driving situation information includes driving direction information of the vehicle (2). [3] Method according to claim 1 or 2, characterized by that the receiving of the driving situation information includes the steering angle of the vehicle (2). [4] Method according to one of the preceding claims, characterized by that the receiving of the driving situation information comprises information on at least one environmental area classified as critical by a driver assistance system of the vehicle (2). [5] Method according to one of the preceding claims, characterized by that receiving the driving situation information includes speed information of the vehicle (2). [6] Method according to one of the preceding claims, characterized bythat adapting the captured image information comprises cropping the image information provided by the camera system (4) to an area that is relevant based on the received driving situation information. [7] Method according to one of the preceding claims, characterized by that adapting the captured image information comprises rectifying image information provided by at least one camera (4.1, 4.2). [8] Method according to one of the preceding claims, characterized bythat when the vehicle (2) is intended to or has actually reversed, a first piece of image information representing a first area (B1) behind the vehicle (2) is cut out of the image information with a first depth (t1) and displayed on the display unit (3), and that when the vehicle (2) is moving forward, a second piece of image information representing a second area (B2) behind the vehicle (2) is cut out of the image information with a second depth (t2) and displayed, wherein the first depth (t1) is smaller than the second depth (t2) and wherein the first area (B1) represents a rear surrounding area that is located closer to the vehicle (2) than the second area (B2). [9] Method according to claim 8, characterized by that when the direction of travel of the vehicle (2) changes, a dynamic cross-fading takes place between the display of the first partial image information and the second partial image information. [10] Method according to one of the preceding claims, characterized by that the camera system (4) comprises at least two cameras (4.1, 4.2), wherein a first camera (4.1) is designed to capture a first area (B1) as a close-up area behind the vehicle (2) and a second camera (4.2) is designed as a camera replacing the rear-view mirror and is homologated for road-approved mirror replacement, wherein the second camera (4.2) captures a second area (B2) behind the vehicle (2) which is further away than the close-up area. [11] Method according to claim 10, characterized by that the image information is formed by combining images from the first and second cameras (4.1, 4.2). [12] Method according to one of the preceding claims, characterized by that while the vehicle (2) is traveling forward, the rear area of ​​the vehicle (2) is displayed on the display unit (3) of the vehicle (2). [13] Method according to claim 12, characterized by that the representation of the rear area of ​​the vehicle (2) as a mirror replacement, in the event of its activation, is permanent or essentially permanent while the vehicle (2) is traveling forward. [14] Method according to claim 12 or 13, characterized by that the representation of the rear area of ​​the vehicle (2) on the display unit (3) is adapted depending on the driving situation in such a way that the displayed image information is at least temporarily tailored to safety-critical areas. [15] System for providing image information of the rear surrounding area (5) of a vehicle (2) on at least one display unit (3) in the interior of the vehicle (2), wherein the system (1) comprises a camera system (4) with at least one camera (4.1, 4.2) and a computing unit (4.3), wherein at least one camera (4.2) of the camera system (4) is adapted for homologation for the road-legal mirror replacement and wherein the computing unit (4.3) is configured to perform the following steps: - capturing image information with the camera system (4) of the vehicle (2), wherein the camera system (4) is designed to capture an environmental area (5) behind the vehicle (2); - receiving driving situation information that characterizes the current driving situation of the vehicle (2); - Adapting the captured image information depending on the received driving situation information; - Providing the adapted image information to the display unit (3) of the vehicle (2).

Images