Vehicle and method for detecting a water crossing in a vehicle

Abstract

The invention relates to a method for detecting a water crossing in a vehicle (1) using SAR data on water depth from a satellite, comprising the steps: - Detection of a water surface ahead in a direction of travel (3), - Retrieving SAR data on water depth for the determined water area (3) and visualizing the water area (3) in a forward area of ​​the vehicle (1) with the retrieved water depths on a display device of the vehicle (1).

Description

[0001] The invention relates to a method for detecting a water crossing in a vehicle according to the preamble of claim 1 and a vehicle according to the preamble of claim 9.

[0002] Today's driver assistance systems support the driver in many ways. Besides providing assistance with longitudinal and lateral control, visualization systems such as navigation assistance or vehicle stability control (SVS) are also important. In addition to assisted driving, manual driving is also supported by appropriate visualizations and vehicle responses. Off-road driving is a prime example. During such journeys, flooding or driving off-road can lead to sections of the road that are partially or completely submerged. Such areas are critical for the vehicle (electric or combustion engine) because, for example, the engine's intake manifold can become flooded, or the battery or other electrical components can come into contact with water.Furthermore, the interior seals are not designed for continuous water exposure, meaning that the interior can also become flooded after a certain period. It is therefore desirable to provide a system that detects water crossings and water depths. This is achieved using vehicle sensors in the current state of the art. For example, systems with ultrasonic sensors for detecting water depths up to 60 cm and with an additional optional water depth sensor for depths up to 90 cm are known. However, the need for an additional sensor in the vehicle increases the risk of failure and maintenance costs. Moreover, such systems cannot detect water predictively; instead, they detect the depth directly at the vehicle. Detection can sometimes occur too late if the vehicle is already in the water above the fording depth.Other options for water level detection include using so-called SVS cameras and / or fisheye cameras integrated into the side mirrors. This method uses active triangulation and an additional light source to measure the depth relative to the mirror. A disadvantage of this approach is that measurements are not possible when crossing water in murky or polluted conditions. Furthermore, this method cannot measure predictively; the measurement is taken directly at the vehicle. Therefore, the motivation is to provide a system that can measure water depth predictively and / or remotely, particularly regardless of the water's composition (still water, river, etc.) and clarity.

[0003] US 2016 / 0272066 A1 describes a vehicle comprising a system for assisting the driver's control of the vehicle when the vehicle is wading through a body of water, wherein the system includes a measuring device for determining the measured water depth through which the vehicle is wading. The measuring device is positioned and arranged relative to the vehicle such that the measured depth indicates the water depth in a first measuring range relative to the vehicle itself. The processor is coupled to the measuring device and configured to calculate an estimated water depth as a function of the measured depth and the vehicle speed.

[0004] US 2024 / 0354976 A1 describes a device and a method for measuring water depth, wherein an underwater environment, such as the shape of a seabed surface, is detected by analyzing a SAR image using a Synthetic Aperture Radar (SAR).

[0005] DE 10 2023 127 223 A1 describes a known method for controlling a vehicle on a flooded roadway, wherein a water detection module is used to sensorially detect the properties of the surface water.

[0006] German patent DE 10 2021 002 496 A1 discloses a method for determining a safe wading route through a body of water. To determine the water depth, the method involves using sensors to observe an occupant of the vehicle and drawing conclusions about the water depth and the safe wading route.

[0007] From DE 10 2021 001 624 A1 a method for crossing a flooded area of ​​a road is known, wherein it is intended to determine a flow velocity and to predict its influence on the safe fording depth of the vehicle.

[0008] DE 10 2018 112 269 A1 discloses a method for characterizing a body of water on the roadway, whereby critical aspects of the body of water, e.g. depth or bottom profile, are to be determined by means of vehicle-mounted sensors.

[0009] The invention is based on the objective of providing a novel method for detecting a water crossing in a vehicle and a novel vehicle.

[0010] The problem is solved according to the invention by a method for detecting a water crossing in a vehicle with the features of claim 1 and a vehicle with the features of claim 9.

[0011] Advantageous embodiments of the invention are the subject of the dependent claims.

[0012] A method for detecting a water crossing in a vehicle using SAR (Synthetic Aperture Radar) data on water depth from a satellite is proposed, comprising the following steps: - Detection of a body of water ahead in a direction of travel, - Retrieving SAR data on water depth for the identified water area, and - Visualization of the water surface ahead with the retrieved water depths in a forward area of ​​the vehicle on a display device of the vehicle.

[0013] According to the invention, the detection of the water surface ahead in the direction of travel is carried out independently of and before the retrieval of SAR data, since the retrieval of SAR data is time-consuming, computationally intensive, and costly. In this way, unnecessary SAR data retrievals can be avoided. The determination of water depths is, for example, disclosed in the article "Shallow Water Bathymetric Surveys by Spaceborne Synthetic Aperture Radar" by WG Huang, B. Fu, CB Zhou, JS Yang, AQ Shi and DL Li, Lab. of Ocean Dynamic Processes and Satellite Oceanography, Second Institute of Oceanography, State Oceanic Administration, PO Box 1207, Hangzhou, 310012, China.

[0014] In one embodiment, data can be read from an electronic map to detect the water surface ahead.

[0015] In one embodiment, measuring points, in particular vertices, are defined in a forward area along a driving trajectory. The height of these measuring points is determined using a sensor system for detecting the surroundings, in particular at least one camera. The forward area is recognized as the surface of a standing body of water if, particularly in the absence of characteristic features of a road, such as road markings and / or edge boundaries, the height differences between the measuring points are below at least a certain threshold. The height differences can be quantified, for example, by fitting a straight line through the vertices of a segment and determining its slope and / or the variance of the vertices' heights. Potentially standing water can be detected, for example, if the heights of the vertices of a segment do not exceed certain thresholds for slope and / or variance.

[0016] In a further or additional embodiment, optical flow vectors are determined and evaluated in the preceding area along the driving trajectory, wherein the preceding area is recognized as the water surface of a flowing body of water if, in the absence of characteristic features of a road, in particular road markings and / or edge boundaries, a histogram of the optical flow vectors with two peaks is detected that exceed a certain predetermined frequency threshold.

[0017] In one embodiment, the area ahead, identified as a water surface, is segmented and transformed into a SAR coordinate system, and water depths are read and assigned from the SAR data for segmentation.

[0018] In one embodiment, the segmentation is reverse-transformed with the associated water depths into a vehicle coordinate system.

[0019] In one embodiment, the determined water depths are compared with a stored possible wading depth of the vehicle, whereby, if the water depth is greater than the wading depth, actuators are controlled to raise the vehicle in order to increase the wading depth.

[0020] In one embodiment, if a vehicle is at least partially automated and the fording depth is insufficient, the route is replanned and the vehicle's actuators that perform the driving activity are controlled to drive the replanned route.

[0021] According to one aspect of the present invention, a vehicle, for example a passenger car, a commercial vehicle or a bus, is proposed to be equipped with a device for detecting a water crossing and sensors for sensing the vehicle's surroundings. According to the invention, the device is configured to carry out the method described above.

[0022] In one embodiment, the vehicle is designed as a vehicle that drives at least partially automatically.

[0023] The solution according to the invention enables predictive detection of water depths regardless of the depth to be detected, water quality, water clarity, and the light and weather conditions, and without driver input. Furthermore, the need for additional sensors and their integration is eliminated. The solution according to the invention is globally functional and also works when stationary, i.e., without any movement of the vehicle. Moreover, the solution according to the invention adaptively detects the presence of water bodies in the vehicle's trajectory. The solution according to the invention is based on standard components, represents a purely software-based adaptation, and requires no additional vehicle hardware.

[0024] This invention disclosure presents an approach for detecting water depth using SAR (Synthetic Aperture Radar) data and providing this information to the vehicle for water crossings. First, all system components involved are transferred to a higher-level coordinate system and / or calibrated. Then, the roadway is detected using the camera. Alternatively, the distance traveled can be estimated based on steering wheel movement. Since accessing satellite data and / or converting between coordinate systems can be computationally intensive, various conditions are checked beforehand to determine the potential presence of a body of water. This is implemented based on two facts: bodies of water are usually either still (standing water, flooded roads, etc.) or flowing (rivers).For standing water bodies, it can be assumed that the surface is homogeneous. For flowing water bodies, the optical flux, which constitutes the current, diverges from the surroundings. First, a 3D detection, i.e., a point cloud, is determined along the extracted route for the detection of standing water bodies. This can be achieved, for example, using radar, a camera, or lidar. Next, a plane is fitted into this point cloud for cyclical sections along the route. If the variance along the plane is below a defined threshold and the absence of road markings is confirmed, then a water crossing is defined, and the subsequent process steps are implemented. Naturally, water crossings are also necessary for rivers. This is determined via the optical flux.Here, a histogram of the optical flow vectors is determined along the route (and dynamic objects are removed beforehand, as these would distort the histogram). When a water crossing is present, the histogram becomes bimodal with two peaks, the second peak being caused by the water flow. If two maxima are detected in the histogram that exceed a certain threshold, the potential water crossing is identified, and the subsequent process steps are carried out analogously to those for still water. Alternatively, the process steps are repeated. The preceding route profile, which is stored in the ego coordinate system, is then converted into the coordinate system of the SAR database (usually WGS-84) using the known GPS information (longitude, latitude, and course).There, the water column is retrieved for an array of vertices spaced x meters apart and transformed back into the ego vehicle, thus determining the water depth relative to the front axle. Based on this information, further actions can be performed. These include adaptively raising the suspension, providing the driver with visual information about the current water level, calculating evasive trajectories around the critical deepest point, displaying alternative routes, and so on. The advantage of the present invention is the adaptive and predictive provision of the water depth before the vehicle reaches the body of water. Likewise, the approach requires no additional optional sensors and is independent of the water's composition or clarity. Furthermore, it allows for worldwide coverage and offers further optimization potential through the ongoing and continuous development in the field of SAR (Search and Rescue) technology.Furthermore, it is also independent of the prevailing weather and lighting conditions and requires no manual input from the driver.

[0025] Exemplary embodiments of the invention are explained in more detail below with reference to drawings.

[0026] This shows: Fig. 1. A schematic flowchart of a procedure for detecting a water crossing in a vehicle, Fig. 2 a schematic view of a forward route with a water surface with a multitude of vertices, Fig. 3 a schematic view of the upcoming route with the water surface and various horizontal segments, Fig. 4 a schematic diagram of the height of the vertices of one of the segments in the static coordinate system, Fig. 5 a schematic diagram of the height of the vertices of another of the segments in the static coordinate system, Fig. 6 a schematic view of a preceding route with the water surface and optical flow vectors, Fig. 7 a schematic histogram of the orientation of the optical flow vectors, Fig. 8 a schematic view of the ahead route with the water surface and its segmentation into tiles, Fig. 9 a schematic view of the ahead route with the water surface and its segmentation with the tiles in the SAR coordinate system, and Fig. 10 A schematic view of the vehicle as it passes through the detected water surface.

[0027] Corresponding parts are marked with the same reference symbols in all figures.

[0028] According to the invention, a method is proposed comprising the following steps: -Detection of a water surface area lying ahead in the direction of travel, i.e., one lying along a travel trajectory of the vehicle 1, - Retrieving SAR (Synthetic Aperture Radar) water depth data for the identified water area, - Visualization of the water surface area in a forward area of ​​vehicle 1 into a display device of vehicle 1.

[0029] In one embodiment, it may be provided that the water surface 3, even if it is not recorded on a map, is detected in advance before a data retrieval takes place, since the retrieval of the SAR data is time-consuming, computationally intensive and costly.

[0030] In one embodiment, water data is therefore read from an electronic map to detect the preceding water surface area.

[0031] In one embodiment, to detect the ahead water area (if none is shown on the map), measuring points (vertices) are defined in the ahead area along the driving trajectory, whereby the height of the measuring points relative to each other is determined with a sensor, for example a front wide camera (FWC), wherein the ahead area is defined as a water area with a standing body of water if the height differences are within a predetermined tolerance range in the absence of characteristic road features (markings, edge boundaries, etc.).

[0032] In one embodiment, to detect the ahead water area (if none is shown on the map), optical flow vectors 5 are determined and evaluated in the ahead area along the driving trajectory, whereby, in the absence of characteristic road features (markings, edge boundaries, etc.), a flowing body of water is defined based on the formation of a histogram of the optical flow.

[0033] In one embodiment, the area ahead, defined as water surface 3, is segmented and transferred to a coordinate system common to the SAR, and the water depths are read from the SAR data.

[0034] In one embodiment, a reverse transformation into a vehicle coordinate system is performed.

[0035] In one embodiment, the determined water depth is compared with a stored possible wading depth of the vehicle 1. If the water depth is greater than the wading depth, actuators can be controlled to raise the vehicle 1 in order to increase the wading depth.

[0036] In the case of a vehicle 1 that is at least partially automated, if the fording depth is insufficient (even with raised chassis), the route 2 can be replanned and the actuators performing the driving activity can be controlled.

[0037] Fig. Figure 1 is a schematic flowchart of a procedure for detecting a water crossing in a vehicle.

[0038] In step VS1, the system components involved in vehicle 1, such as a driver observation camera, are calibrated in a reference coordinate system. After calibration, the extrinsic position of the system components relative to an ego-coordinate system of vehicle 1 is known.

[0039] In step VS2, a trajectory and / or lane to be traveled by vehicle 1 is detected, for example, using data from at least one camera. This means that the trajectory planned by vehicle 1 can be identified in the data captured by the camera.

[0040] In step VS3, a 3D reconstruction of the road ahead is performed using data acquired from the vehicle's environmental sensors (VS1), which include, for example, at least one camera and / or at least one lidar sensor and / or at least one radar sensor. An array of three-dimensional vertices 4 is then superimposed onto the road identified in step VS2, i.e., the physical route 2.

[0041] Fig. Figure 2 is a schematic view of a roadway 2 ahead with a water surface 3 and a multitude of vertices 4. The transparency of the vertices 4 can represent a height z within a static coordinate system. Constant transparency of several vertices 4 means that there is no difference in height between them.

[0042] In step VS4, potentially standing water is detected along the preceding driving path 2, for example using data recorded by a camera, such as a front-wide camera, or using data from the 3D reconstruction from step VS3.

[0043] Fig. Figure 3 is a schematic view of the preceding route 2 with the water surface 3 and various horizontal segments S1, S2, S3.

[0044] Fig. Figure 4 is a schematic diagram of the height z of vertices 4 of segment S1 in the static coordinate system. It can be seen that all vertices 4 of segment S1 have nearly the same height z. A straight line G fitted by the vertices 4, for example, has a very low slope of m = 0.02. The variance of the heights z of the vertices 4 of segment S1 is, for example, 0.12. Potentially standing water can be detected, for example, if the heights z of the vertices 4 of one of the segments S1, S2, S3 do not exceed certain threshold values ​​for slope and / or variance.

[0045] Fig. Figure 5 is a schematic diagram of the height z of the vertices 4 of segment S3 in the static coordinate system. A straight line G fitted by the vertices 4, for example, has a significantly higher slope of m = 5.82. The variance of the heights z of the vertices 4 of segment S3 is, for example, 0.74. The heights z of the vertices 4 of segment S3 exceed, for example, the specified threshold values ​​for the slope and / or the variance, thus indicating that there is no potentially standing water in segment S3.

[0046] If no potentially standing water is detected upstream, then step VS5 is performed. Otherwise, the process continues with step VS6.

[0047] In step VS5, the detection of potentially flowing water along the preceding roadway is performed by comparing optical flow vectors 5. The optical flow is determined in the 3D reconstruction of the preceding roadway or in the acquired data from the environmental sensors. Here, a histogram of the optical flow vectors 5 is determined along the driving path 2 (and dynamic objects are removed from this beforehand, as they would distort the histogram). If flowing water is detected, a step to VS6 is also performed; otherwise, a step to VS3 is performed.

[0048] Fig. Figure 6 is a schematic view of a preceding route 2 with the water surface 3 and optical flow vectors 5.

[0049] Fig. Figure 7 is a schematic histogram of the orientation of the optical flux vectors. 5. In the histogram, a bimodal histogram with two peaks, P1 and P2, is displayed when a water crossing by a flowing body of water is detected, with the second peak, P2, being caused by the water flow. If two maxima or peaks, P1 and P2, are detected in the histogram that exceed a certain predefined frequency threshold, then a potential water crossing by a flowing body of water is detected, and the subsequent process steps are executed analogously to those for a still body of water. If no potentially flowing water is detected, the process returns to step VS3. Otherwise, it continues at step VS6.

[0050] In step VS6, the segmented route from step VS2 is transformed into a SAR coordinate system (for example, WGS84).

[0051] Fig. Figure 8 is a schematic view of the upcoming route 2 with the water surface 3 and its segmentation into tiles T, which, for example, have a size of 25 cm x 25 cm. The segmented route, which is available in the Ego coordinate system from step VS2, is converted into the coordinate system of a SAR database (usually WGS-84) using known GPS information (longitude, latitude, and course).

[0052] Fig. Figure 9 is a schematic view of the preceding route 2 with the water surface 3 and its segmentation with the tiles T in the SAR coordinate system.

[0053] In step VS7, the SAR data for water depth is assigned to the transformed and segmented route profiles from step VS6, i.e., to the individual tiles T. For example, the water depth is retrieved for an array of vertices 4 with an equidistant distance of x meters.

[0054] In step VS8, the data from step VS7 is transformed back into the Ego coordinate system using the known GPS information (Long, Lat and Course), so that the data in Fig. The 7 tiles shown are each assigned a water depth.

[0055] In step VS9, vehicle 1 reacts, for example by raising its suspension, visualizing the detected water crossing and / or planning a different route.

[0056] Fig. Figure 10 is a schematic view of vehicle 1 as it passes through the detected water surface 3. The predictive detection of water depth enables, for example, adaptive raising of the suspension. The water depth can be symbolized, for example, by different shades of color of the tiles T on a display unit of vehicle 1, where, for example, a light blue can indicate a shallower depth than a darker blue.

[0057] In step VS10, there is a return to step VS2, so that steps VS2 to VS9 are repeated cyclically based on an interval or event.

Claims

[1] Method for detecting a water crossing in a vehicle (1) using SAR water depth data from a satellite, comprising the steps: - Detection of a water surface ahead in a direction of travel (3), - Retrieving SAR data on water depth for the determined water area (3) and visualizing the water area (3) in a forward area of ​​the vehicle (1) with the retrieved water depths on a display device of the vehicle (1). [2] Method according to claim 1, characterized by , that data from an electronic map are read to detect the water surface ahead (3). [3] Method according to claim 1 or 2, characterized by, that in a preceding area along a driving trajectory measuring points, in particular vertices (4), are defined, wherein a height (z) of the measuring points is determined using a sensor system for detecting an environment, in particular at least one camera, wherein the preceding area is recognized as a water surface (3) of a standing body of water, if, in the absence of characteristic features of a road, in particular road markings and / or edge boundaries, height differences between the measuring points are below at least a certain threshold value. [4] Method according to any one of the preceding claims, characterized by, that optical flow vectors (5) are determined and evaluated in the preceding area along the driving trajectory, wherein the preceding area is recognized as a water surface (3) of a flowing body of water if, in the absence of characteristic features of a road, in particular road markings and / or edge boundaries, a histogram of the optical flow vectors (5) with two peaks (P1, P2) is detected which exceed a certain predetermined frequency threshold. [5] Method according to any one of the preceding claims, characterized by , that a segmentation of the preceding area identified as water surface (3) and its transformation into a SAR coordinate system and a reading and assignment of water depths from the SAR data for segmentation takes place. [6] Method according to claim 5, characterized by , that a reverse transformation of the segmentation with the associated water depths into a vehicle coordinate system takes place. [7] Method according to any one of the preceding claims, characterized by , that a comparison of the determined water depths with a stored possible wading depth of the vehicle (1) is carried out, whereby if the water depth is greater than the wading depth, actuators to raise the vehicle (1) to increase the wading depth are controlled. [8] Method according to any one of the preceding claims, characterized by , that in the case of a vehicle (1) that is at least partially automated, if the fording depth is insufficient, the route will be replanned and the vehicle's actuators (1) that perform the driving activity will be controlled to drive on the replanned route. [9] Vehicle (1) with a device for detecting a water crossing and sensors for detecting the area around the vehicle (1), characterized by that the device is configured to carry out the method according to one of the preceding claims. [10] Vehicle (1) according to claim 9, characterized by that the vehicle (1) is designed as a vehicle capable of driving at least partially automatically (1).

Images