Automated visual perception by vehicles using camera and ultrasonic sensor systems

Abstract

According to a method for automated visual perception, a first feature map (17) is generated from a camera image (7) by a first encoder module (11) of a neural network (6), and the first feature map (17) is transformed to a top-view perspective. Ultrasonic pulses are emitted into the environment, and an ultrasonic sensor signal (8) is generated in response to reflected portions of the emitted ultrasonic pulses. A spatial ultrasonic map (9) is generated in response to the ultrasonic sensor signal (8), and a second feature map (22) is generated from the ultrasonic map (9) by a second encoder module (12) of the neural network (6). The transformed first feature map (20) and second feature map (22) are fused, and a visual perception task is performed by decoder modules (15a, 15b, 15c) of the neural network (6) in response to the fused feature map.

Description

[Technical Field]

[0001] The present invention is directed to a method for automated visual perception by a vehicle, an electronic vehicle guidance system for a vehicle, a vehicle having such an electronic vehicle guidance system, and a computer program product. [Background technology]

[0002] Reliable automatic detection of objects in the environment of the ego-vehicle is crucial for partially or fully automatic driving functions. It is known to use algorithms for automatic visual perception, also referred to as computer vision algorithms, based on camera images from the ego-vehicle's camera to detect, localize, and / or characterize objects in the environment of the ego-vehicle. Typical visual perception tasks include object detection tasks, object bounding box detection, semantic segmentation tasks, object size regression, object height regression, etc.

[0003] However, for certain environmental conditions, object types, or object locations within the vehicle's environment, reliable recognition and characterization of objects is a challenging task for known algorithms for automatic visual recognition, resulting in reduced reliability and / or accuracy of the corresponding output. For example, such environmental conditions include low-light scenarios, such as twilight or nighttime scenarios, where camera images contain very limited information. Furthermore, detecting and characterizing objects in the vehicle's environment that have a certain low height from the ground, such as curbs, low walls, poles, or unclassified or unknown objects, is particularly challenging. This is even more true when such objects are to be characterized and detected in low-light scenarios.

[0004] Algorithms for automated visual perception can be based on trained artificial neural networks, particularly convolutional neural networks (CNNs). In the paper by T. Roddick and R. Cipolla, "Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks," 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, or the corresponding preprint arXiv:2003.13402v1 (hereafter "Roddick and Cipolla"), the authors describe a method for estimating a bird's-eye view map of an autonomous vehicle's environment directly from monocular images using a single end-to-end deep learning architecture. The architecture consists of a backbone network, a feature pyramid network, a bird's-eye view transformation network, and a top-down network. The backbone network, which can be based on ResNet-50, extracts image features at multiple resolutions from the input image, and the feature pyramid network augments high-resolution features from lower pyramid layers with spatial context. A stack of dense transformation layers in the bird's-eye view transformation network maps image-based features to bird's-eye views, and a top-down network processes the bird's-eye view features and predicts the final semantic occupancy probability.

[0005] The feature pyramid network is based on the publication T. Lin et al., "Feature Pyramid Networks for Object Detection," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, or the corresponding preprint arXiv:1612.03144v2 (hereinafter "Lin et al."). [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] T. Roddick and R. Cipolla “Predicting Semantic Map Representations from Images using Pyramid Occupancy Networks”, 2020 IEEE / CVF Conference on Computer Vision and Pattern Recognition(CVPR), 2020 [Non-patent document 2] T. Lin et al., “Feature Pyramid Networks for Object Detection”, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition(CVPR), 2017 Summary of the Invention [Problem to be solved by the invention]

[0007] An object of the present invention is to increase the reliability and / or accuracy of automated visual perception by vehicles. [Means for solving the problem]

[0008] This object is achieved by the subject matter of the respective independent claims. Further implementations and preferred embodiments are the subject matter of the dependent claims.

[0009] The invention is based on the idea of ​​extracting features from camera images by a trained artificial neural network, transforming them into a top-view perspective, and fusing the transformed features with features extracted by the neural network from an ultrasonic map of the environment obtained from ultrasonic sensor signals of the vehicle's ultrasonic sensor system. A visual perception task is then performed depending on the fused features.

[0010] According to one aspect of the present invention, there is provided a method for automated visual perception by a vehicle, particularly an automobile. According to the method, a camera image representing the vehicle's environment is generated by a camera of the vehicle. At least one first feature map is generated, particularly by at least one computing unit of the vehicle, by applying a first encoder module of a trained artificial neural network to the camera image. A top-view transformation module of the neural network, particularly by the at least one computing unit, is applied to the at least one first feature map to transform the at least one first feature map from a camera image plane perspective of the camera to a top-view perspective, also referred to as a bird's-eye view or a bird's-eye view perspective.

[0011] Ultrasonic pulses are emitted into the environment by the vehicle's ultrasonic sensor system, and at least one ultrasonic sensor signal is generated by the vehicle's ultrasonic sensor system in response to a reflected portion of the emitted ultrasonic pulse. A spatial ultrasonic map, particularly a two-dimensional spatial ultrasonic map, is generated in a top-view perspective in response to the at least one ultrasonic sensor signal, particularly by at least one computing unit. At least one second feature map is generated by applying a second encoder module of a neural network to the ultrasonic map. A set of fused feature maps is generated by fusing the transformed at least one first feature map and the at least one second feature map, particularly by at least one computing unit, for example by applying a fusion module of the neural network to the transformed at least one first feature map and the at least one second feature map. A first visual perception task is performed by a first decoder module of the neural network in response to the set of fused feature maps, particularly by at least one computing unit using the first decoder module of the neural network in response to the set of fused feature maps.

[0012] The method according to the invention for automated visual perception can also be considered as a method comprising the described steps of generating a camera image and at least one ultrasonic sensor signal, and applying an algorithm for automated visual perception to the camera image and the at least one ultrasonic sensor signal. The algorithm for automated visual perception, which may also be denoted a computer vision algorithm or a machine vision algorithm, comprises a trained artificial neural network.

[0013] An algorithm for automated visual perception can be considered a computer-implemented algorithm for automatically performing one or more visual perception tasks. A visual perception task, also referred to as a computer vision task, can be understood, for example, as a task for extracting visually perceptible information from image data. In particular, a visual perception task may, in principle, be performed by a human being who is capable of visually perceiving an image corresponding to the image data. However, in the present context, the visual perception task is performed automatically without the need for human assistance.

[0014] In the context of the present invention, the method and algorithm for automatic visual perception are not only based on camera images as sensor input, but also on at least one ultrasonic sensor signal or ultrasonic map, respectively. Ultrasound or pulses are invisible to humans. Nevertheless, the algorithm used is indicated as an algorithm for automatic visual perception, and therefore the method is indicated as a method for automatic visual perception.

[0015] Since the involved sensor inputs, in particular camera images and at least one ultrasonic sensor signal, are generated by the vehicle's sensor systems, in particular a camera and ultrasonic sensor system, the method can be considered as a method for automatic visual perception by a vehicle.

[0016] For example, the trained artificial neural network may be provided in a computer-readable manner, for example stored in a storage medium of the vehicle, in particular at least one computing unit.

[0017] The neural network comprises several modules, including a first encoder module, a top-view conversion module, and a first decoder module. The modules may be understood as software modules or parts of the neural network. The software modules may be understood as software codes functionally connected and coupled to units. The software modules may comprise or implement several processing steps and / or data structures.

[0018] A module may particularly represent a neural network or a sub-network itself. Unless otherwise specified, a module of a neural network may be understood as a trainable, particularly trained, module of the neural network. For example, the neural network, and thus all of its trainable modules, may be trained end-to-end before the method for automatic visual perception is performed. However, in other implementations, different modules may be trained or pre-trained individually. In other words, the method according to the present invention corresponds to the development phase of a neural network.

[0019] The training itself may be done using conventional methods for training neural networks, particularly convolutional neural networks, CNNs, such as supervised training approaches.

[0020] To generate an ultrasound map in response to the at least one ultrasonic sensor signal, a further software module may be applied to the at least one ultrasonic sensor signal, where the further software module is not necessarily part of the neural network. In particular, the further software module is not necessarily a trained or trainable module. However, in other implementations, the further software module may be a trainable or trained module of the neural network.

[0021] For example, the first and second encoder modules may comprise one or more convolutional layers, and the first decoder module comprises one or more deconvolutional layers.

[0022] The encoder module may comprise one or more convolutional layers that extract features of at least one first feature map from the camera image. The spatial dimensions of the at least one first feature map may generally be different from the spatial dimensions of the camera image depending on the architecture of the encoder, in particular the convolutional layers, e.g., the convolution kernels involved or used. In general, the camera image can be considered as a three-dimensional tensor of size H×W×C. H×W then denotes the spatial size of the camera image, i.e., its height H and width W in terms of the pixels of the camera image. C is the channel dimension, which may correspond, for example, to different color channels in the case of the camera image. However, it is also possible that C is equal to 1 for the camera image. Each feature map i of the at least one first feature map also has a respective spatial size H i xW i and channel number C i The exact height H of one of the first feature maps is i and width W i and channel number C idepends on the number of convolutional layers used by the encoder module and the size and number of convolutional kernels used. However, the features of the at least one feature map can still be considered to be given in the same camera image plane perspective as the camera image. In other words, the directions corresponding to the height and width of the feature map of the at least one first feature map correspond to the height and width directions in the original camera image, even though there may not be a one-to-one pixel correspondence between the features and the pixels of the camera image.

[0023] Algorithms for converting a camera image from a camera image plane perspective to a top-view perspective are known per se. The camera image plane may, for example, be perpendicular to a predetermined longitudinal direction of the camera coordinate system, for example parallel to the optical axis of the camera. The top-view perspective corresponds to a perspective with a top-view plane perpendicular to a predetermined height axis. The height axis may, for example, be perpendicular to the road surface on which the vehicle is positioned, or in other words parallel to the vehicle's height axis, which may possibly be parallel to a further coordinate axis of the camera coordinate system. In general, the camera coordinate system may be tilted or rotated.

[0024] However, according to the present invention, the camera image is not transformed from a camera image plane perspective to a top-view perspective, but rather each individual first feature map of the at least one first feature map is transformed, thereby achieving that the at least one first feature map is directly comparable and fused with the at least one second feature map derived from the ultrasound map, which also represents the environment in the top-view perspective.

[0025] The top-view transformation module can be designed, for example, as described in the Roddick and Cipolla publications for a multi-scale dense transformer or a stack of dense transformer layers, respectively.

[0026] The ultrasonic sensor system includes an ultrasonic emitter configured to emit ultrasonic pulses in an environment. The ultrasonic sensor system further includes at least one ultrasonic detector, each of the ultrasonic detectors configured to generate one of the at least one ultrasonic sensor signal. Alternatively, the ultrasonic sensor emitter and / or one or more additional ultrasonic emitters of the ultrasonic sensor system can emit one or more additional ultrasonic pulses, and the at least one ultrasonic sensor signal can also depend on reflected portions of the additional emitted pulses.

[0027] In general, the emitter and detector of an ultrasonic sensor system may be designed in combination as an ultrasonic transducer or may be implemented separately from each other. In particular, the reflected portion of an ultrasonic pulse emitted by an ultrasonic sensor may be detected by the same ultrasonic sensor, denoted as a direct signal path, or by another ultrasonic sensor of the ultrasonic sensor system, denoted as an indirect signal path.

[0028] Each ultrasonic sensor signal can be considered to represent, for example, the envelope of an ultrasonic wave corresponding to the reflected portion of the emitted pulse. Each ultrasonic sensor signal is then given by its amplitude as a function of time. Since the speed of sound is known, particularly for a given temperature in the environment, the amplitude as a function of time can be directly converted to amplitude as a function of radial distance, taking into account appropriate interpolation or averaging in the case of an indirect signal path. In principle, it is possible to use different ultrasonic detectors at different locations and evaluate at least one ultrasonic sensor signal to obtain not only radial distance information but also angular information regarding reflectors in the environment relative to the ultrasonic emitter and / or detector. In this way, a two-dimensional top-view ultrasonic map representing the environment can be generated based on at least one ultrasonic sensor signal.

[0029] However, this is possible even if angular information is not extracted from the combination of ultrasonic sensor signals. For example, in a simple approach, each angular position or each angular position within a predetermined angle of view may be treated equally, so that the contribution of each ultrasonic sensor signal to the ultrasonic map is essentially circular. However, heuristics or empirical information may also be used to improve the accuracy of the ultrasonic map. For example, the contribution of each ultrasonic sensor signal may be limited to the angle of view of the respective emitter and / or detector of the ultrasonic sensor system. Also, a radial cutoff may be used.

[0030] However, alternatively or additionally, a more accurate angular distribution can be used to take into account the angular emission and / or detection characteristics of the ultrasound emitter or each ultrasound detector, respectively. For example, the intensity of the emitted ultrasound pulse may be greatest along the longitudinal direction of the emitter, and / or the detector sensitivity of the corresponding ultrasound detector may be greatest along the longitudinal direction of the respective detector. An empirically or heuristically determined distribution, such as a beta distribution, can be used to approximate the ultrasound characteristics. In combination with radial information obtained directly from at least one time-dependent ultrasound sensor signal, the ultrasound map may be obtained as a two-dimensional spatial map, or in other words, an ultrasound image in a top-view perspective.

[0031] This has the advantage that the ultrasound map can be processed in a similar manner to camera images by a neural network, for example, by passing it through one or more convolutional layers of the second encoder module to extract the respective features. Even if the contents of the ultrasound map may not be immediately interpretable by humans, a trained neural network can automatically interpret the encoded information in the same way as camera images. By fusing features obtained from the ultrasound map with features obtained from the camera image, the reliability of the first visual perception task and its output can be significantly improved, especially in low-light scenarios and / or for objects relatively low above the ground. In particular, ultrasonic sensor signals represent reflective objects in the vehicle's environment regardless of lighting conditions. Ultrasonic sensor systems are typically mounted relatively low above the ground, for example on the rear bumper, allowing for reliable detection and characterization of low objects.

[0032] Fusing the at least one transformed first feature map and the at least one second feature map may include applying known mechanisms for fusing feature maps, such as concatenation of the respective feature maps. For fusion, in some implementations, one or more of the at least one transformed first feature map and / or the at least one second feature map may be upsampled or downsampled to achieve a desired spatial dimension for further processing. For example, the fusion module is not necessarily trainable or trained.

[0033] To perform a first visual perception task, a first decoder module may be applied to the set of fused feature maps. In other implementations, a further network module of a neural network, such as a top-down network module, may be applied to the set of fused feature maps, and then a first decoder module is applied to the corresponding output of the further network module.

[0034] In particular, the field of view of the camera may partially overlap with the field of view of the ultrasonic sensor system, in particular the joint field of view of all ultrasonic sensors of the ultrasonic sensor system involved in generating the at least one ultrasonic sensor signal. As a result, the camera image and the at least one ultrasonic sensor signal at least partially represent the same spatial region of the environment. For example, the camera may be a rear camera or a side camera of the vehicle, and the ultrasonic sensor system may be mounted correspondingly on the side and / or rear end of the vehicle.

[0035] According to some implementations of the method for automated visual perception, a set of intermediate feature maps is generated by applying a top-down network module of a neural network to the set of fused feature maps, and a first visual perception task is performed by applying a first decoder module to the set of intermediate feature maps.

[0036] For example, the top-down network architecture described in the publication by Roddick and Cipolla can be used. However, according to the present invention, the top-down network module does not fully perform the first visual perception task. Rather, the top-down network can be considered a task-independent processing block or a task-independent decoder block that processes the set of fused feature maps to provide a first, and in each implementation, intermediate, set of feature maps that can form the basis for further visual perception tasks. In other words, the top-down network represents a task-independent processing block, while the first decoder module is a task-specific decoder module specifically trained to perform the first visual perception task.

[0037] For example, the second visual perception task may be performed by a second decoder module of the neural network in response to the set of fused feature maps, and / or the third visual perception task may be performed by a third decoder module of the neural network in response to the set of fused feature maps. In particular, the first visual perception task, the second visual perception task, and the third visual perception task may be different from each other.

[0038] For example, a second visual perception task may be performed by applying a second decoder module to the set of intermediate feature maps, and / or a third visual perception task may be performed by applying a third decoder module to the set of intermediate feature maps.

[0039] As mentioned for the first decoder module, the top-down network can also include one or more deconvolutional layers, and the same is equally true for the second and / or third decoder modules in each implementation.

[0040] The second and third decoder modules are task-specific decoder modules specifically trained for the second and third visual perception tasks, respectively, as described for the first decoder module. Meanwhile, the output of the top-down network, i.e., a set of intermediate feature maps, can be used for all of the first, second, and third visual perception tasks. It is also possible to include more than three visual perception tasks.

[0041] The first visual perception task may be one of an object height regression task, a semantic segmentation task, and a bounding box detection task. Similarly, the second visual perception task may be one of an object height regression task, a semantic segmentation task, and a bounding box detection task. Similarly, the third visual perception task may be one of an object height regression task, a semantic segmentation task, and a bounding box detection task.

[0042] Preferably, the first and second visual perception tasks are different from each other. In each implementation, the third visual perception task is preferably different from the first visual perception task and different from the second visual perception task. For example, the first visual perception task is an object height regression task, the second visual perception task is a semantic segmentation task, and the third visual perception task is a bounding box detection task.

[0043] The use of two, three or even more task-specific decoder modules to perform a respective number of visual perception tasks allows the present invention to be used in a particularly efficient manner, as all of the decoder modules share the same set of fused feature maps or sets of intermediate feature maps, respectively, for performing different visual perception tasks.

[0044] The output of each decoder module for the object height regression task comprises a height map from a top-view perspective. For example, if the first visual perception task is an object height regression task, the output of the first decoder module comprises a height map from a top-view perspective. The height map from a top-view perspective includes predicted object heights of one or more objects in the environment.

[0045] For example, the height map may correspond to a grid, where each grid cell of the grid is associated with or located at a corresponding two-dimensional spatial location in a top-view perspective, and the height map comprises predicted or estimated height values ​​for each of the grid cells. For example, the grid cells may be rectangular or quadratic cells with respective side lengths on the order of one centimeter or a few centimeters.

[0046] By fusing information obtained from at least one ultrasonic sensor signal with information obtained from camera images, a height map can be calculated in a particularly reliable and accurate manner, which can be used as a valuable input for driver assistance functions or other automated or semi-automated driving functions of the vehicle.

[0047] The output of each decoder module according to the semantic segmentation task comprises a semantically segmented image in a top-view perspective, for example, if the second visual perception task is a semantic segmentation task, the output of the second decoder module comprises a semantically segmented image in a top-view perspective.

[0048] A semantically segmented image may be understood as a rectangular array of pixels, each of which is assigned an object class of a plurality of predetermined object classes. There may be a one-to-one correspondence between the pixels of the semantically segmented image and a version of the camera image projected or transformed to a top-view perspective. However, this is not necessarily the case, as the resolution or size of the semantically segmented image may also differ from the correspondingly transformed camera image.

[0049] By fusing information obtained from at least one ultrasonic sensor signal with information obtained from camera images, a semantically segmented image can be calculated in a particularly reliable and accurate manner, which can be used as valuable input for driver assistance functions or other automated or semi-automated driving functions of the vehicle.

[0050] The output of each decoder module for the bounding box detection task comprises a respective position and size of at least one bounding box in a top-view perspective of at least one object in the environment. For example, if the third visual perception task corresponds to a bounding box detection task, the output of the third decoder module comprises a respective position and size of at least one bounding box.

[0051] The bounding box may be, for example, a rectangular bounding box. However, other geometric shapes may also be used. For example, in the case of a rectangular bounding box, the position may be given by the center position of the rectangle, or the corner position of the rectangle, or another defined position of the rectangle. In this case, the size of the bounding box may be given by the width and / or height of the rectangle, or by equivalent quantities.

[0052] By fusing information obtained from the at least one ultrasonic sensor signal with information obtained from the camera images, the position and size of the at least one bounding box can be calculated in a particularly reliable and accurate manner, which can be used as valuable input for driver assistance functions or other automated or semi-automated driving functions of the vehicle.

[0053] According to some implementations, the first encoder module includes at least two encoder branches, and by applying the first encoder module to the camera image, each of the at least two encoder branches generates a respective first feature map of the at least one first feature map, the size of which is reduced relative to the size of the camera image according to a predetermined scale factor of the respective encoder branch.

[0054] Here, the size of the camera image and each first feature map corresponds to a spatial size according to the camera image plane viewpoint, as described above. In particular, the scale factors of the at least two encoder branches can be realized by using convolution kernels adapted accordingly.

[0055] For example, the first encoder module includes a feature extraction block and a feature extension block following the feature extraction block. Each of the at least two encoder branches includes a corresponding portion of the feature extraction block and an associated portion of the feature extension block. The feature extraction block extracts image features according to different scale factors, while the feature extension block can extend spatial features by spatial contacts from other scales, e.g., lower scales.

[0056] The first encoder module can be designed, for example, as described in the publication by Roddick and Cipolla, where, using the terminology of the publication, the feature extraction block corresponds to a backbone network and the feature expansion block corresponds to a feature pyramid or a feature pyramid network.

[0057] The use of a first encoder module having at least two encoder branches as described above is particularly beneficial in combination with a transformation of at least one first feature map to a top-view perspective.

[0058] According to some implementations, the one or more first feature maps comprise at least two first feature maps, the sizes of which are reduced relative to the size of the camera image according to different predetermined scale factors.

[0059] According to some implementations, fusing the transformed at least one first feature map and the at least one second feature map includes concatenating the transformed at least one first feature map and the at least one second feature map.

[0060] In this way, particularly efficient fusion can be achieved without requiring substantial computation time.

[0061] According to some implementations, the top view transformation module comprises a transformer pyramid network.

[0062] In particular, the top-view transformation module includes a respective transformer layer, in particular a dense transformer layer, for each of the at least two encoder branches, such that each of the at least one first feature map can be transformed to a top-view perspective according to its spatial dimension or scaling.

[0063] According to some implementations, for each of the at least one ultrasonic sensor signal, the amplitude of the respective ultrasonic sensor signal as a function of time is converted to an amplitude as a function of radial distance from the ultrasonic sensor system. For each of the at least one ultrasonic sensor signal, a distribution amplitude is calculated as a product of the amplitude as a function of radial distance and the respective predetermined angular distribution. Generating the ultrasonic map may include or consist of summing the distribution amplitudes, particularly for all of the at least one ultrasonic sensor signal.

[0064] The angular distribution describes, for example, how the amplitude as a function of radial distance differs for different angles, in particular polar angles in a top view perspective. Apart from the angular distribution, a radial cutoff or a maximum radial distance or a further radial cutoff, i.e. a minimum radial distance, can also be defined, with the values ​​of the ultrasound map being, for example, zero for radial distances below the minimum radial distance or above the maximum radial distance.

[0065] The respective angular distributions can be defined using empirical, experimental, or heuristic knowledge. In this way, two-dimensional information can be obtained from at least one ultrasonic sensor signal in a particularly simple manner. In general, the angular distributions may be different for different ultrasonic sensor signals. However, the angular distributions may be identical up to position shifts and / or rotations that reflect different positions of the ultrasonic emitter and / or detector.

[0066] The angular distribution can be considered as a representation of the emission and detection characteristics of the ultrasonic sensor system. In other words, the angular distribution can be considered as describing how relatively large the amplitude of each ultrasonic sensor signal will be when a given reflecting object in the vehicle's environment is positioned at different angles. The amplitude may eventually fall below the noise level at very large angles, which is taken into account by using the corresponding angular distribution.

[0067] For example, the angular distribution may be given by at least one beta distribution. The angular distribution may be given by a beta distribution, or by a weighted sum, average, or minimum of two or more beta distributions. In particular, if the ultrasonic emitter generating the ultrasonic sensor signal and the respective ultrasonic detector are not located at the same position, the angular distribution may be considered for both positions, for example, a beta distribution may be considered for both positions, and the actual angular distribution may be given by the minimum, average, etc. of the individual angular distributions of the different positions.

[0068] The beta distribution may be given, for example, by:

[0069] f p,q (x)~x p-1 (1-x) q-1 , Specifically, p=q=2, so that f 2,2 (x)~x(1-x) with an appropriate normalization factor, where x denotes a quantity that depends on, e.g., is proportional to, the polar angle in the top view perspective, in particular with respect to the longitudinal axis of the corresponding ultrasonic sensor of the ultrasonic sensor system. In this way, it has been found that the actual characteristics of the ultrasonic sensor system can be well modeled.

[0070] According to a further aspect of the present invention, there is provided a method for at least partially automatically guiding a vehicle, in particular an automobile, comprising performing a method for automated visual perception according to the present invention, the method further comprising generating at least one control signal for guiding the vehicle at least partially automatically depending on the outcome of a first visual perception task.

[0071] At least one control signal may be provided to one or more actuators of the vehicle that, for example, automatically or partially automatically influence or effect lateral power longitudinal control of the vehicle.

[0072] In the event of use cases or usage situations that may arise in the present method and that are not explicitly described herein, the present method may provide for error messages and / or prompts for user feedback to be output and / or for default settings and / or predetermined initial states to be set.

[0073] According to a further aspect of the present invention, there is provided an electronic vehicle guidance system for a vehicle. The electronic vehicle guidance system includes a camera for the vehicle, a storage device for the vehicle that stores a trained artificial neural network, at least one computing unit for the vehicle, and an ultrasonic sensor system for the vehicle. The camera is configured to generate a camera image representative of the vehicle's environment, particularly when mounted on the vehicle accordingly. The at least one computing unit is configured to generate at least one first feature map by applying a first encoder module of the neural network to the camera image. The at least one computing unit is configured to convert the at least one first feature map from a camera image plane perspective to a top-view perspective by applying a top-view conversion module of the neural network to the at least one first feature map.

[0074] The ultrasonic sensor system, particularly when mounted on a vehicle according to the present invention, is configured to emit ultrasonic pulses and generate at least one ultrasonic sensor signal in response to a reflected portion of the emitted ultrasonic pulses. The at least one computing unit is configured to generate a spatial ultrasonic map in a top-view perspective in response to the at least one ultrasonic sensor signal and generate at least one second feature map by applying a second encoder module of a neural network to the ultrasonic map. The at least one computing unit is configured to generate a set of fused feature maps by fusing the transformed at least one first feature map and the at least one second feature map, and perform a first visual perception task in response to the set of fused feature maps by using a first decoder module of the neural network. The at least one computing unit is configured to generate at least one control signal for guiding the vehicle in response, at least in part, to a result of the first visual perception task.

[0075] An electronic vehicle guidance system may be understood as an electronic system configured to guide a vehicle in a fully automated or fully autonomous manner, particularly without manual intervention or control by the vehicle's driver or user. The vehicle performs all necessary functions, such as steering, deceleration, and / or acceleration maneuvers, as well as automatically monitoring and recording road traffic and corresponding reactions. In particular, an electronic vehicle guidance system may implement a fully automated or fully autonomous driving mode in accordance with Level 5 of the SAE J3016 classification. An electronic vehicle guidance system may also be implemented as an advanced driver assistance system (ADAS) that assists the driver for partially automated or partially autonomous driving. In particular, an electronic vehicle guidance system may implement a partially automated or partially autonomous driving mode in accordance with Levels 1 to 4 of the SAE J3016 classification. Herein and hereinafter, SAE J3016 refers to the respective standard dated June 2018.

[0076] Thus, at least partially automatically guiding the vehicle may include guiding the vehicle in accordance with a fully automatic or fully autonomous driving mode according to Level 5 of the SAE J3016 classification. At least partially automatically guiding the vehicle may also include guiding the vehicle in accordance with a partially automatic or partially autonomous driving mode according to Levels 1-4 of the SAE J3016 classification.

[0077] A computing unit may in particular be understood as a data processing device comprising processing circuits. A computing unit may therefore in particular process data and perform computational operations. This may also include operations for performing indexed access to data structures, e.g. look-up tables, LUTs.

[0078] In particular, a computing unit may include one or more computers, one or more microcontrollers, and / or one or more integrated circuits, such as one or more application specific integrated circuits, ASICs, one or more field programmable gate arrays, FPGAs, and / or one or more systems on a chip, SoCs. A computing unit may also include one or more processors, such as one or more microprocessors, one or more central processing units, CPUs, one or more graphics processing units, GPUs, and / or one or more signal processors, in particular one or more digital signal processors, DSPs. A computing unit may also include a physical or virtual cluster of computers or other of the above units.

[0079] In various embodiments, a computing unit includes one or more hardware and / or software interfaces and / or one or more memory units.

[0080] The memory unit may be implemented as a volatile data memory, such as a dynamic random access memory, DRAM, or static random access memory, SRAM, or as a non-volatile data memory, such as a read only memory, ROM, programmable read only memory, PROM, erasable programmable read only memory, EPROM, electrically erasable programmable read only memory, EEPROM, flash memory or flash EEPROM, ferroelectric random access memory, FRAM, magnetoresistive random access memory, MRAM, or phase change random access memory, PCRAM.

[0081] In this disclosure, when it is referred to that a component of an electronic vehicle guidance system according to the present invention, in particular at least one computing unit of the electronic vehicle guidance system, is adapted, configured, designed, etc. to perform or realize a particular function, achieve a particular effect, or serve a particular purpose, this can be understood as going beyond the component being usable or suitable for this function, effect, or purpose, to be specifically and actually capable of performing or realizing the function, achieving the effect, or serving the purpose, by corresponding adaptation, programming, physical design, etc.

[0082] Further implementations of the electronic vehicle guidance system according to the invention follow directly from the various embodiments of the method according to the invention, and vice versa. In particular, individual features and corresponding descriptions and advantages relating to the various implementations of the method according to the invention can be transferred in the same way to the corresponding implementations of the electronic vehicle guidance system according to the invention. In particular, the electronic vehicle guidance system according to the invention is designed or programmed to carry out the method according to the invention. In particular, the electronic vehicle guidance system according to the invention carries out the method according to the invention.

[0083] According to a further aspect of the invention there is provided a vehicle, in particular a motor vehicle, comprising an electronic vehicle guidance system according to the invention, wherein a camera and an ultrasonic sensor system are mounted on the vehicle.

[0084] According to a further aspect of the present invention, there is provided a computer program product comprising instructions which, when executed by an electronic vehicle guidance system according to the present invention, in particular by at least one computing unit of the electronic vehicle guidance system, cause the electronic vehicle guidance system to perform the method for automated visual perception according to the present invention or the method for at least partially automatically guiding a vehicle according to the present invention.

[0085] According to a further aspect of the present invention there is provided a computer readable storage medium storing a computer program according to the present invention.

[0086] The computer program and the computer-readable storage medium may be referred to as respective computer program products comprising instructions.

[0087] Further features of the present invention are apparent from the claims, the drawings, and the description of the drawings. Features and combinations of features mentioned above in the description, as well as features and combinations of features mentioned below in the description of the drawings and / or shown in the drawings, may be included in the present invention not only in the respective combinations described, but also in other combinations. In particular, embodiments and combinations of features that do not have all the features of the originally formulated claims may also be included in the present invention. Furthermore, embodiments and combinations of features that go beyond or deviate from the combinations of features described in the enumeration of the claims may be included in the present invention.

[0088] The present invention will now be described in detail with reference to certain exemplary implementations and respective schematic drawings. In the drawings, identical or functionally identical elements may be designated by the same reference numerals. Descriptions of identical or functionally identical elements are not necessarily repeated with respect to different figures. [Brief explanation of the drawings]

[0089] [Figure 1] 1 is a diagram illustrating a schematic diagram of a motor vehicle according to an exemplary implementation of the present invention. [Figure 2] 1 shows a block diagram that schematically illustrates an exemplary implementation of a method for automated visual perception according to the present invention; [Figure 3] FIG. 2 is a block diagram of a first encoder module of a neural network for use in a further exemplary implementation of a method for automated visual perception according to the present invention; [Figure 4] FIG. 10 shows a schematic diagram of the transformation of a feature map from the camera image plane to a top-view perspective. [Figure 5] 3A and 3B show schematic diagrams of parts of a method according to a further exemplary implementation of the invention for automated visual perception; DETAILED DESCRIPTION OF THE INVENTION

[0090] 1 shows an example of a vehicle 1 according to the invention. The vehicle 1 is equipped with an exemplary implementation of an electronic vehicle guidance system 2 according to the invention.

[0091] The electronic vehicle guidance system 2 includes a camera 4, e.g., a rearview camera, mounted on the vehicle 1. The electronic vehicle guidance system 2 further includes an ultrasonic sensor system including one or more ultrasonic sensors 5a, 5b, 5c, 5d, 5e. Each ultrasonic sensor 5a, 5b, 5c, 5d, 5e can operate as both an ultrasonic emitter and an ultrasonic detector. However, different implementations are also possible. The ultrasonic sensors 5a, 5b, 5c, 5d, 5e are mounted, for example, at the rear end of the vehicle 1, e.g., on or within the rear bumper of the vehicle 1. In particular, the full field of view of the ultrasonic sensors 5a, 5b, 5c, 5d, 5e at least partially overlaps with the field of view of the camera 4.

[0092] The electronic vehicle guidance system 2 further comprises a storage device (not shown) that stores a trained artificial neural network 6 (see, e.g., FIG. 2). The electronic vehicle guidance system 2 also comprises a computing unit 3 that is capable of applying the artificial neural network 6 to input data, in particular camera images 7 (see, e.g., FIG. 2) and ultrasound maps 9 (see, e.g., FIGS. 2 and 5), in order to perform one or more predetermined visual perception tasks. The computing unit 3 can then generate at least one control signal for one or more actuators (not shown) of the vehicle 1 in response to respective results or outputs of the one or more visual perception tasks to at least partially automatically guide the vehicle 1.

[0093] In particular, electronic vehicle guidance systems can perform exemplary implementations of the method for automated visual perception according to the present invention.

[0094] FIG. 2 shows a schematic block diagram illustrating such a method as well as a trained artificial neural network 6.

[0095] According to the method, the camera 4 generates a camera image 7 representing the environment of the vehicle 1, and the ultrasonic sensor system, e.g., ultrasonic sensors 5a, 5b, 5c, 5d, 5e, emits one or more ultrasonic pulses into the environment, detects reflected portions of the emitted ultrasonic pulses, and generates at least one ultrasonic sensor signal 8 in response to the detected reflected portions.

[0096] The computing unit 3 generates a spatial ultrasound map 9 in a top-view perspective in response to the at least one ultrasound sensor signal 8, the ultrasound map 9 comprising a two-dimensional amplitude distribution in the top-view perspective. To this end, the computing unit 3 can apply a software module 10, which can also be stored in a storage device, to the at least one ultrasound sensor signal 8. For further details, see the discussion regarding FIG. 5 further below.

[0097] The computing unit 3 applies a neural network 6 to input data including the ultrasound map 9 and the camera image 7. In particular, at least one first feature map 17 is generated by applying a first encoder module 11 of the neural network 6 to the camera image 7. A top view transformation module 13 of the neural network 6 is applied to the at least one first feature map 17 to transform the at least one first feature map 17 from a camera image plane perspective to a top view perspective. For further details, see the discussion of Figures 3 and 4 below.

[0098] At least one second feature map 22 is generated by applying the second encoder module 12 of the neural network 6 to the ultrasound map 9. A set of fused feature maps is generated by fusing the transformed at least one first feature map 20 (see, for example, FIG. 4 ) with the at least one second feature map 22, in particular by concatenating the transformed at least one first feature map 20 with the at least one second feature map 22. One or more visual perception tasks, three in this example of FIG. 2 , are performed in response to the set of fused feature maps by one or more respective decoder modules 15 a, 15 b, 15 c of the neural network 6, in this example of FIG. 2 , the first decoder module 15 a, the second decoder module 15 b, and the third decoder module 15 c.

[0099] For example, a set of intermediate feature maps 23 can be generated by applying the top-down network module 14 of the neural network 6 to the set of fused feature maps. Visual perception tasks can then be performed by applying respective decoder modules 15 a, 15 b, 15 c to the set of intermediate feature maps 23. In particular, a first visual perception task, e.g., an object height regression task, can be performed by applying the first decoder module 15 a to the set of intermediate feature maps 23, a second visual perception task, e.g., a semantic segmentation task, can be performed by applying the second decoder module 15 b to the set of intermediate feature maps 23, and a third visual perception task, e.g., a bounding box detection task, can be performed by applying the third decoder module 15 c to the set of intermediate feature maps 23.

[0100] Generally, the encoder module of a neural network can contain a series of convolutional layers for deep feature extraction. Roughly speaking, the deeper the encoder, the better or more detailed the features. However, the type of encoder can be chosen based on the constraints of the embedded platform. For example, standard encoder families such as ResNet, VGG, and Inception can be used.

[0101] The ultrasonic map 9 may be, for example, a single-channel top-view map of the vehicle 1's immediate surroundings. For example, the ultrasonic map may be provided on a grid with a size in meters, e.g., 6 m x 12 m, and a cell side length in centimeters, e.g., 1 cm. In this way, the maximum detection range of the ultrasonic sensors 5a, 5b, 5c, 5d, 5e, e.g., approximately 5 m, may be covered, taking into account their position relative to the camera 4, which defines the center of the coordinate system via projection onto the ground. Therefore, the ultrasonic map 9 may be quite large and relatively sparse, with only small regions having high amplitudes. Therefore, the second encoder module 12 is used to input the ultrasonic map 9 into a feature space, where it can be matched with at least one transformed first feature map 20 obtained from the camera image 7.

[0102] In particular, first, the ultrasound map 9 is converted into at least one second feature map by passing through a convolutional layer of the second encoder module 12, so that the second feature map outputs features with the same dimensions, i.e., the same height and width, as the output of the top-view conversion module 13. The feature maps are then stacked along the depth or channel direction and passed through a convolutional layer of the top-down network module 14. In particular, the second encoder module 12 converts the sparse ultrasound information contained in the ultrasound map 9 into dense information in a top-view perspective, i.e., at least a second feature map, which is then processed, for example, by the top-down network module 14.

[0103] As shown in more detail in the exemplary implementation of FIG. 3 , the first encoder module 11 may include a series of convolutional layers 16 a, 16 b, 16 c, 16 d, and 16 e with different output scales and may include regularization techniques. The first encoder module may be designed as or include a feature pyramid network. The first encoder module 11 may receive the camera image 7 as input and generate first feature maps 17 a, 17 b, 17 c, 17 d, and 17 e at different scales. For example, in FIG. 3 , the first encoder module 11 generates five first feature maps 17 a, 17 b, 17 c, 17 d, and 17 e with scale factors of 1 / 8, 1 / 16, 1 / 32, 1 / 64, and 1 / 128, respectively.

[0104] The top-view transformation module 13 can be implemented, for example, as a dense transformer pyramid, as shown schematically in Figure 4. The top-view transformation module transforms each first feature map 17 into a top-view perspective, or in other words, into a bird's-eye view feature map. The input and output coordinate systems may be completely different from each other.

[0105] As shown in FIG. 4, a transformation layer converts a first feature map 17 of dimensions H×W with C channels into a transformed first feature map 20 in the top-view plane with dimensions Z×X and C channels. The transformation layer first creates a relationship between the total horizontal length of the camera image 7 and a specific vertical length, and the top-view map. Next, in an intermediate stage 18, the vertical dimension H and channel dimension C of the first feature map 20 are reduced to a bottleneck of size B, while the vertical dimension W remains unchanged. Next, in a further intermediate stage 19, a 1D convolution is applied along the horizontal axis to reshape the resulting feature map to dimensions Z×X with C channels. This process creates a trapezoidal shape in the orthographic top-view space due to perspective differences. This is then transformed to a Cartesian coordinate system by resampling using the known camera focal length and known horizontal offset of camera 4.

[0106] The top-down network module 14 processes the transformed at least one first feature map and at least one second feature map, the output of which is processed by multiple decoder modules 15a, 15b, 15c for their respective purposes, e.g., height regression, semantic segmentation, and bounding box generation, respectively.

[0107] The at least one ultrasonic sensor signal 8 is obtained from a time series measurement and can therefore represent ultrasonic echo amplitudes recorded over a fixed period of time.

[0108] Typically, a peak in at least one ultrasonic sensor signal 8 originates from an object in the environment of the vehicle 1 that reflects the ultrasonic pulse emitted from one ultrasonic sensor 5a, 5b, 5c, 5d, 5e back to another or the same ultrasonic sensor 5a, 5b, 5c, 5d, 5e. As a result, the distance traveled by the ultrasonic pulse before being reflected can be calculated and, for example, the exact speed of sound can be determined taking into account the ambient temperature.

[0109] To convert the 1D amplitude data as a function of time to the spatial domain, the distance of the echo can be calculated, and additionally, ignorance of the angular position of the object reflecting the echo can be taken into account. The object can be located on the longitudinal sensor axis of the ultrasonic sensors 5a, 5b, 5c, 5d, and 5e, but due to their large field of view, it can also be off the longitudinal sensor axis by a large angle. Up to 70° can be reasonable, as long as the object is positioned so that the echo amplitude falls and reflects back to the sensor before it becomes indistinguishable from random noise.

[0110] 5, the software module 10 is capable of calculating an ultrasonic map 9 representing the environment of the vehicle 1 in a top-view perspective, in particular in a vehicle coordinate system, with the center of the rear axle of the vehicle 1 at the origin of the coordinate system. The position and orientation of each of the ultrasonic sensors 5a, 5b, 5c, 5d, 5e are predetermined and known.

[0111] A grid is then generated that has a grid cell size that is approximately the size of the field of view of the ultrasonic sensors 5a, 5b, 5c, 5d, 5e and is small enough to provide a high enough resolution that it can still match the distance resolution of the at least one ultrasonic sensor signal 8. For example, quadratic grid cells with a side length of 1 cm can be used. The distance of each grid cell to the emitting ultrasonic sensor 5a, 5b, 5c, 5d, 5e and the receiving ultrasonic sensor 5a, 5b, 5c, 5d, 5e can be calculated for each of the at least one ultrasonic sensor signals 8 and each time step.

[0112] An angular distribution 21, representing the likelihood of an object based on its angle from the longitudinal sensor axis, is multiplied with each amplitude as a function of radial distance. The angular distribution 21 can be thought of as an amplitude scale factor. For example, a beta distribution can be used, as shown schematically in FIG. 5. For non-identical emitting and receiving ultrasonic sensors 5a, 5b, 5c, 5d, and 5e, also referred to as indirect signal paths, the minimum of the two calculated scale factors can be used. However, it is generally conceivable to use a multiplication of both factors or the square root of both factors. The result is an intermediate ultrasonic map 9' for each of at least one ultrasonic sensor signal 8, which can be summed to obtain the ultrasonic map 9.

[0113] As described, particularly with respect to the figures, in some implementations, the present invention enables CNN-based low-level object detection, semantic segmentation, and height regression in top-view regions using camera images, particularly fisheye camera images, and ultrasound data.

Claims

1. A method for automatic visual perception by a vehicle (1), comprising: a camera image (7) representing an environment of the vehicle (1) is generated by a camera (4) of the vehicle (1), and at least one first feature map (17, 17a, 17b, 17c, 17d, 17e) is generated by applying a first encoder module (11) of a trained artificial neural network (6) to the camera image (7); a top-view transformation module (13) of the neural network (6) is applied to the at least one first feature map (17, 17a, 17b, 17c, 17d, 17e) to transform the at least one first feature map (17, 17a, 17b, 17c, 17d, 17e) from a camera image plane perspective to a top-view perspective; an ultrasonic pulse is emitted into the environment by an ultrasonic sensor system (5a, 5b, 5c, 5d, 5e) of the vehicle (1), and at least one ultrasonic sensor signal (8) is generated by the ultrasonic sensor system (5a, 5b, 5c, 5d, 5e) in response to a reflected portion of the emitted ultrasonic pulse; a spatial ultrasound map (9) is generated in the top view perspective in response to the at least one ultrasound sensor signal (8); At least one second feature map is generated by applying a second encoder module (12) of the neural network (6) to the ultrasound map (9); a set of fused feature maps is generated by fusing the transformed at least one first feature map (20) with the at least one second feature map (22); A method wherein a first visual perception task is performed by a first decoder module (15a, 15b, 15c) of the neural network (6) in response to the set of fused feature maps.

2. A set of intermediate feature maps is generated by applying a top-down network module (14) of the neural network (6) to the set of fused feature maps; 2. The method of claim 1, wherein the first visual perception task is performed by applying the first decoder module (15a, 15b, 15c) to the set of intermediate feature maps.

3. 3. The method of claim 2, wherein a second visual perception task is performed by a second decoder module (15a, 15b, 15c) of the neural network (6) in response to the set of fused feature maps.

4. 4. The method of claim 3, wherein the second visual perception task is performed by applying the second decoder module (15a, 15b, 15c) to the set of intermediate feature maps.

5. the first visual perception task is an object height regression task, and the output of the first decoder module (15a, 15b, 15c) comprises a height map in the top-view perspective containing predicted object heights of one or more objects in the environment; and / or the second visual perception task is a semantic segmentation task, and the output of the second decoder module (15a, 15b, 15c) comprises a semantic segmentation image in the top view perspective; and / or 5. The method according to claim 3, wherein a third visual perception task is performed by a third decoder module (15a, 15b, 15c) of the neural network (6) in response to the set of fused feature maps, the third visual perception task being a bounding box detection task, and the output of the third decoder module (15a, 15b, 15c) comprises respective positions and sizes of at least one bounding box in the top-view perspective for one or more objects in the environment.

6. the first visual perception task is an object height regression task, and the output of the first decoder module (15a, 15b, 15c) comprises a height map at the top view perspective containing predicted object heights of one or more objects in the environment; or the first visual perception task is a semantic segmentation task, and the output of the first decoder module (15a, 15b, 15c) comprises a semantic segmentation image in the top view perspective; or 5. The method according to claim 1, wherein the first visual perception task is a bounding box detection task and the output of the first decoder module (15a, 15b, 15c) comprises a respective position and size of at least one bounding box in the top-view perspective for at least one object in the environment.

7. the first encoder module (11) comprises at least two encoder branches (16a, 16b, 16c, 16d, 16e); 7. The method according to claim 1, wherein by applying the first encoder module (11) to the camera image (7), each of the at least two encoder branches (16a, 16b, 16c, 16d, 16e) generates a respective first feature map of the at least one first feature map (17, 17a, 17b, 17c, 17d, 17e), the size of which is reduced relative to the size of the camera image (7) according to a predetermined scale factor of the respective encoder branch (16a, 16b, 16c, 16d, 16e).

8. 7. The method according to any one of claims 1 to 6, characterized in that the one or more first feature maps (17, 17a, 17b, 17c, 17d, 17e) comprise at least two first feature maps (17, 17a, 17b, 17c, 17d, 17e), the size of which is reduced relative to the size of the camera image (7) according to different predetermined scale factors.

9. 9. The method according to claim 1, wherein fusing the transformed at least one first feature map (20) with the at least one second feature map comprises concatenating the transformed at least one first feature map (20) with the at least one second feature map.

10. Method according to any one of claims 1 to 9, characterized in that the top view transformation module (13) comprises a transformer pyramid network.

11. for each of the at least one ultrasonic sensor signal (8), the amplitude of the respective ultrasonic sensor signal (8) as a function of time is converted into an amplitude as a function of radial distance from the ultrasonic sensor system (5a, 5b, 5c, 5d, 5e); for each of said at least one ultrasonic sensor signal (8), a distribution amplitude is calculated as the product of said amplitude as a function of said radial distance and a respective predetermined angular distribution (21); The method according to any one of claims 1 to 10, characterized in that generating the ultrasound map (9) comprises summing the distribution amplitudes.

12. 12. The method of claim 11, wherein the angular distribution is given by at least one beta distribution.

13. An electronic vehicle guidance system (2) for a vehicle (1), comprising a camera (4), a storage device for storing a trained artificial neural network (6), at least one computing unit (3), and an ultrasonic sensor system (5a, 5b, 5c, 5d, 5e), the camera (4) is configured to generate a camera image (7) representative of the environment of the vehicle (1); the at least one computing unit (3) is configured to generate at least one first feature map (17, 17a, 17b, 17c, 17d, 17e) by applying a first encoder module (11) of the neural network (6) to the camera image (7); the at least one computing unit (3) is configured to convert the at least one first feature map (17, 17a, 17b, 17c, 17d, 17e) from a camera image plane perspective to a top view perspective by applying a top view conversion module (13) of the neural network (6) to the at least one first feature map (17, 17a, 17b, 17c, 17d, 17e); the ultrasonic sensor system (5a, 5b, 5c, 5d, 5e) is configured to emit ultrasonic pulses and generate at least one ultrasonic sensor signal (8) in response to reflected portions of the emitted ultrasonic pulses; the at least one computing unit (3) is configured to generate a spatial ultrasound map (9) in the top view perspective in response to the at least one ultrasound sensor signal (8), and to generate at least one second feature map by applying a second encoder module (12) of the neural network (6) to the ultrasound map (9); the at least one computing unit (3) is configured to generate a set of fused feature maps by fusing the transformed at least one first feature map (20) and the at least one second feature map, and to perform a first visual perception task according to the set of fused feature maps by using a first decoder module (15a, 15b, 15c) of the neural network (6); The at least one computing unit (3) is configured to at least partially automatically generate at least one control signal for guiding the vehicle (1) depending on the result of the first visual perception task.

14. A vehicle (1) equipped with an electronic vehicle guidance system (2) according to claim 13, wherein the camera (4) and the ultrasonic sensor system (5a, 5b, 5c, 5d, 5e) are mounted on the vehicle (1).

15. A computer program product comprising instructions which, when executed by an electronic vehicle guidance system (2) according to claim 13, cause said electronic vehicle guidance system (2) to perform the method according to any one of claims 1 to 12.

Images