Method for detecting at least one obstacle in automatic and / or at least semi-autonomous driving system

Through machine learning model, the occlusion tag is determined in the autonomous driving system and self-supervised training is solved, and the problem of traditional algorithms is difficult to detect obstacles in a monocular camera is realized, and reliable obstacle detection in a 3D environment is suitable for different scenarios navigation.

CN119027906APending Publication Date: 2024-11-26ROBERT BOSCH GMBH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410573417.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-05-26
Filing Date
2024-05-10
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Traditional learning-based object recognition algorithms are difficult to learn general representations of dangerous objects in 3D environments, especially in the absence of sufficient labeled data, and commonly used algorithms require stereo cameras, which cannot be effectively promoted in monocular camera systems.

Method used

Using machine learning models, the occlusion labels are determined through image data and self-supervised training is designed to design high-quality general detectors for hazardous objects, which can detect obstacles in monocular or stereo cameras, and train with optical flow estimation and loss functions. , generate occlusion tags and obstacle point clouds, and combine them with the motion planning system for autonomous control.

Benefits of technology

It realizes reliable obstacle detection in automatic and semi-autonomous driving systems, can work effectively in monocular cameras, reduces dependence on labeled data, improves the universality and accuracy of detection, and is suitable for navigation in different scenarios. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119027906A_ABST
    Figure CN119027906A_ABST
Patent Text Reader

Abstract

The invention relates to a method for detecting at least one obstacle in an automatic and / or at least semi-autonomous driving system, said method comprising the following steps:-providing (101) image data, said image data being dedicated to recording the environment of the driving system (60); -performing (102) an evaluation of the provided image data, the evaluation being performed on the basis of the application of a machine learning model (50) by means of which an occlusion label is determined for at least one occlusion of the environment,-performing (103) the detection of the at least one obstacle on the basis of the determined occlusion label.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to a method for detecting at least one obstacle in an automatic and / or at least semi-autonomous driving system. The invention also relates to a training method, a computer program, a device, a computer-readable storage medium and a machine learning model. Background Art

[0002] One of the most important challenges in perception for autonomous mobile robots or driver assistance systems is the reliable detection of dangerous objects. Therefore, the object of the present invention is to enable reliable navigation in 3D environments.

[0003] Conventional learning-based object recognition algorithms using convolutional neural networks (abbreviated as CNN) as a basis are generally unable to learn a general representation of dangerous objects without being provided with a sufficient number of human-annotated examples of all possible variations of the dangerous objects.

[0004] Assuming that it is practically impossible to manually label all possible generic objects, algorithms based on heuristics and deterministic formulas are often used to detect dangerous objects. An example of such an algorithm is given in the following paper: P. Pinggera, U. Franke, and R. Mester, "High-performance long range obstacle detection using stereo vision," 2015 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 2015, pp. 1308-1313, doi: 10.1109 / IROS.2015.7353537. Although this approach can indeed detect some unexpected obstacles, it usually does not generalize to different scenarios and also requires stereo cameras. However, many robotic systems use monocular cameras. Summary of the invention

[0005] In the context of the present invention, features and details described in conjunction with the method according to the present invention may obviously also be applied in conjunction with the training method according to the present invention, the computer program according to the present invention, the device according to the present invention, the computer-readable storage medium according to the present invention, and the machine learning model according to the present invention, and vice versa, and reference is therefore always made to or may be made to the various aspects of the present invention.

[0006] The object of the present invention is in particular a method for detecting at least one obstacle in an automatic and / or at least semi-autonomous driving system, said method comprising the following steps, which are preferably performed sequentially and / or repeatedly:

[0007] - providing image data dedicated to recording the environment surrounding the driving system,

[0008] - performing an evaluation of the provided image data, wherein the evaluation may be based on the application of a machine learning model, with the aid of which an occlusion label is determined for at least one occlusion of the environment,

[0009] - Performing a detection of the at least one obstacle, in particular a dangerous object, based on the determined occlusion signature.

[0010] The present invention may have the advantage of overcoming the limitations of conventional learning-based methods. This may be related to the lack of labeled data. According to the present invention, in this case, as an intermediate step, occlusion labels may first be determined based on the image data with the aid of machine learning instead of immediately classifying the dangerous object. The present invention also enables training of a high-quality general detector for dangerous objects, which performs detection based on the determined occlusion labels. As will be described in more detail below, self-supervised training using specific supervisory elements may be used for this purpose. The method according to the present invention may be reliably used not only in stereo cameras, but also in monocular camera systems. In other words, the image data may include individual images instead of image sequences, so that motion information may be omitted for the application of the machine learning model.

[0011] It is also advantageous that the training of the machine learning model is based on determining occlusion areas based on motion in the camera recording. For this purpose, the optical flow can be estimated in the image sequence generated by the camera recording. The machine learning model can then be trained based on the estimated optical flow to determine the occlusion label, in particular to determine the occlusion label based only on image data in the form of a single image. The training can preferably be carried out in the form of a self-supervised training process. The machine learning model can also be designed as a CNN. During training, the machine learning model can obtain an image sequence comprising two (or more) consecutive images as input from a monocular video and provide an estimate of the optical flow on this basis. Minimization of the photometric error can be used as a loss function. The occlusion area may refer to a spatial area that is blocked by obstacles in the environment.

[0012] A special feature of the training of machine learning models is that it can, among other things, exploit the fact that when the camera moves, optical flow can never detect all pixels. This aspect can be used to directly determine occlusion labels in the form of an obstacle map, or to indirectly generate occlusion labels for another type of obstacle map according to the "further loss options" described below. The occlusion labels thus determined can also be designed as an obstacle point cloud. Another special feature of the present invention may be that self-supervised training is able to rely on a large amount of unlabeled data. Furthermore, bounding boxes can optionally be created from the obstacle point cloud, and a false positive reduction classifier can be run on the detected candidate objects in the next stage.

[0013] It is also conceivable within the scope of the invention that the image data, in particular in the inference mode, comprises at least one or exactly one single image, which results from a recording using a monocular or stereo camera. In other words: after a training process as described above, exactly one single image can be used in the inference mode. Preferably, the image data used for the machine learning model as input for determining the occlusion labels are limited to a single image. In other words, the machine learning model does not require motion information as input for determining, preferably generating, the occlusion labels.

[0014] It may optionally be provided that the occlusion label is specific to at least one occlusion and / or is designed as an occlusion map, which identifies at least one or more regions in the image data that are occluded by at least one object in the environment. In this case, the occlusion map may also be designed as an occlusion mask, which indicates, for example (preferably in a binary manner), whether individual elements such as pixels of the image data are occluded by the at least one object.

[0015] Within the scope of the present invention, it can preferably be provided that the detection of (the) at least one obstacle comprises an evaluation of the occlusion label, preferably by means of a classifier and preferably by means of a classifier trained by machine learning, during which evaluation a classification of one of the objects in the form of a dangerous object associated with the respective occlusion and detected in the image data is performed with reference to the occlusion label. A dangerous object may in particular be cargo that has fallen from a truck. Furthermore, a dangerous object may also be referred to as an object that may be potentially dangerous for a mobile vehicle and / or a robot including a driving system. The classifier may be limited to determining whether an object is a dangerous object. Thus, the classification of other classes may be omitted and the classifier may be designed in a class-agnostic manner.

[0016] It can also be optionally provided that, based on the evaluation and / or detection, at least partially autonomous control of the ego vehicle and / or the robot is performed with the aid of a driving system, preferably by means of a motion planning system. The driving system can, for example, be designed as a driver assistance system or an autonomous driving system for use, for example, in an autonomous mobile robot or an autonomous vehicle. In this case, the perception system can provide a representation of the 3D environment, and this representation can be used as input to the motion planning system, which then decides how the ego vehicle should be maneuvered.

[0017] The present invention also provides a training method for training a machine learning model, the method comprising:

[0018] - providing training data, wherein the training data comprises at least one sequence of images representing an environment of the driving system during a trip, and the training data may also comprise annotation data, in particular in the form of ground truth, indicating an occlusion label of at least one occlusion representing the environment during a trip,

[0019] -The machine learning model is trained on the training data, during which the optical flow in the image sequence is considered to predict the occlusion labels.

[0020] Therefore, the training method according to the invention provides the same advantages as those described in detail with respect to the method according to the invention for detecting at least one obstacle. In this case, the machine learning model applied to the method according to the invention for detecting at least one obstacle is preferably generated by the training method according to the invention. The object of the invention is also the machine learning model obtained by the training method according to the invention.

[0021] With respect to the training method, a fundamental matrix may also be calculated from the estimated optical flow and an occlusion label in the form of an occlusion map indicating at least one occlusion, preferably indicating a calibration matrix of the camera. As a further step, 3D point triangulation and / or depth estimation may be performed. With reference to the relative transformation between two images in the image sequence, triangulation may be applied in order to obtain a 3D point for each point correspondence from the optical flow.

[0022] The object of the present invention is also a computer program, in particular a computer program product comprising instructions, which, when executed by a computer, prompt the latter to perform the method according to the invention. The computer program according to the invention therefore provides the same advantages as those described in detail with respect to the method according to the invention.

[0023] The invention also relates to a device for data processing, which is configured to perform the method according to the invention. For example, a computer may be provided as a device for executing a computer program according to the invention. The computer may comprise at least one processor for executing the computer program. A non-volatile data storage device may also be provided, in which the computer program is stored and from which the processor can read the computer program for execution.

[0024] The object of the invention may also be a computer-readable storage medium comprising a computer program according to the invention and / or comprising instructions which, when executed by a computer, prompt the latter to perform the method according to the invention. The storage medium is designed, for example, as a data storage device such as a hard disk and / or a non-volatile memory and / or a memory card. The storage medium may, for example, be integrated into a computer.

[0025] The method according to the invention can also be designed as a computer-implemented method. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Other advantages, features and details of the invention are apparent from the following description, in which embodiments of the invention are described in detail with reference to the accompanying drawings. In this context, each of the features mentioned in the claims and the description is essential to the invention, both by themselves and in any combination. In the drawings:

[0027] Figure 1 is a schematic diagram of a method, device, storage medium, machine learning model, training method, and computer program according to an exemplary embodiment of the present invention.

[0028] Figure 2 is another schematic diagram for illustrating a training method according to an exemplary embodiment of the present invention.

[0029] Figure 3 It is an illustration of using a moving monocular camera to determine occlusion labels.

[0030] Figure 4 Another illustration of using a moving monocular camera to determine occlusion labels. DETAILED DESCRIPTION

[0031] Figure 1Schematically shown in the figure are a method 100, a device 10, a storage medium 15 and a computer program 20 according to an exemplary embodiment of the present invention. In this case, it is shown that in the method 100 for detecting at least one obstacle in an automatic and / or at least semi-autonomous driving system 60, image data can first be provided according to a first method step 101. The image data can be dedicated to recording the environment of the driving system 60. According to a second method step 102, an evaluation of the image data can then be provided, wherein the evaluation is carried out based on the application of a machine learning model 50, with the help of which an occlusion label is determined for at least one occlusion of the environment. In a third method step 103, the at least one obstacle can then be detected based on the determined occlusion label. In addition, based on the evaluation and / or detection, at least partially autonomous control of the ego vehicle 5 and / or the robot 5 can be performed by the driving system 60, preferably by a motion planning system.

[0032] Figures 2 to 4 It is further shown that the training of the machine learning model 50 can be based on occlusion areas 304, 401, which are determined based on the movement 303 in the camera recording, wherein for this purpose the optical flow is preferably estimated in the image sequence obtained from the camera recording, and the machine learning model 50 is trained with reference to the estimated optical flow (preferably called the total loss L_o) to determine the occlusion label. The training is preferably performed in the form of a self-supervised training process. The occlusion area can also be determined by calculating the best possible normal describing the pixel (represented by L_repr). The latter occlusion label can also be obtained from a stereo image. In this case, the occlusion label can be specific to at least one occlusion and is preferably designed as an occlusion map, which identifies at least one or more areas in the image data that are occluded by at least one object 70 in the occlusion area 304, 401 in the environment.

[0033] Figure 1 Also shown is a training method 200 for training a machine learning model 50, in which training data is provided according to a first training step 201, and training of the machine learning model 50 is provided according to a second training step 202, so as to predict occlusion labels. In this way, a trained machine learning model 50 can be obtained.

[0034] Embodiments of the present invention may have the advantage of providing a trainable algorithm without requiring a large amount of labeled data. While it is relatively easy to collect a large amount of unlabeled data, it is very expensive or impractical to process and label this data for supervised algorithms (such as CNN-based object recognition) in the case of an unknown number of objects (such as dangerous objects). According to an exemplary embodiment of the present invention, an algorithm may be provided in this case that is also applicable to a monocular camera setting. Compared to deterministic algorithms, learning-based algorithms according to exemplary embodiments of the present invention may be adaptive. In other words, they can be trained by adding data to solve problematic situations. However, for non-learning-based methods, such adaptation and training to difficult situations that were unknown before the algorithm was first released and applied is not possible. Embodiments of the present invention are not only applicable to stereo cameras, but also to monocular cameras.

[0035] Exemplary embodiments may enable detection of hazardous objects to enable navigation of autonomous systems. HD maps may also be created.

[0036] In an advanced driver assistance system or autonomous driving system, the perception system provides a representation of the 3D environment, and the representation is used as an input to the motion planning system, which then decides how the ego vehicle should be manipulated. Key aspects of perception system technology include identifying where the vehicle can travel and what the environment around the car looks like. It is known that traditional computer vision techniques are generally not robust enough because they cannot learn like machine learning techniques. In contrast, learning-based methods provide excellent results, but require a large amount of labels, i.e., manually annotated data. The exemplary embodiment of the present invention adopts a high-quality learning-based method, and the labeling problem can be solved by self-supervised pre-training, as a result of which the number of required data annotations is significantly reduced. In this case, self-supervised training can be based on training methods related to machine learning or artificial intelligence, whereby the model learns from unlabeled data by comparing its own predictions with actual results, and learns from this process without relying on manually annotated data. However, in semi-supervised training, the model is trained using labeled data and unlabeled data to improve performance and generalization ability.

[0037] like Figure 2 The illustrated semi-supervised general algorithm for obstacle detection can be designed as follows. First, self-supervised training 201 of CNN 202 can be performed. In this case, the training can include at least one of the following steps:

[0038] -Self-supervised CNN training process for optical flow,

[0039] - compute an occlusion mask based on the computed optical flow,

[0040] - estimate the fundamental matrix,

[0041] -3D point triangulation and depth estimation, and

[0042] -Training a single image occlusion CNN.

[0043] In this case, the fundamental matrix can be a matrix calculated based on the pixel correspondence between the two camera images. This matrix describes the relationship between the camera positions in this way and enables the reconstruction of the position of the object in three-dimensional space. The calculation of the fundamental matrix is ​​performed, for example, by using an algorithm such as RANSAC (Random Sample Consensus) or by introducing constraints (e.g., epipolar geometry).

[0044] Figure 2 Shown are, for example, a camera image 203 (particularly an RGB color model) and an occlusion mask 204. A supervised tuning 210 of an algorithm 220 based on machine learning or deep learning techniques or DBScan-based clustering 221 may then be performed. A reduction in false positive results 222 may then be provided by a supervised classifier processing each candidate obstacle.

[0045] Self-supervised pre-training of the CNN can be provided during the first stage. This stage can overcome the problem of lack of labeled data for all possible dangerous objects. The operating principle will be explained below. Any raised object may cause occlusion (see Figure 3 ). Therefore, CNN 202 can be trained to detect occlusions in a self-supervised manner (see Figure 2 ), because the occlusion has high-quality features and the area of ​​interest, where the dangerous object may be located. For a continuous image sequence from a monocular camera 40 or a stereo camera 40, a CNN can be trained by minimizing the photometric loss between images to obtain the optical flow and maximizing the unocclusion loss to obtain the occlusion detector. The input of this first stage is, for example, a video sequence from a monocular camera 40 and a calibration matrix K. Figure 3 301, 302 are shown in a side view of the vehicle 60, which change with a movement 303 of the driving system 60. Also shown is an obstacle 70. Due to the movement 303, an upper area 304 blocked by the obstacle 70 can be determined. Figure 4 A top view of the obstacle 70 is shown, and two camera positions 301, 302 are also shown. Furthermore, it is shown that due to the movement 303 of the driving system 60, obstructed side areas 401 can be determined.

[0046] Exemplary embodiments of the present invention may provide self-supervised training of optical flow. In other words, the training of a self-supervised optical flow CNN may be performed in a first step. To this end, the CNN may obtain two (or more) consecutive images from a monocular video as input and provide an estimate of the optical flow based thereon. Minimization of the photometric error may be used as a loss function:

[0047]

[0048] The element-wise multiplication is represented by ·, O is the occlusion mask, and It′→t=InverseWarp(opticalflowt→t′,It′) is the distorted image from the source image It′ to the target image It when using optical flow. The photometric loss is:

[0049]

[0050] Where SSIM is structural similarity. Edge-aware smooth flow can also be applied:

[0051]

[0052] The smoothing loss provides smoothing of the optical flow in uniform regions of the image and enables flow changes at edges. The total loss is expressed as:

[0053] L=w1*L p +w2*L smooth

[0054] In this context, w1, w2 are weights on the loss components, and opticalflowt→t′ is the optical flow from the target image It to the source image It′. When calculating the occlusion mask, CNN can also be used to calculate the reverse optical flow opticalflowt→t′ (from the source image to the target image) as follows:

[0055]

[0056] where V(x,y) is the region map at position (x,y) on the image at height H and width W, and opticalflowt′→tx opticalflowt′→ty is the horizontal or vertical optical flow component. The occlusion map, also known as the occlusion label, can be determined by thresholding as follows:

[0057] O=min(1,V(x,y))

[0058] In this case, the occlusion map has soft values ​​between 0 and 1, where 0 indicates that the image is occluded and 1 indicates that the image is not occluded.

[0059] The fundamental matrix can then be estimated. Using the optical flow and occlusion mask from the previous step, and the camera calibration matrix K, the fundamental matrix E can be estimated, and the relative rotation R and translation B between the images can be determined with the help of the fundamental matrix decomposition algorithm. The fundamental matrix describes the relationship between the pixels in the two images given the coplanar condition, as follows:

[0060] k x′ T E k x″=0

[0061] Derived: kx′T=x′T(K′)-T

[0062] x′T is the kth transposed pixel position vector from the first image, and (K′)-T is the inverse and transposed calibration matrix of the first image,

[0063] Derivation: kx″=(K″)-1x″

[0064] x″ is the kth pixel position vector from the second image, and (K″)-1 is the inverse calibration matrix of the second image. To estimate the fundamental matrix E, a five-point algorithm can be used, which involves determining five corresponding points on the two images and solving the resulting constrained optimization problem in the form of a least squares formula. Assuming that the optical flow may contain many outliers, all hidden points can be first masked and RANSAC can be used to handle these outliers.

[0065] As a further step, 3D point triangulation and / or depth estimation may be performed. Triangulation may be applied with reference to the relative transformation between the two images to obtain the 3D point corresponding to each point from the optical flow. A closed-form two-vector intersection solution may be used to initialize the triangulation, and then a least squares method may be used to minimize the reprojection error.

[0066] Relative transformation between reference images, calibration, occlusion masks, and triangulated depth. A single image occlusion CNN can be trained as follows: the network can receive separate images as input, or it can work on stereo images and then on a binary mask for occluding objects, which is a vector field of normals to the plane. Each point forms a narrow environment Ω with the surrounding points and outputs a depth estimate. The confidence mask can be trained in a supervised manner by using a binary cross entropy loss from optical flow and an occlusion mask as the ground truth.

[0067] L o =binaryCrossEntropy(prediction,O)

[0068] The prediction in this case is the prediction of the CNN in the range [0,1], where 1 means no occlusion and 0 means occlusion. O describes the occlusion mask from the optical flow, where 1 means unoccluded and 0 means occluded. The depth of occluded objects can also be learned using the L1 loss.

[0069]

[0070] where d is the predicted disparity, is the actual disparity. The depth can be calculated as Depth = 1.0 / d, and the surface perpendicular to the raised object can be calculated by first calculating the homography:

[0071]

[0072] Where H is the homography, K is the calibration matrix, g is the scaling factor, is the translation vector, and is the vector perpendicular to the surface plane at position i∈Pos, and where Pos refers to all spatial positions in the vector field generated by the CNN. The position of the plane at position i can be obtained using To identify, where d i is the depth at position i. There are two options for defining the loss function. The first option aims to directly regress the surface normal This ignores whether there are problems with obstacles or the street surface. In this case a smooth L1 loss can be used:

[0073] L repr =∑ i∈Pos ∑ j∈Ω ||HomographyWarp(I(j), Θ) i )-I(j)||1

[0074] where HomographyWarp distorts part of the image with a homography, and I is the original image.

[0075] With reference to the calculated angle of the normal n, an estimate can then be calculated at each point as to whether a dangerous object is located at that point. This is performed in particular when the angle exceeds a certain angle g. This can generate an obstacle map or a point cloud.

[0076] The second option is more complex, but it offers the advantage of generating additional obstacle point clouds with the aid of hypothesis testing. In this option, the CNN returns two vector fields: It represents street level or open space; and which represents the surface of the object. What complicates the loss calculation is that there must be a ground truth or decision about which is the “correct” normal vector. This is because the loss should only add up the contribution of the “correct” normal vector. To do this in an unsupervised manner, a “street-level / open space” label f ​​can be used if:

[0077]

[0078] With this allocation, another (second) loss option can be formulated:

[0079]

[0080] For this other (second) loss option, a barrier point cloud can be created based on a ratio hypothesis test:

[0081]

[0082] Where γ is a threshold calibrated with reference to the validation dataset and describes whether the point is considered an object or an obstacle. The total loss can be defined as:

[0083] L=w1L o +w2L d +w3L repr

[0084] Hence, both a supervised option for the second step and a hybrid solution including a supervised element are available.

[0085] Figure 2 The first solution 210 shown uses, among other things, supervised fine-tuning of the resulting CNN to detect occlusions in the first stage. A machine learning classifier can be trained in a supervised manner to learn a classification of self-supervised features of individual 2D boxes. The classifier can be a SVM, logistic regression, or CNN-based head that learns a direct classification between occlusions and 2D objects in the image. The CNN can be trained in a class-independent manner, meaning that there is only one "obstruction" class.

[0086] Also there Figure 2The second solution 211 shown in uses, among other things, the occlusion points generated during the first stage and performs clustering based on DBScan using an R-tree. The outermost points of the cluster define the bounding box of the obstacle object. This post-processing can be the same as in P. Pinggera, U. Franke and R. Mester, "High-performance long range obstacle detection using stereo vision," 2015 IEEE / RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 2015, pp. 1308-1313, doi: 10.1109 / IROS.2015.7353537. A small classifier can be run on the bounding box area (e.g., a CNN or a fully networked neural network) as a mechanism for reducing false alarms. The classifier can be fully trained under supervision and includes more classes than just the "obstacle" class.

[0087] The foregoing description of the embodiments describes the invention only in the context of examples. Of course, the individual features of the embodiments can be freely combined with one another if this makes technical sense without departing from the scope of the present invention.

Claims

1. A method (100) for detecting at least one obstacle in an automatic and / or at least semi-autonomous driving system (60), the method comprising the following steps: - providing (101) image data, wherein the image data is dedicated to recording the environment of the driving system (60), - performing (102) an evaluation of the provided image data, wherein the evaluation is performed based on the application of a machine learning model (50) by means of which an occlusion label is determined for at least one occlusion of the environment, - performing (103) detection of the at least one obstacle based on the determined occlusion signature.

2. The method (100) according to claim 1, characterized in that: The training of the machine learning model (50) is based on occlusion areas (304, 401), which are determined based on the movement (303) in the camera recording, wherein, for this purpose, the optical flow is preferably estimated in the image sequence obtained from the camera recording, and the machine learning model (50) is trained with reference to the estimated optical flow in order to determine the occlusion label, wherein the training is preferably performed in the form of a self-supervised training process.

3. The method (100) according to any one of the preceding claims, characterized in that The image data, in particular in inference mode, comprises at least one or exactly one single image, which is generated by means of a recording with a monocular or stereo camera, wherein the image data used as input for the machine learning model (50) for determining the occlusion label is preferably limited to the single image.

4. The method (100) according to any one of the preceding claims, characterized in that The occlusion label is specific to the at least one occlusion and is preferably designed as an occlusion map, which identifies at least one or more regions in the image data that are occluded by at least one object in the environment.

5. The method (100) according to any one of the preceding claims, characterized in that The detection of the at least one obstacle comprises an evaluation of the occlusion labels, preferably by means of a classifier, during which a classification of one of the objects is performed with reference to the occlusion labels, which object is in the form of a dangerous object associated with the respective occlusion and detected in the image data, wherein the dangerous object comprises in particular cargo that has fallen from a truck.

6. The method (100) according to any one of the preceding claims, characterized in that Based on the evaluation and / or detection, at least partially autonomous control of the ego vehicle (5) and / or the robot (5) is performed by the driving system (60), preferably by a motion planning system.

7. A training method (200) for training a machine learning model (50), the training method comprising: - providing (201) training data, wherein the training data comprises at least one sequence of images representing an environment of a driving system (60) during a trip, wherein the training data further comprises annotation data indicating an occlusion label representing at least one occlusion of the environment during the trip, - performing (202) training of the machine learning model (50) based on the training data, during which optical flow in the image sequence is taken into account in order to predict the occlusion labels.

8. A computer program (20) comprising instructions which, when the computer program (20) is executed by a computer (10), prompt the computer (10) to perform the method (100) according to any one of the preceding claims.

9. A device (10) for data processing, the device being configured to execute the method (100) according to any one of claims 1 to 7.

10. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), prompt the computer (10) to perform the method (100) according to any one of claims 1 to 7.

11. A machine learning model (50), wherein the machine learning model (50) is obtained by the training method (200) according to claim 7.