Obstacle detection in the track region by means of combining railway line data and image data

Abstract

The invention relates to a method for identifying obstacles for a rail vehicle (61). In the method for identifying obstacles for a rail vehicle (61), image data (BD) are captured from a region (U) surrounding the rail vehicle (61). Rails (S) in the surrounding region (U) are located and detected. Furthermore, segmentation data (SD) are determined on the basis of the detected and located rails (S). The image data (BD) and the segmentation data (SD) are input into a basic computer vision model (VFM) which generates result segments (ES). The result segments (ES) of the basic computer vision model (VFM) can be regarded as possible obstacles (H). The invention also relates to an obstacle identification device (50). The invention further relates to a rail vehicle (61).

Description

[0001] Description

[0002] Obstacle detection in the track area by combining track data and image data

[0003] The invention relates to a method for obstacle detection for a rail vehicle. The invention also relates to an obstacle detection device for a rail vehicle. Furthermore, the invention relates to a rail vehicle.

[0004] In rail transport, it occasionally happens that objects such as people or road vehicles, shopping trolleys thrown onto the tracks, or even boulders or fallen trees end up on the track and therefore pose a threat to the safety of rail traffic. In the case of people and road vehicles, they are also highly endangered themselves due to the possibility of a collision with a moving rail vehicle. Therefore, such objects must be detected in time to initiate braking for an approaching rail vehicle so that a collision between the rail vehicle and the detected objects can be prevented.

[0005] Therefore, the detection of potential obstacles of any shape or form that may block the rails is a safety-critical requirement for all types of rail vehicles. Obstacle detection plays a particularly crucial role in autonomous driving or automated assistance in the control of rail vehicles. There are many approaches to solving this problem, with solutions based on artificial intelligence (AI) being the most promising.

[0006] In recent years, deep learning (DL)-based approaches for computer vision have demonstrated superior performance over traditional approaches based on manual feature engineering and are therefore particularly suitable and promising for addressing the challenges in the field of automatic train operation, in particular the detection and perception of obstacles in the environment of driverless rail vehicles. For complex scenarios requiring sophisticated perception (e.g., driverless driving on rails), it is foreseeable that they cannot be mastered without the aid of DL.

[0007] One particular task that has proven to be extremely challenging, but at the same time extremely important for safe autonomous driving, is generic obstacle detection: the detection of any kind of obstacle to the moving train that would pose a safety risk.

[0008] The challenge of this task lies in the way AI systems are typically trained: Object classes and types that need to be classified, recognized, or segmented must be part of an annotated training dataset so that the AI ​​model can learn what the object looks like from the image data. However, typical AI models explicitly trained on domain datasets cannot be generic in the sense that they could recognize objects of any class or type.

[0009] To date, there is no completely generic approach to this problem. Attempts have been made either to solve the problem through supervised learning, in which all object types and classes that could act as obstacles are explicitly learned, or the problem has been posed as an anomaly detection problem. For example, the anomaly detection approach proposes using reconstruction-based methods (e.g., using a variational auto-encoder) to reconstruct the input image, and areas with poor reconstruction would be labeled as anomalies and could thus be considered obstacles. Other approaches attempt to detect obstacles by calculating pixel-by-pixel uncertainties across an image. Areas of high uncertainty then indicate a potential obstacle.However, these models and approaches have not yet shown satisfactory performance beyond the prototype level.

[0010] The task is therefore to provide a method and a device for the generalized detection of obstacles for rail vehicles.

[0011] This object is achieved by a method for obstacle detection for a rail vehicle according to patent claim 1, an obstacle detection device according to patent claim 11 and a rail vehicle according to patent claim 12.

[0012] In the method according to the invention for obstacle detection for a rail vehicle, image data of an area surrounding the rail vehicle is recorded. The surrounding area comprises at least the clearance area of ​​the track bed located in front of the rail vehicle in the direction of travel and corresponding edge areas as safety areas in which, for safety reasons, no obstacle should under any circumstances be located. Furthermore, rails in the surrounding area are detected and localized. These rails preferably comprise the rails running in the direction of travel that belong to the track bed on which the rail vehicle is currently traveling. However, the method according to the invention can be used to monitor adjacent rail lines in addition to the rail line being traveled on. Segmentation data is determined on the basis of the detected and localized rails.The image data and segmentation data are input into a baseline computer vision model, which generates result segments. If an obstacle is detected, a result segment is output as the region encompassing the detected obstacle. The result segments can be used as the final result.

[0013] Optionally, the result segment(s) of the base model for computer-based vision are used as candidate segments for a potentially occurring obstacle. With this advantageous configuration, false-positive candidate segments are sorted out from the candidate segments using AI. Advantageously, obstacles can be detected even more reliably and robustly thanks to the additional check.

[0014] The captured image data, which are preferably captured by a sensor unit, particularly preferably by an image recording unit, most particularly preferably by a camera, comprise 2D image data and / or 3D image data of an area surrounding the rail vehicle.

[0015] The imaged surroundings of the rail vehicle preferably comprise a travel channel of the rail vehicle running in front of the rail vehicle.

[0016] If 3D image data is initially acquired, it can also be converted into 2D image data by projecting it onto a 2D plane whose orientation preferably corresponds to the viewing perspective of the rail vehicle or of the sensor unit monitoring the surroundings of the rail vehicle. This viewing perspective also preferably includes the perspective of a driver looking forward from the rail vehicle or in the direction of travel and / or the perspective with which at least some of the sensors, which are preferably aligned in the direction of travel or in the direction of the longitudinal axis of the rail vehicle, detect the surrounding area.

[0017] As already mentioned, rails are detected and localized in the image data, preferably 2D image data. A method for rail detection is described, for example, in the application DE 10 2020 215 754 . 5 . Systems based on machine learning and in particular on deep learning, which can identify and localize a rail track based on image data, are particularly suitable for the detection and localization of rails. Such a system can conventionally be designed using a deep learning-based segmentation model, which provides a pixel-by-pixel binary classification that provides information about whether a pixel of an image is part of a rail track or not.A particularly suitable AI-based system is a so-called U-Net Convolutional Neural Network, which was trained for the task of systematically segmenting rails. This model has better rail segmentation properties than a generic AI-based model for object detection, such as the aforementioned basic model for computer-based vision, because it was specifically trained for the detection and segmentation of rails and rail tracks. In order to still be able to detect all possible types of objects as obstacles, the results of rail detection and rail localization are combined with the results of the basic model for computer-based vision.

[0018] The combination of rail detection and object detection in the method according to the invention allows the detection of all types of obstacles of any shape and form, since the basic model for computer-based vision is particularly well suited for generic object detection, and its weakness in rail detection is compensated for by the combination with additional, separate rail detection. As already mentioned, a machine learning-based method can be used for rail detection, which is enabled to detect rails in image data through a training process.

[0019] Rail detection is preferably carried out alternatively or additionally by self-localization and the use of information from digital map data as well as by incorporating knowledge of the intrinsic parameters of a sensor unit used to capture the image data. Digital maps include information about the stationary railway infrastructure, such as masts, signals, switches and rails. By determining the location of the rail vehicle in the digital map through self-localization, preferably satellite-based self-localization, it is possible to determine the position or route of the rails traveled by the rail vehicle in the digital map.If the intrinsic parameters of a sensor unit for capturing image data from the environment are known, a three-dimensional course of the rails can be projected into these image data, allowing the rails to be localized and detected in the image data. Intrinsic parameters serve to determine the relationship between the camera coordinate system and the image coordinate system. Intrinsic parameters include the focal length and the optical center of the image plane.

[0020] The aforementioned basic model for computer-based vision (also known as the "Vision Foundation model") is preferably designed as a so-called "SAM" (abbreviation for "Segment Anything Model"). Such a model is provided by META Research at the internet address https: / / segment-anything.com. The model comprises an encoder which encodes an input image into an abstract feature vector. In addition to the input or encoder for image data, the model also comprises an input for segments or points as so-called prompts, which according to the invention were determined on the basis of an AI-based model specialised in rail recognition.

[0021] The abstract feature vector of the input image is then combined with an encoded prompt in a decoder to produce a suitable result. The prompt can in particular have points, segmentation masks or bounding boxes, for which different encoders are used. Segmentation masks or point grids are particularly preferred for the inventive application of the basic model for computer-based vision in order to find out whether objects, in particular obstacles, overlap with the rails. Since localization of rails using the basic model for computer-based vision is not particularly exact, the detection and localization of the rails is carried out separately and the result is then compared with the encoded image data or the input of the prompts, i.e. the abstract feature vectors based on the prompts.the abstract feature vectors based on the image data are fused. For this purpose, the respective feature vectors are entered into a so-called lightweight mask decoder, which is also part of the basic model for computer-based vision. This lightweight mask decoder creates a list of segment masks or bounding boxes of the detected objects that overlap with the rail segments based on the feature vectors from different data sources. In addition to the image data, the basic model for computer-based vision is also given information about the position of the rails, so that this information no longer needs to be determined by the basic model for computer-based vision. Due to the use of this additional information, the robustness of the basic model for computer-based vision is increased.

[0022] To make obstacle detection even more robust, the application of the basic model for computer-based vision is preferably followed by the application of another model which checks or validates the results of the basic model for computer-based vision. For this purpose, the result segments output by the basic model, i.e. in particular the candidate segments, are checked to determine whether they are objects or the rails themselves. This classification into rails and non-rails is preferably carried out by a highly specialized AI-based model, preferably a highly specialized artificial neural network or a model based on machine learning. The robustness of the obstacle detection is advantageously further improved.

[0023] The obstacle detection device according to the invention has a sensor data interface for capturing image data from an area surrounding a rail vehicle. As already explained, the image data can be generated by a sensor unit, in particular by a camera, of the surroundings of the rail vehicle and forwarded to the sensor data interface.

[0024] Part of the obstacle detection device according to the invention is also a localization unit for detecting and localizing rails in the surrounding area. The localization unit comprises technical means for determining the position of the rails. Localization can be performed either directly based on the image data or based on additional information determined by additional measurements, in particular by self-localization.

[0025] The obstacle detection device according to the invention also has a segmentation unit for determining segmentation data based on the detected and located rails. This segmentation data comprises image segments of the image data acquired from the surrounding area.

[0026] Part of the obstacle detection device according to the invention is also a basic model for computer-based vision. The basic model comprises an input unit for inputting the image data and the segmentation data into the basic model for computer-based vision. This input unit comprises a so-called prompt encoder, which generates an abstract feature vector based on the segmentation data, which includes the positions of the rails, and an image data encoder, which generates an abstract feature vector based on the image data.

[0027] The obstacle detection device according to the invention, and in particular the basic model for computer-based vision of the obstacle detection device according to the invention, also comprises a data fusion unit for generating a result segment of the basic model for computer-based vision as a region that includes a detected obstacle, in the event that an obstacle has been detected. The result segment is preferably used as the final result. Alternatively, the result segment is used as a candidate segment for subsequent testing and validation of the result segment.

[0028] The data fusion unit preferably comprises a high-performance mask decoder, in particular a lightweight encoder, with which objects detected in the area of ​​the rails are segmented.

[0029] If the result segment is to be used as a candidate segment for subsequent testing, the obstacle detection device according to the invention preferably has a classification unit for AI-based sorting of false-positive candidate segments from the candidate segments in order to make obstacle detection even more robust. The obstacle detection device according to the invention shares the advantages of the method according to the invention for obstacle detection for a rail vehicle.

[0030] The rail vehicle according to the invention has a sensor unit for capturing image data from the surroundings of the rail vehicle. Furthermore, the rail vehicle according to the invention comprises the obstacle detection device according to the invention. Furthermore, the rail vehicle according to the invention has a control device for controlling the driving behavior of the rail vehicle depending on whether the obstacle detection device has detected an obstacle in the surroundings of the rail vehicle. The rail vehicle according to the invention shares the advantages of the obstacle detection device according to the invention.

[0031] Some components of the obstacle detection device according to the invention can be implemented predominantly in the form of software components executed by a processor of a computing system of a rail vehicle. This particularly applies to the sensor data interface, the localization unit, the segmentation unit, the basic model for computer-based vision, the input unit, the data fusion unit, and the optional classification unit.

[0032] In principle, these components can also be partially implemented, particularly when particularly fast calculations are required, in the form of software-supported hardware, such as FPGAs or the like. Likewise, the required interfaces, for example, when only data is transferred from other software components, can be designed as software interfaces. However, they can also be implemented as hardware interfaces that are controlled by suitable software.

[0033] A largely software-based implementation has the advantage that even computer systems already present in a rail vehicle can be easily retrofitted with a software update after possible addition of additional hardware elements, such as a sensor unit, in particular an image camera, in order to operate in the manner according to the invention. In this respect, the object is also achieved by a corresponding computer program product with a computer program that can be loaded directly into a memory device of such a computer system, with program sections in order to carry out the steps of the method according to the invention that can be carried out by software when the computer program is executed in the computer system.

[0034] Such a computer program product may, in addition to the computer program, include additional components, such as documentation and / or additional components, including hardware components, such as hardware keys (dongles, etc.) for using the software.

[0035] A computer-readable medium, such as a memory stick, a hard disk, or another portable or permanently installed data storage device, can be used for transport to the storage device of the computer system and / or for storage on the computer system, on which the program sections of the computer program that can be read and executed by a computer unit are stored. For this purpose, the computer unit can, for example, have one or more cooperating microprocessors or the like.

[0036] The dependent claims and the following description each contain particularly advantageous embodiments and developments of the invention. In particular, the claims of one claim category can also be developed analogously to the dependent claims of another claim category and their description parts. Furthermore, within the scope of the invention, the various features of different embodiments and claims can also be combined to form new embodiments.

[0037] In one embodiment of the method according to the invention for obstacle detection for a rail vehicle, the image data acquired from the surrounding area comprises 2D image data. Advantageously, the amount of data to be processed is significantly reduced compared to 3D image data, and the detection and localization of rails and obstacles in the area of ​​the rails is simplified due to the smaller amount of data.

[0038] In one variant of the method according to the invention, the image data is captured as 3D image data by a stereo camera. Advantageously, 3D information about the surroundings of a rail vehicle can be captured directly. 3D information has the advantage that distances between objects and the rail vehicle can be determined more easily.

[0039] The 3D image data can also be dynamically captured using a mono camera. In this variant, image data is captured from different directions, and a 3D image is generated from this data. Advantageously, a technically complex stereo camera is not required. The 3D image data preferably includes RGB data. Color differences can be used to distinguish between objects and image sections.

[0040] In one embodiment of the method according to the invention for obstacle detection for a rail vehicle, the detection and localization of rails takes place by a specific AI-based identification of the rails in the acquired image data. Image data can advantageously be segmented using AI in order to separate image sections in which rails run from other image sections. In particular, specific detection and localization of the rails allows a more robust separation of the rails from other detected objects. Thus, potential false-positive obstacles can be more easily recognized as rails. Since generalized object detection methods usually have problems distinguishing obstacles from rails, rail detection can enable more reliable and robust rail localization and segmentation.

[0041] For AI-based identification of rails, methods based on machine learning are particularly suitable and can be easily trained on certain objects, such as rails.

[0042] In a variant of the method according to the invention for obstacle detection for a rail vehicle, the localization of the rails is carried out by semantic segmentation based on deep learning, as a special form of machine learning. In semantic segmentation, pixels are divided into classes. Each pixel is annotated in a training process. Pixels of rail tracks are divided into two classes, which relate to the foreground and the background of the image, respectively. Rails can advantageously be separated and segmented from the background of the image data. Alternatively, or in combination with the method described above, the detection and localization of the rails is carried out on the basis of

[0043] - a digital map,

[0044] - self-localization of the rail vehicle and

[0045] - knowledge of the intrinsic parameters of a sensor unit used to acquire the image data, in particular a camera.

[0046] Advantageously, additional information about the surrounding area of ​​the rail vehicle and the location of the rail vehicle as well as the pose of the sensor unit used to capture the image data of the surrounding area of ​​the rail vehicle are used to localize the rails in the captured image data.

[0047] If the acquired image data comprises 2D image data, a 3D model of the area surrounding the rail vehicle is preferably generated based on the digital map and self-localization. The rails are detected and localized in the 3D model, and the 3D model is projected onto a 2D model based on knowledge of the intrinsic parameters of the sensor unit used to acquire the image data. Finally, the segmentation data is determined based on the 2D model. As already explained, intrinsic parameters serve to restore the relationship between a camera coordinate system and the image coordinate system. Such parameters include the focal length and the optical center in the image plane of the image data.

[0048] In the method according to the invention, the base model for computer-based vision preferably comprises an image encoder and a prompt encoder. The image data captured from the surrounding area are input to the image encoder, and the segmentation data are input to the prompt encoder as prompt data. The image data are converted into an abstract feature vector, and the segmentation data are converted into an abstract feature vector. The abstract feature vectors can be used as input vectors for an AI-based model.

[0049] Particularly preferably, the basic model for computer-based vision comprises a high-performance mask decoder with a reduced number of input parameters, which generates the result segments, in particular candidate segments, for a potentially occurring obstacle based on the encoded prompt data of the prompt encoder and the encoded image data. This high-performance mask decoder combines the information from the output data of the rail detection and the image analysis. Due to a reduced number of input parameters, this mask decoder is particularly powerful. Such a high-performance mask decoder is described in the "SAM" documentation at the Internet address https: / / segment-anything.com.

[0050] Particularly preferably, the AI-based sorting of false-positive result segments, in particular candidate segments, from the result segments, in particular candidate segments, is carried out by an AI-based model of one of the following model types:

[0051] - a machine learning-based process,

[0052] - a support vector machine,

[0053] - a decision tree or decision forest,

[0054] - a classification of nearest neighbors .

[0055] Depending on the area of ​​application, one of the above approaches can be identified as particularly effective and precise through empirical evaluation of the models and can be selected for this area of ​​application.

[0056] The invention will be explained in more detail below with reference to the accompanying figures using exemplary embodiments. Therein: FIG. 1 shows a flowchart illustrating a method for obstacle detection for a rail vehicle according to an exemplary embodiment of the invention,

[0057] FIG 2 is a schematic representation of image data acquired in the method illustrated in FIG 1,

[0058] FIG 3 is a schematic representation of image data similar to the image data already illustrated in FIG 2, with different objects being detected in the rail area,

[0059] FIG 4 shows a schematic representation of an obstacle detection system according to an embodiment of the invention and the models and data used therein,

[0060] FIG 5 is a schematic representation of an obstacle detection device according to an embodiment of the invention,

[0061] FIG 6 is a schematic representation of a rail vehicle according to an embodiment of the invention.

[0062] FIG 1 shows a flow chart 100 which illustrates a method for obstacle detection for a rail vehicle according to an embodiment of the invention.

[0063] In step 1.1, three-dimensional color image data BD, also referred to as RGB image data, is acquired of the surroundings or a surrounding area U in front of a rail vehicle 61 (see FIG. 6). The image data BD is acquired by a sensor unit 62 arranged on the rail vehicle 61 (see FIG. 6). The sensor unit 62 comprises an optical camera for recording color image data BD. The image data BD comprise information about the surrounding area U of the rail vehicle 61. In this exemplary embodiment, the surrounding area U comprises a front area in front of the rail vehicle 61 into which the rail vehicle 61 wishes to enter. In step 1.II, the rails S running in front of the rail vehicle 61 are detected and localized in the image data BD on the basis of the image data BD. In particular, positions of the rails S are determined.

[0064] In step l . III, segmentation data SD are determined on the basis of the detected and localized rails S .

[0065] In step 1.IV, the image data BD and the segmentation data SD are input into a base model for computer-based vision VFM, which generates result segments ES. First, the image data BD and the segmentation data SD are decoded, generating abstract feature vectors that are input into a mask decoder, which generates a valid mask as the result vector based on the abstract feature vectors.

[0066] This mask is output as the result segment ES of the basic model VEM for computer-based vision.

[0067] In step 1. V, the result segment ES is classified as a candidate segment KS for a possible obstacle H.

[0068] In step 1.VI, the candidate segment KS is tested by a classification system to determine whether it may contain a false-positive candidate segment FP-KS. If the candidate segment KS is validated as an obstacle H, the process proceeds to step 1.VI I, in which the candidate segment KS is output as an obstacle H or obstacle segment. If it is detected that the candidate segment KS represents a false-positive result, which is marked with "n" in FIG 1, the process proceeds to step 1.VI II, in which the candidate segment KS is sorted out as a false-positive segment FP-KS.

[0069] FIG. 2 shows a schematic representation 20 of image data BD captured by a camera of a rail vehicle. FIG. 2 illustrates step 1. II, in which the rails S are located and identified.

[0070] In FIG. 3, the image representation similar to FIG. 2 is shown as a schematic representation 30 with a marked safety zone or segmented rail area. The safety zone SB or rail area SB marks the area in the image in which objects are perceived as collision obstacles.

[0071] In FIG. 3, a mast PL is detected as a candidate object or candidate segment KS. Through validation in a second step, it can be determined that the detected object is a mast, and therefore does not represent a potential obstacle H.

[0072] FIG. 4 shows a schematic representation 40 of obstacle detection and the models and data used therein. The input of image data BD is symbolized at the bottom right of FIG. 4. The image data BD can be used both for image processing in the basic model for computer-based vision VFM and for detecting and localizing rails S. For detecting and localizing rails S, the image data BD are used as input data for a rail detection model SDM.

[0073] The rail detection model SDM is based on a semantic segmentation of rails S in image data BD and outputs segmentation data SD . Alternatively, position data from a satellite navigation system GPS can be used as input data for a digital map program, which generates digital map data DLK . With the help of the geoposition of the rail vehicle, the map program finds the position of the rail vehicle on the digital map and can determine the currently traveled rail line in the digital map . This detected rail line can then be projected into the image data space depending on the pose of the rail vehicle's camera and serve there as a synthetic image with segmented rails S . The segmentation data generated in this way is then entered as mask data or point data into a basic model for computer-based vision VFM .

[0074] There they are converted by a so-called prompt encoder PE into an abstract feature vector MVP. In order to be able to use the image data BD directly for object detection, the image data BD is input to an image encoder BE, which also converts the image data BD into an abstract feature vector MVB. The abstract feature vectors are input to a so-called lightweight mask decoder LMD. The lightweight mask decoder LMD determines a list of candidate segments KS and associated feature vectors. This information is passed to a classification model KM, which outputs validated obstacles H as result data.

[0075] FIG 5 shows a schematic representation of an obstacle detection device 50 according to an embodiment of the invention.

[0076] The obstacle detection device 50 comprises a sensor data interface 51 shown on the left in FIG 5 . The sensor data interface 51 is set up to receive image data BD from an environmental area U of a rail vehicle 61 (see FIG 6 ) in which the obstacle detection device 50 is arranged. Furthermore, the obstacle detection device 50 also comprises a localization unit 52 for detecting and localizing rails S in the environmental area U . The localization unit 52 receives both image data BD and self-localization data SLKD and digital map data DLK for determining the course of rails S in the image data space of the acquired image data BD . The self-localization data SLKD can be determined on the basis of a satellite navigation system and the digital map data DLK is carried by the rail vehicle in a map file.Part of the obstacle detection device 50 is also a segmentation unit 53 which is designed to determine segmentation data SD on the basis of the detected and localized rails S.

[0077] The obstacle detection device 50 has a basic model VEM for computer-based vision. Part of this basic model VMF is an input unit 54 for inputting the image data BD and the segmentation data SD into the basic model for computer-based vision VEM.

[0078] Also part of the basic model for computer-based vision is a data fusion unit 55 for generating a result segment ES or multiple result segments ES of the basic model for computer-based vision VEM, as an area that includes a detected obstacle H, in the event that an obstacle H is detected. The result segment ES or the result segments ES is or are classified as a candidate segment KS or candidate segments KS for a possibly occurring obstacle H or multiple possibly occurring obstacles H.

[0079] Finally, the obstacle detection device 50 comprises a classification unit 56 which is configured to sort out AI-based generated false-positive candidate segments FP-KS from the candidate segment KS or, if applicable, the determined candidate segments KS and to determine and output validated obstacles H.

[0080] FIG. 6 shows a schematic representation 60 of a rail vehicle 61 according to an exemplary embodiment of the invention. The rail vehicle 61, shown schematically on the left in FIG. 6, travels from left to right in the direction of the arrow on rails S of a track and records image data BD from an environmental region U in front of the rail vehicle 61. For this purpose, the rail vehicle 61 comprises a sensor unit or camera unit 62 in its front region. The image data BD are transmitted to an obstacle detection device 50, which is also part of the rail vehicle 61 and has the structure illustrated in FIG. 5. The obstacle detection device 50 then determines on the basis of the image data BD that a collision obstacle H is located in the surroundings, more precisely on the right rail S of the track shown in FIG. 6.This result H is transmitted to a control device 63 included in the rail vehicle 61, which, in response to this collision obstacle H, transmits control commands to individual functional units of the rail vehicle 61. For example, the brakes of the rail vehicle 61 are activated in order to bring the rail vehicle 61 to a stop before the collision obstacle.

[0081] Finally, it is pointed out once again that the methods and devices described above are merely preferred embodiments of the invention and that the invention can be varied by a person skilled in the art without departing from the scope of the invention, insofar as it is defined by the claims. For the sake of completeness, it is also pointed out that the use of the indefinite articles "a" or "an" does not exclude the possibility that the features in question may be present in multiple copies. Likewise, the term "unit" does not exclude the possibility that this may consist of several components, which may also be spatially distributed. Regardless of the grammatical gender of a particular term, persons with male, female or other gender identity are included.

Claims

Patent claims 1. Method for obstacle detection for a rail vehicle (61), comprising the steps: - capturing image data (BD) from an environmental area (U) of the rail vehicle (61), - Detecting and locating rails (S) in the surrounding area (U), - Determination of segmentation data (SD) based on the detected and localized rails (S), - Inputting the image data (BD) and the segmentation data (SD) into a basic model for computer-based vision (VFM), which, in the event that an obstacle (H) is detected, generates a result segment (ES) as an area that includes the obstacle (H), - Outputting the result segment (ES) as an area which includes a detected obstacle (H) in case an obstacle (H) has been detected.

2. The method according to claim 1, wherein - the output result segment (ES) of the basic model for computer-based vision (VEM) is classified as a candidate segment (KS) for a potentially occurring obstacle (H) and - the candidate segment (KS) is checked using AI and, if it is detected as a false positive candidate segment (FP-KS), is sorted out.

3. The method according to claim 2, wherein the AI-based checking and optionally sorting out of the candidate segment (KS) as a false positive candidate segment (FP-KS) is carried out by an AI-based model of one of the following model types: - a machine learning-based process, - a support vector machine, - a decision tree or decision forest, - a classification of nearest neighbors.

4. Method according to one of the preceding claims, wherein the image data (BD) acquired from the surrounding area (U) comprise one of the following types of image data: - 2D image data, - 3D image data .

5. Method according to one of the preceding claims, wherein the detection and localization of rails (S) is carried out by an AI-based identification of the rails (S) in the acquired image data (BD).

6. The method according to claim 5, wherein the AI-based identification of the rails (S) is based on machine learning.

7. Method according to one of the preceding claims, wherein the detection and localization of the rails (S) is based on - a digital map (DLK), - self-localization of the rail vehicle (61) and - the knowledge of the intrinsic parameters of a sensor unit (62) used to acquire the image data (BD).

8. The method according to claim 7, wherein - the acquired image data (BD) includes 2D image data, - a 3D model of the surrounding area (U) of the rail vehicle (81) is generated on the basis of the digital map (DLK) and the self-localization, - the rails (S) are detected and localized in the 3D model and - the 3D model is projected onto a 2D model based on the knowledge of the intrinsic parameters of the sensor unit (62) used to acquire the image data (BD), - the segmentation data (SD) are determined based on the 2D model.

9. Method according to one of the preceding claims, wherein the basic model for computer-based vision (VEM) comprises an image encoder (BE) and a prompt encoder (PE) and the image data (BD) acquired from the surrounding area (U) are input into the image encoder (BD) and the segmentation data (SD) are input as prompt data into the prompt encoder (PE).

10. The method according to claim 9, wherein the basic model for computer-based vision (VEM) comprises a high-performance mask decoder (LMD) with a reduced number of input parameters, which generates the result segment (ES) for a possibly occurring obstacle (H) on the basis of the encoded prompt data (MVP) of the prompt encoder (PE) and the encoded image data (MVB).

11. Obstacle detection device (50) comprising: - a sensor data interface (51) for acquiring image data (BD) from an environmental area (U) of a rail vehicle (61), - a localization unit (52) for detecting and localizing rails (S) in the surrounding area (U), - a segmentation unit (53) for determining segmentation data (SD) on the basis of the detected and localized rails (S), - a basic model for computer-based vision (VEM), comprising: - an input unit (54) for inputting the image data (BD) and the segmentation data (SD) into the basic model for computer-based vision (VEM), - a data fusion unit (55) for generating a result segment (ES) of the basic model for computer-based vision (VEM) as a region which includes a detected obstacle (H), in the event that an obstacle (H) has been detected.

12. Rail vehicle (61) comprising - a sensor unit (62) for capturing image data (BD) from the surroundings of the rail vehicle (61), - an obstacle detection device (50) according to claim 11, - a control device (63) for controlling a driving behavior of the rail vehicle (61) depending on whether an obstacle (KH) in the environment of the rail vehicle (61) has been detected by the obstacle detection device (50).

13. Computer program product with a computer program which can be loaded directly into a memory unit of a control device (63) of a rail vehicle (61), with program sections to carry out all steps of a method according to one of claims 1 to 11 when the computer program is executed in the control device (63).

14. A computer-readable medium on which program sections executable by a computer unit are stored in order to carry out all steps of the method according to one of claims 1 to 11 when the program sections are executed by the computer unit.

Images