Method for locating a track-guided vehicle

Abstract

The invention encompasses the following: a method for locating a track-guided vehicle on a route, wherein a measurement result is generated with at least one sensor of the locating device, and location information representing the location of the vehicle is calculated from the measurement result, wherein either in a first routine a first sensor, which is not an imaging sensor, is used and the calculated location information is used to train a locating system for the vehicle by means of computer-aided object recognition based on images generated as a measurement result by an imaging second sensor, wherein objects recognized in the images are each stored linked with the location information calculated from the measurement result of the first sensor at the time of image acquisition, and / or in a second routine the first sensor is used and the calculated location information is utilized.to validate vehicle location by object recognition using images generated by the second sensor, wherein objects detected in the images are compared with stored objects linked to stored location information, and if a match is found, the stored location information is compared with the calculated location information and object recognition is validated; and / or in a third routine, the second sensor is used to generate images, objects detected in the images are compared with stored objects linked to stored location information, and if a detected object matches a stored object, the stored location information is used to control (ATP) and / or monitor (ATP) the vehicle movement.where each of the aforementioned first, second, and third routines is executable by the computing environment.

Description

Technical field

[0001] The invention comprises a method for locating a track-guided vehicle. Furthermore, the invention comprises a vehicle with a tracking device. The invention also comprises a computer program product containing program instructions. Finally, the invention comprises a computer-readable storage medium containing data. Technical background

[0002] The current state of the art involves the use of relatively complex odometry based on, for example, balises, radar, GPS, and odometers. Absolute positioning is achieved, for instance, by balises and / or GPS. Relative positioning is achieved, for example, by radar and odometers, by measuring the vehicle's speed from the last known location determined by absolute positioning and then extrapolating the vehicle's position based on the measured speed and known route. This process introduces propagation errors. Thus, positioning is relatively accurate when absolute positioning is possible and increases in uncertainty when positioning is based solely on relative probing, with the propagation error growing the longer the relative positioning takes.

[0003] In particular, CBTC (Communication Based Train Control) systems require the ability to locate trains on the track. Modern systems rely on position reports from the vehicles to achieve short headways. For safe train operation, it is also necessary for trains to know their speed. This is required, among other things, to determine and monitor braking curves and safety distances, and indirectly to update the train's position (relative positioning) when absolute positioning data is unavailable. The speed information must be as accurate as possible and as independent as possible from wheel slippage, which affects conventional odometry (using speedometers). Currently, this problem is solved in positioning systems by additionally using, for example,Signals from displacement pulse generators and slip-independent radar sensors are used and fused to determine a safe speed using sensor error models.

[0004] Document US 2016 / 0121912 A1 describes how the localization of a track-guided vehicle can also be achieved through object recognition of objects located along the track. After the objects have been recognized, their position, which is stored, for example, in a database, is used as location information.

[0005] Document US 2017 / 0327138 A1 describes a method for training object recognition for lane-guided vehicles using artificial intelligence (AI) with reinforcement learning (RL). RL is a machine learning (ML) technique used to train software to make decisions in order to achieve optimal results. It mimics the learning process by which humans achieve their goals through trial and error. Once a recognition process has been learned or trained, it can be used to locate the lane-guided vehicle. A distinction is made between a training phase of the system, during which the system is not yet available, and an application phase, during which location can be achieved using object recognition.

[0006] According to document EP 4067202 A1, a method is known for locating track-guided vehicles in a vehicle depot. This is achieved using cameras that recognize defined location markers within the depot in images whose positions are known. This is particularly easy to implement in a depot, as it is typically located in an enclosed area, meaning that the location markers are highly susceptible to external influences such as vandalism, allowing the method to be applied reliably.

[0007] The problem arising from the described state of the art is that positioning methods and speed determination methods that rely on object detection are subject to changes, particularly on open (freely accessible) track. This means that object detection, even if a computer has been trained for it, becomes less reliable over time. It must be considered that the safety requirements applicable to rail traffic must always be met. This necessitates repeating the training process to regain the required reliability. This, in turn, leads to restrictions in regular train operations. Summary of the invention

[0008] The object of the invention is to solve the problems described in the prior art. In particular, it is an object to provide a method for locating and / or determining the speed of a track-guided vehicle that can operate stably even during extended periods of operation and can permanently meet the high safety requirements applicable to rail traffic. Furthermore, it is an object of the invention to provide a vehicle, a computer program, and a computer-readable storage medium with which the improved method can be implemented.

[0009] According to a first aspect of the invention, a method for locating a track-guided vehicle on a route is described, in which a tracking device installed in the vehicle generates location information, wherein a) a measurement result is generated with at least one sensor of the tracking device, b) location information representing the location of the vehicle is calculated from the measurement result in a computing environment.

[0010] A tracking device within the meaning of the invention is a device that generates a measurement result by means of a sensor. Additionally, data processing can take place within the tracking device, for example, by means of a processor. The sensors can operate using different functional principles, and the measurement results can be either analog or digital. However, when imaging sensors are mentioned within the scope of this invention description, then the measurement result of the sensor, or the processing of this measurement result by data processing, is image information. This image information can be represented two-dimensionally as an image on a screen. The image information can be digital, preferably as a matrix of pixels, or analog.

[0011] A device is computer-aided or computer-implemented if it has a computing environment, or a method is computer-implemented if a computing environment performs at least one step of the method.

[0012] A computing environment is an IT infrastructure consisting of functional components such as processors, memory units, programs, and the data to be processed by these programs. This data is used to execute at least one application, which has a specific task to perform. Additional functional components can include sensors and actuators, which enable the computing environment to interact with the outside world. The IT infrastructure can also be organized as a network of these functional components.

[0013] A cloud (also known as a computing cloud or data cloud) is a computing environment for cloud computing. It refers to an IT infrastructure that is made available via network interfaces such as the internet. It typically includes storage space, computing power, or software as a service, without requiring these to be installed on a computing instance using the cloud. The services offered within the framework of cloud computing encompass the entire spectrum of information technology and include, among other things, IT infrastructure, platforms, software, and computing power. The cloud provider distributes the offered resources to cloud users according to demand, with the aim of optimizing resource utilization.

[0014] Since railway technology is subject to high safety standards regarding the functionality (operational reliability, safety) and vulnerability (transmission security, security) of computer-implemented solutions, the functionalities of a cloud used in railway technology are typically limited with respect to their shared availability. In particular, restrictions are therefore necessary regarding access by a potentially unlimited number of cloud users. Access must also be limited with regard to the sharing of computing resources among different computing instances, in order to ensure necessary redundancy. A technology that takes these restrictions into account for railway technology is also referred to as a private cloud in the context of this invention, even though a private cloud only partially fulfills the technical characteristics associated with cloud technology.

[0015] Within a computing environment, computing instances form functional units that can be assigned to applications (defined, for example, by a number of program modules) and can execute them. During application execution, these functional units form self-contained systems, either physically (e.g., computer, processor) and / or virtually (e.g., program module).

[0016] Computers are electronic devices consisting of several functional components and possessing data processing capabilities. For example, computers can be clients, servers, handheld computers, communication devices, and other electronic devices for data processing, which may include processors and memory units and may also be interconnected via interfaces to form a network.

[0017] Processors can be, for example, converters, sensors for generating measurement signals, or electronic circuits. A processor can be a central processing unit (CPU), a microprocessor, a microcontroller, or a digital signal processor, possibly in combination with a memory unit for storing program instructions and data. The term "processor" can also refer to a virtualized processor or a soft CPU.

[0018] Storage units can be implemented on computer-readable storage devices in the form of random-access memory (RAM) or data storage devices (hard disk or data carrier).

[0019] Program modules are individual software functional units that enable a program sequence of process steps according to the invention. These software functional units can be implemented in a single computer program or in several communicating computer programs. The interfaces implemented here can be implemented in software within a single processor or in hardware if multiple processors are used.

[0020] Interfaces can be implemented using hardware, for example wired or wireless connections, or software, for example as interaction between individual program modules of one or more computer programs, and serve to exchange data, preferably in the form of digital data sets or analog signals.

[0021] To avoid misunderstandings, it should be noted that individual claim features are numbered with lowercase Latin letters, without regard to the claim numbering. This means that each letter appears only once in the entire claim set, allowing for unambiguous addressing of the relevant claim features without mentioning the claim number. Therefore, the order of the letters is irrelevant.

[0022] According to the invention, it is provided that in a first routine (data set creation for image recognition for localization) c) a first sensor, which is not an imaging sensor, is used as the sensor and the calculated location information is used to train a vehicle location system using computer-aided object recognition based on images generated by a second imaging sensor as a measurement result, wherein d) objects detected in the images of the route and its surroundings are each stored linked to the location information calculated from the measurement result of the first sensor at the time the image was captured, and / or in a second routine (validation of data sets for image recognition for location), e) the first sensor is used as the sensor, and the calculated location information is used to validate a vehicle location system using object recognition performed in the computing environment based on images generated by the second sensor, wherein f) objects detected in the images are linked to stored objects.which are linked to stored location information, g) if a detected object matches a stored object, the stored location information is compared with the calculated location information and the object recognition is validated if a match between the calculated location information and the stored location information could be found within a tolerance range, and / or in a third routine (location with image sensor based on validated objects), h) the second sensor is used as a sensor to generate images, i) objects detected in the images are compared with stored objects linked to stored location information, j) if a detected object matches a stored object, the stored location information is used to control (ATP) and / or monitor (ATP) the vehicle movement, with the computing environment set up.that each of the aforementioned first, second, and third routines is executable by the computing environment.

[0023] It is important that the execution of the method according to the invention is only meaningful if, depending on the existing situation, the appropriate of the three routines is carried out, or several of the appropriate ones are carried out in parallel. In other words, the method must be designed such that each of the routines can be carried out on the vehicle. The individual routines do not represent exclusive alternatives, and one or more of the routines can be carried out simultaneously.

[0024] Object recognition must be trained in a known manner. Computer-aided artificial intelligence can be used for this purpose. In the context of this invention, artificial intelligence (hereinafter also abbreviated as AI) refers specifically to the capability of computer-aided machine learning (hereinafter also abbreviated as ML). This involves the statistical learning of algorithm parameterization, preferably for highly complex applications. Using ML, the system recognizes and learns patterns and regularities in the acquired process data based on previously inputted training data. With the aid of suitable algorithms, ML can independently find solutions to emerging problems.ML is divided into three fields - supervised learning, unsupervised learning and reinforcement learning, with more specific applications, for example regression and classification, structure recognition and prediction, data generation (sampling) or autonomous action.

[0025] In supervised learning, the system is trained by observing the relationship between input and corresponding output of known data, thereby learning approximate functional relationships. The availability of suitable and sufficient data is crucial, because if the system is trained with unsuitable (e.g., non-representative) data, it will learn incorrect functional relationships. In unsupervised learning, the system is also trained with example data, but only with input data and without a connection to a known output. It learns how to form and extend data groups, what is typical for the respective use case, and where deviations or anomalies occur. This allows use cases to be described and errors to be detected.In reinforcement learning, the system learns through trial and error by proposing solutions to given problems and receiving positive or negative feedback on these proposals. Depending on the reward mechanism, the AI ​​system learns to perform corresponding functions.

[0026] An advantage of the invention is that the method's functionality can be ensured even over extended periods of operation by adapting to the situation. The first routine is used to create the data set for image recognition for localization. This routine is performed when the method is first implemented on a specific route. Furthermore, this routine is also performed if, during the third routine's execution, it is determined that not enough validated objects are available for complete localization using the image sensor. This may be due to changes in the conditions of the localization (for example, weather conditions or surface contamination), or if validated objects are no longer available along the route (for example, due to vandalism or construction work).

[0027] The second routine is always performed when objects have been trained and still need to be validated due to safety requirements in railway operations before they are used in the third routine.

[0028] The third routine is used when validated objects are available along the relevant route. It is irrelevant whether these validated objects are sufficient for seamless vehicle tracking or not. It must be considered that other tracking methods are also used on the vehicle, resulting in redundant tracking. While a reliable tracking method is in use, gaps along the route where no validated objects are available can be filled by the first and second routines. This typically requires several vehicle trips. These trips can be performed during operation, provided the vehicle's localization is secured by other methods.

[0029] According to a further aspect of the invention, a vehicle with a tracking device is described, comprising a first sensor and a second sensor and at least one computing instance. According to this aspect, the invention provides that u) the second sensor is configured as an imaging sensor and the first sensor and the second sensor are set up to perform at least the above-mentioned process steps c), e) and h), v) the at least one computing instance is part of a computing environment, wherein the computing environment is set up to perform at least the above-mentioned process steps d), f), g), i) and j).

[0030] The advantages associated with this aspect of the invention have already been explained above, and reference is made to these advantages.

[0031] According to a further aspect of the invention, a computer program product is described, containing program instructions that can be executed by a computing environment. According to this aspect, the invention provides that at least the process steps d), f), g), i) and j) of the aforementioned method are executed.

[0032] According to the invention, a computer program product containing program modules with program instructions is described, wherein the program modules can run in the same computing instance or in several computing instances of the computing environment. The computer program product, which can comprise one or more computer programs, can be used to carry out the method according to the invention and / or its exemplary embodiments, and the advantages described above are achieved through its implementation.

[0033] According to a further aspect of the invention, a computer-readable storage medium containing data, which is stored as data records on the storage medium, is described. According to this aspect, the invention provides that the data records make the computer program product described above, according to the last preceding claim, executable.

[0034] Furthermore, a provisioning device for storing and / or providing the computer program in the form of a computer-readable storage medium is described. The provisioning device is, for example, a storage unit that stores the computer program and makes it available for retrieval. Alternatively or additionally, the provisioning device is a network service, a computer system, a server system, in particular a distributed computer system, such as a cloud-based system or virtual computer system, which stores the computer program on a computer-readable storage medium and preferably makes it available in the form of a data stream.

[0035] The provision of the computer program product takes the form of program modules describing program data sets as a file, in particular as a download file, or as a data stream, in particular as a download data stream. The computer program product is transferred, for example, using the provisioning device to a computing environment so that the method according to the invention can be executed in one or more computing instances of this computing environment. Embodiments of the invention

[0036] Further developments of the invention, describing variants, are explained below without limiting the basic idea of ​​the invention.

[0037] According to one variant, the aspects of the invention explained above are determined by the fact that a fourth routine is performed in order to carry out the first routine in the computing environment. k) which checks whether a specified minimum number of objects and / or objects with a specified spatial density are stored for a section of the route, i) in the event that the specified minimum number and / or the specified spatial density is not reached in the section of the route, the first routine is carried out, whereby additional objects are stored, each linked to the location information calculated from the measurement result of the first sensor at the time the image is generated.

[0038] A minimum number of objects can be specified if the length of the relevant track segment is known. A local density for the objects can be specified if it is to be independent of the length of the relevant track segment. In this case, the density is determined along the track's path, i.e., two-dimensionally. The minimum number of objects on a given track segment, as well as the density, can vary locally. This depends on the track's location. For example, a high traffic density and frequent branching of the track require more objects than a straight, uninterrupted stretch.

[0039] One advantage of this approach is that the number of detected objects can be adapted to the specific conditions of the track section and the operational requirements in that area. These conditions can change during operation (depending on the time of day) or depending on the location of the track. For example, traffic density in a particular section of the railway network may increase, necessitating more objects. It is also possible that previously validated objects (as already mentioned) may be removed from the track section, or that conditions may change, for example, due to extreme (and not yet trained) weather conditions or similar events. In such cases, the need to detect new objects with the first routine is preferably determined using computer-aided algorithms, and the response to execute the first routine is triggered accordingly.

[0040] According to one variant, the aspects of the invention explained above are determined by the fact that a fifth routine is performed to carry out the second routine. m) which checks whether there are any unvalidated objects stored for a section of the route, n) in the event that there are any unvalidated objects stored, the second routine is performed.

[0041] One advantage of this variant is that the second routine is only executed if necessary due to the presence of unvalidated objects. Since executing the fifth routine requires less computing power than the second, this saves computing resources. As a result, the procedure can be carried out with less effort. The computing resources are then available for other control tasks within the railway operation.

[0042] According to one variant, the aspects of the invention explained above are determined by the fact that in the third routine, at least one redundant location information is calculated in parallel with the first sensor and / or at least one further sensor.

[0043] The third routine can be executed in the computing environment in which the track-guided vehicle is integrated as soon as at least one object linked to the associated stored location information has been saved. The third routine can be executed continuously as soon as the track-guided vehicle is in operation on the relevant route. In particular, however, the third routine is used in parallel with other positioning methods that are in use in the computing environment, especially on the track-guided vehicle itself. This multiple positioning is referred to as positioning redundancy within the scope of this invention description.

[0044] In the context of the invention, redundant location information refers to location information that, taking into account predefined tolerance ranges, describes the same actual location of the vehicle, which has been determined multiple times using different methods. Redundant location information is therefore location information that can lead to ambiguity in the vehicle's location if this location information differs from one another (for example, due to measurement errors). In this case, a location determination must be performed within the computing environment as part of data processing to resolve this ambiguity in order to ultimately obtain a result for the location (for example, by a majority decision, also known as voting).

[0045] Redundancy serves the purpose of increasing the reliability of continuous (i.e., uninterrupted) tracking of the tracked vehicle and ensuring multiple backups of the tracking result. The methods used to back up the tracking result are well-known and can readily be integrated into the process supporting tracking redundancy by integrating the tracking results obtained through the third routine. As an example of how to support such a process, the use of a Kalman filter can be mentioned. However, other methods are also generally known and can be employed.

[0046] According to one variant, the aspects of the invention explained above are determined by the fact that a sixth routine is performed using the redundant location information. o) which checks whether the object stored with the associated stored location information has been recognized in a current image at the location marked by the redundant location information, p) in the event that the stored object in question has not been recognized, the redundant location information is used to control the vehicle and the second routine is repeated for the object in question with the current image, whereby in the case of successful validation the current image is stored as an alternative for the object in question.

[0047] One advantage of this variant is that the reliability of the third routine in the procedure for determining the current location of the track-guided vehicle can be increased. If an object is not detected at the expected location under certain external conditions (lighting, weather), but is verified at precisely that location by the second routine, this indicates that the object is indeed at the expected location and was simply not detected under the prevailing external conditions. Therefore, the object is verified again for the prevailing external conditions, and an alternative method for detecting the object in question is available. In other words, with repeated execution of this variant, the object is detected with increasing reliability because all previously trained external conditions reliably lead to the detection of the object in question.

[0048] According to one variant, the aspects of the invention explained above are determined by the fact that for objects to be recognized in the first routine, a first data pool of objects is created which contains types of objects to be recognized in the image recognition and / or a second data pool of objects is created which, if recognized in the image recognition, are not to be used for localization.

[0049] Railway infrastructure in general shares the common feature of recurring objects along the track. These include stations, switches, axle counters, signal masts, sleepers, and other infrastructure elements. This commonality is used to create a general, self-learning approach for train localization using artificial intelligence (AI). Training the AI ​​on these objects simplifies object recognition across all routes, as they are not unique to any specific track section. Nevertheless, they are suitable for locating the train, especially when the sequence of their recognition is stored in a digital track atlas for a given section.

[0050] On the other hand, there are objects that do not belong to the railway infrastructure and / or do not remain permanently in the designated location. Parked vehicles are one example. These must be excluded if they are recognized as vehicles, as they can lead to false results if they move. Furthermore, they do not remain permanently in one place, so training for these objects would not be worthwhile.

[0051] According to one variant, the aspects of the invention explained above are determined by the fact that objects for the first data pool and / or objects for the second data pool are recognized in images in a seventh routine during the vehicle's journey and are stored without any associated location information.

[0052] This approach requires trains to be equipped with imaging sensors that face the direction of travel. For general AI training, images from various train journeys (covering different infrastructure and weather conditions) along with speed information are recorded. Once the objects are stored in the first and / or second data pool, they are advantageously available for all monitored routes. Here, too, multiple alternatives can be stored for each generalized object, which, as already explained, take into account, for example, different lighting conditions or weather situations.

[0053] According to one variant, the aspects of the invention explained above are determined by the fact that recognized objects are included in a digital route atlas, taking into account the associated location information.

[0054] One advantage of this approach is that a digital route atlas allows the sequence of expected objects to be defined in a traceable manner for computer-aided recognition. These objects can be general ones present in the initial data pool, as well as individual objects characteristic of a specific route segment and specifically trained for that segment.

[0055] According to one variant, the aspects of the invention explained above are determined by the fact that The speed of a track-guided vehicle traveling on a route is determined in an eighth routine by: q) examining a sequence of images of the route and its surroundings, whereby the respective acquisition times of the images are stored in such a way that they are assigned to the respective images; r) performing object recognition in the images and recognizing objects; s) determining a perspective shift of at least one recognized object in the images of the sequence, which arises due to the movement of the vehicle; t) converting the perspective shift into a speed, taking into account the acquisition times of the selected number of images.

[0056] This approach solves the problem of needing speed information in addition to location data by deriving the speed information from the temporal sequence of captured images. This will be explained using an example below. Accordingly, at least two images must be captured at a known time interval. These images must show at least partially the same content and are captured by the same imaging sensor, which is preferably oriented in the direction of travel.

[0057] Using algorithms such as Harris Corner, SIFT (Scale Invariant Feature Transform), SURF (Speeded Up Robust Feature), FAST (Features from Accelerated Segment Test) or ORB (Oriented FAST and Rotated BRIEF), which are known in themselves, objects in the images can be determined that are preferably invariant, for example, against different lighting conditions.

[0058] If an absolute location is known, along with the time at which the vehicle was last located, then the speed over time can be used to determine the vehicle's current location within the framework of relative location. This determination is specifically performed along the route (this is a one-dimensional location based on the route, preferably depicted in a digital road map). If the route is known on a two-dimensional map, the location along the route can simultaneously be used to determine the location on this two-dimensional, preferably digital, map. The location method itself is already known using odometric methods (e.g., speedometers). However, the method according to the invention advantageously offers higher accuracy.For example, cameras as imaging sensors are inexpensive to purchase and operate, which advantageously increases the cost-effectiveness of the process.

[0059] Orienting the sensor in the direction of travel allows the system to track the perspective changes of detected objects as the vehicle approaches, causing them to enlarge and move towards the edge of the image. Once an object is detected, tracking it is simplified because it appears larger in the image. Of course, orienting the imaging sensor against the direction of travel, i.e., at the rear of the vehicle, also works, in which case the process is reversed (objects move towards the center of the image and appear smaller).

[0060] In other words, this variant describes a method for determining speed based on a temporal sequence of camera images and the analysis of objects. Due to the optical measurement method, the determined speed is independent of wheel slippage or skidding (a problem with speedometer measurements) and is therefore very valuable for odometry. Alternatively or additionally (for redundancy), other positioning methods can of course also be used.

[0061] According to one variant, the aspects of the invention explained above are determined by the fact that the vehicle is located relatively from a known location of the vehicle at a known time, taking into account the speed.

[0062] The positioning method used in this variant is a relative positioning method, since the objects must be identified by analyzing the images, but do not need to be linked to a specific location along the route. The known location at a given time is preferably determined using an absolute positioning method. This has the advantage that the measurement accuracy for the starting point of the relative positioning method is comparatively precise. The same applies to the accuracy of the relative positioning itself: compared to known and odometric methods, relative positioning is comparatively accurate, and the cumulative error can therefore be kept small.

[0063] According to one variant, the aspects of the invention explained above are determined by the fact that the sensor is aligned with a detection direction at least substantially in the direction of travel.

[0064] An alignment in the direction of travel can be described as follows: a detection direction (for example, the optical axis of a lens of an image sensor, also called the image axis) of the imaging sensor is essentially aligned in the direction of travel. An alignment essentially in the direction of travel means that the angle between the image axis and the direction of travel need not be zero. An angle of up to 5°, preferably up to 2°, between the detection direction and the direction of travel is also possible without impairing the functionality of the method. Advantageously, the smaller the angle, the more accurate the speed measurement becomes. However, an angle of up to 5° can be accepted without the measurement error becoming too large. Exemplary embodiments of the drawing

[0065] Further details of the invention are described below with reference to the drawing. Identical or corresponding drawing elements are provided with the same reference numerals in each figure and are only explained more than once to the extent that differences arise between the individual figures.

[0066] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual variants of the invention, which can be considered independently of one another. Each of these variants further develops the invention independently and can therefore be regarded as part of the invention, either individually or in a combination other than that shown. Furthermore, the described components can also be combined with the variants of the invention described above. Figure 1Figure 1 schematically shows an embodiment of the device according to the invention (track-guided vehicle on a track) with its interactions between the functional components used. Figure 2 shows an exemplary embodiment of a computing environment for the device according to Figure 1 as a block diagram of the individual functional components and the interfaces formed between them, wherein individual computing instances execute program modules that can each run in one or more of the exemplary computers shown, and wherein the interfaces shown can accordingly be implemented in software in one computer or in hardware between different computers. Figure 3 and Figure 4 The images shown are exemplary and were taken with the imaging sensor according to the inventive method. A track and objects or characteristic image areas near the track can be seen in these images. Figure 5 An embodiment of the method according to the invention is shown as a flowchart, wherein the process steps shown can be implemented individually or in groups by program modules, and wherein the computing instances and interfaces are defined according to Figure 2 are indicated by example. Detailed description of the exemplary implementations

[0067] In Figure 1 A railway environment BU is shown in which the inventive method takes place. This involves a Figure 2 The depicted computing environment RU is used. Figure 1The diagram depicts a track-guided vehicle (FZ) traveling in one direction (FR) on track (GL). A signal box (STW) is located along the track formed by track GL. This signal box controls functional components of the railway environment (BU), such as a balise (BL). The balise can be read by means of a balise antenna (BA) on the vehicle (FZ). The vehicle (FZ) also has a first image sensor (BS1) and a second image sensor (BS2), one with its image axis (BAC) oriented in the direction of travel (FR) and the other with its image axis (BAC) oriented in the opposite direction (FR).

[0068] Furthermore, a control center (LZ) is planned, which can, for example, monitor compliance with a timetable for vehicles (FZ) operating in the railway environment (BU). The signal box (STW), the control center (LZ), and the vehicle (FZ) are equipped with antennas (AT), enabling radio-based communication between these units. Additionally, a private cloud (CLD) is planned, allowing data exchange, for example, with a service provider (DL) that offers artificial intelligence in the form of computing power.

[0069] The computing environment RU, in which the inventive method takes place, can be considered jointly by Figure 1 and Figure 2The computing instances and functional components used interact with each other via interfaces. A first interface, S1, connects the track-guided vehicle (FZ) and the control center (LZ). A second interface, S2, connects the vehicle (FZ) and the interlocking system (STW). A third interface, S3, connects the control center (LZ) and the cloud (CLD). A fourth interface, S4, connects the cloud (CLD) and the service provider (DL). A fifth interface, S5, connects the control center (LZ) and the interlocking system (STW). A sixth interface, S6, connects the interlocking system (STW) and the balise (BL).

[0070] According to Figure 2The computers forming the respective computing instances are described in more detail below. In the first computer CP1 at the service provider DL, a first processor PR1 is connected to a first memory unit SE1 via an eleventh interface S11. In the second computer CP2 in the control center LZ, a second processor PR2 is connected to a second memory unit SE2 via a twelfth interface S12. In the third computer CP3 in the signal box STW, a third processor PR3 is connected to a third memory unit SE3 via a thirteenth interface S13. In the fourth computer CP4 in the vehicle FZ, a fourth processor PR4 is connected to a fourth memory unit SE4 via a fourteenth interface S14. In the first image sensor BS1, a fifth processor PR5 is connected to a fifth memory unit SE5 via a fifteenth interface S15. The fourth processor PR4 and the fifth processor PR5 are connected to each other via a seventh interface S7.An eighth interface, S8, connects the fourth processor, PR4, of the first image sensor, BS1, and a sensor, SN (even if the second image sensor, BS2, is in . Figure 2 (Not shown, it could be connected to the fourth processor PR4 in an analogous manner). The sensor SN can be a positioning sensor with any operating principle. It could be a GPS sensor, an odometry sensor in the form of a speedometer, or a distance radar. The fourth processor PR4 and the balise antenna BA are connected via a ninth interface S9.

[0071] If, within the scope of this description of the invention, only computers, processors, storage units or interfaces are mentioned, the information generally refers to all of the computers, processors, storage units and other functional components named above in detail, which, when connected via the interfaces, contribute to the formation of the computing environment RU.

[0072] Figures 3 and 4The figures show two superimposed images, each captured by sensor SN, which faces the direction of travel. Arrows denote displacement vectors between pairs of feature points FP from the two images (hereinafter referred to as feature point pairs). These can only be visualized in the image because the feature points FP from both images are represented in a single figure. Such a visualization is, of course, not necessary for carrying out the procedure. The path of the vectors is calculated computationally by comparing the respective images. The viewing axis vector is shown vertically downwards to illustrate the calculation of the scalar products SP (in Figure 4 to graphically illustrate the first scalar product SP1, a second scalar product SP2 and a third scalar product SP3) between the displacement vectors and the viewing axis.

[0073] The Figure 3An image composition can be discerned in which a straight track schematically leads to a vanishing point FPK on a horizon HZ. This simplifies the explanation of the procedure. How Figure 4 As can be seen, track GL, for example in a curve, is also suitable for carrying out the procedure under certain conditions. Various objects are located along track GL that can be recognized by image recognition and to which feature points (FP) can be attached. For example, a light signal (LS) is shown. The balise (BL) can also be used to generate feature points (FP). However, a motor vehicle (KFZ) is unsuitable, as it can also move and could therefore cause incorrect speed measurements. If it is detected during object recognition, it is excluded from the creation of feature points (FP).

[0074] Independent of object detection, characteristic image areas (CBB) can also be used in the process. These can be identified, for example, by a strong contrast to neighboring image areas, without necessarily representing a specific object. These characteristic image areas (CBB) also "migrate" towards the image edge in successive images, thus enabling the definition of feature points (FP) and, as described in Figure 3 The first displacement vector VV1 is shown. Another feature point FP is formed by the base of the light signal LS, which is used to generate a second displacement vector VV2.

[0075] According to Figure 4The image is captured while the vehicle FZ is traveling around a curve, which is why the track GL appears curved in the image. In this example, the first displacement vector, VV1, represents a feature point pair that has moved relative to itself in the direction of travel. Here, the dot product SP1 = 0.99 is significantly larger than the threshold h = 0.9. The second displacement vector, VV2, lies within a curve. Therefore, SP2, at 0.95, is smaller than SP1 but still larger than the threshold and within the acceptable range. Finally, an erroneous third displacement vector, VV3, is sketched further along the curve. Here, SP3, at 0.88, is smaller than the threshold h, and therefore this feature point pair is discarded.

[0076] The following describes the method according to the invention by way of example, as shown in the flowchart according to Figure 3 will be presented and explained step by step. Figure 3Furthermore, the boxes provide an example of how functional components and computing instances are contained within them. Figures 1 and 2 The individual steps can be carried out. Computer-aided steps take place in the processors, which are not shown in detail. The reading and saving of data to the storage units is shown as an example. Insofar as the interfaces are as described above... Figures 1 and 2 These can also be used in Figure 3 marked.

[0077] In the first step (1), the procedure is started (abbreviated: START). This can preferably happen simultaneously with the vehicle FZ starting its journey. The subsequent steps belong to three routines: a first routine RT1, a second routine RT2, and a third routine RT3. Their execution depends on the conditions described in more detail below and can run in parallel, but this is not mandatory. Therefore, they are in Figure 5represented by three parallel strands of the flowchart.

[0078] In a second step 2, a query is performed to determine whether the density of already recorded objects is too low for the relevant route on which the vehicle FZ will travel (in short: DNS). <?). Ist dies der Fall, wird auf dieser Fahrt die erste Routine RT1 gestartet. Reicht die Dichte jedoch aus, wird die erste Routine RT1 auf dieser Fahrt nicht gestartet. Dabei kann in nicht dargestellter Weise auch zwischen einzelnen Streckenabschnitten der Gesamtstrecke unterschieden werden, so dass die erste Routine RT1 nur für Streckenabschnitte einer zu geringen Dichte an bereits erkannten Objekten durchgeführt wird (nicht näher dargestellt).

[0079] In a third step, location tracking is performed using the first sensor SN (abbreviated: LOC-SN). This ensures that the vehicle's position is known at all times. The subsequent process assumes that this location is available for detecting new objects. This can also be verified, though not shown, through repeated queries. If multiple sensors are available for redundant location tracking, additional sensors can be used for this third step alongside the first sensor SN.

[0080] In step 4, images are created using the first image sensor BS1 (GN-PCT). These images are then used for image recognition to identify new objects in sections of the route with an insufficient density of detected objects. This takes place in step 5.

[0081] In a fifth step (5), the images are analyzed by the service provider DL, which provides the AI ​​(ANA-PCT). Here, objects along the route are identified and correlated with the corresponding location information, which was obtained in the third step and also transmitted to DL. In a sixth step (6), the identified objects are then stored (SV-OBJ). This can preferably be done at DL (since the objects are not yet validated; more on this below) or at the railway operator.

[0082] In a seventh step (7), parallel to step 2, a query is performed to determine whether any detected objects still require validation (abbreviated as VAL?). If present, these objects were already detected and recorded in a previous run using the first routine RT1, as described. If no objects are found, the second routine RT2 is not started for the current run. If objects requiring validation are found, the process continues with step 8.

[0083] In an eighth step (8), location is determined using the first sensor SN (abbreviated: LOC-SN), corresponding to step 3. In a ninth step (9), images are created using the first image sensor BS1 (abbreviated: GN-PCT), corresponding to step 4.

[0084] In a tenth step, the vehicle FZ is located based on the objects detected in the images (LOC-PCT). A position has already been stored for each of these objects, indicating where the vehicle FZ should have been located when the corresponding image was taken. This location information must match the result from step 8 based on the detected object.

[0085] In the eleventh step (11), a comparison is made to see if the location result from step 8 and step 10 match (in short: LOC=?). If so, the object has been verified and can be stored in the fourth storage unit, SE4. If not, the process continues to step 12. In this twelfth step (12), the object is then deleted from the relevant database, as in the example shown. Figure 5 in the first storage unit SE1, because a validation of the object failed (abbreviated: CAN-OBJ).

[0086] In step 13, a query is made to determine whether verified objects are available for location tracking (specifically according to...). Figure 5 in the fourth memory unit SE4). This is usually the case after several training runs of the vehicle FZ on the relevant track. If no objects are present, the third routine RT3 is not executed. Otherwise, the third routine RT3 is (normally) started.

[0087] Optionally, in step 3, a location is then determined using the first sensor SN. This is in Figure 5However, this is not shown because it has no effect on the execution of the third routine, RT3. It merely creates redundancy in the measurement, which increases the reliability of the localization (and is therefore usually performed). In step 14, images are taken during the vehicle FZ's journey, corresponding to step 4, and in step 15, the vehicle FZ is located based on the objects detected in the images, corresponding to step 10 (LOC-PCT).

[0088] Since the location has already been validated by verifying the aforementioned objects in the relevant image, the location information obtained in step 15 can be considered reliable. Alternatively (not shown), this measurement result can be further verified using other location methods. This redundant procedure is well-known and will not be described in detail here. In step 16, the vehicle (FZ) is controlled based on the reliable localization (CTL-FZ).

[0089] While the third routine RT3 is executed until the vehicle FZ reaches its destination, the first routine RT1 and the second routine RT2 are repeatedly executed until no more objects need to be detected or validated. In step 17, after each execution of the respective routine, a query is performed to determine whether the first routine RT1 or the second routine RT2 should be stopped (abbreviated as STP?). If so, the process continues to step 18; otherwise, the first routine RT1 or the second routine RT2 is repeated. In step 18, the procedure is then terminated (abbreviated as STOP). This preferably occurs at the latest simultaneously with the vehicle FZ's departure from a depot, for example. Reference symbol list

[0090] ATA Antenna BABalisen antenna BAC Image axis BLBalise BS1 First image sensor BS2 Second image sensor BU Railway environment BV Image axis vector CBB Characteristic image area CLD Cloud CP1 First computer CP2 Second computer CP3 Third computer CP4 Fourth computer DL Service provider FP Feature point FPK Vanishing point FR Direction of travel FZ Vehicle GL Track HZ Horizon FZ Motor vehicle LS Light signal LZ Control center PR1 First processor PR2 Second processor PR3 Third processor PR4 Fourth processor PR5 Fifth processor RT1 First routine RT2 Second routine RT3 Third routine RUR Computing environment S1 First interface S11 Eleventh interface S12 Twelfth interface S13 13th interface S14 14th interface S15 15th interfaceInterface S2 second interface S3 third interface S4 fourth interface S5 fifth interface S6 sixth interface S7 seventh interface S8 eighth interface S9 ninth interface SE1 first storage unit SE2 second storage unit SE3 third storage unit SE4 fourth storage unit SE5 fifth storage unit SNSensor SPS scalar product SP1 first scalar product SP2 second scalar product SP3 third scalar product STW signal box VV1 first displacement vector VV2 second displacement vector VV3 third displacement vector.

Claims

1. Method for locating a track-guided vehicle on a route, wherein a tracking device installed in the vehicle generates location information, wherein a) a measurement result is generated with at least one sensor of the tracking device, b) in a computing environment a location information representing the location of the vehicle is calculated from the measurement result, characterized by the fact that in a first routine c) a first sensor which is not an imaging sensor is used and the calculated location information is used to train a vehicle location system by means of computer-aided object recognition based on images generated by an imaging second sensor as a measurement result, wherein d) objects detected in the images of the route and its surroundings are stored linked to the location information calculated from the measurement result of the first sensor at the time of image acquisition, and / or in a second routinee) the first sensor is used as the sensor, and the calculated location information is used to validate the location of the vehicle by object recognition performed in the computing environment using images generated by the second sensor, wherein f) objects detected in the images are compared with stored objects linked to stored location information, g) if a detected object matches a stored object, the stored location information is compared with the calculated location information and the object recognition is validated if a match between the calculated location information and the stored location information is found within a tolerance range. and / or in a third routineh) the second sensor is used as a sensor to generate images, i) objects detected in the images are compared with stored objects linked to stored location information, j) if a detected object matches a stored object, the stored location information is used to control (ATP) and / or monitor (ATP) the vehicle movement, where The computing environment is set up so that each of the aforementioned first routine, second routine, and third routine can be executed by the computing environment.

2. Method according to claim 1, characterized by the fact thatto execute the first routine in the computing environment, a fourth routine is performed, k) which checks whether a specified minimum number of objects and / or objects with a specified spatial density are stored for a section of the route, i) in the event that the specified minimum number and / or the specified spatial density is not reached in the section of the route, the first routine is performed, whereby additional objects are stored, each linked to the location information calculated from the measurement result of the first sensor at the time the image is generated.

3. Method according to claim 1 or 2, characterized by the fact that a fifth routine is performed to carry out the second routine, m) which checks whether there are any unvalidated objects stored for a section of the route, n) in the event that there are any unvalidated objects stored, the second routine is carried out.

4. Method according to any one of the preceding claims, characterized by the fact that In the third routine, at least one redundant location piece of information is calculated in parallel with the first sensor and / or at least one other sensor.

5. Method according to claim 4, characterized by the fact that A sixth routine is performed using the redundant location information, o) which checks whether the object stored with the associated stored location information has been recognized in a current image at the location marked by the redundant location information, p) in the event that the stored object in question has not been recognized, the redundant location information is used to control the vehicle and the second routine is repeated for the object in question with the current image, whereby in the case of successful validation the current image is stored as an alternative for the object in question.

6. Method according to any one of the preceding claims, characterized by the fact that For objects that are to be recognized in the first routine, a first data pool of objects is created, containing types of objects that are to be recognized during image recognition, and / or a second data pool of objects is created that, if recognized during image recognition, are not to be used for localization.

7. Method according to claim 6, characterized by the fact that Objects for the first data pool and / or objects for the second data pool are recognized in images in a seventh routine during the vehicle's journey and are saved without associated location information.

8. Method according to any one of the preceding claims, characterized by the fact that Identified objects, taking into account the associated location information, will be included in a digital route atlas.

9. Method according to any one of the preceding claims, characterized by the fact thatThe speed of a track-guided vehicle traveling on a route is determined in an eighth routine by: q) examining a sequence of images of the route and its surroundings, whereby the respective acquisition times of the images are stored in such a way that they are assigned to the respective images; r) performing object recognition in the images and recognizing objects; s) determining a perspective shift of at least one recognized object in the images of the sequence, which arises due to the movement of the vehicle; t) converting the perspective shift into a speed, taking into account the acquisition times of the selected number of images.

10. Method according to any one of the preceding claims, characterized by the fact that The vehicle is located relatively from a known location of the vehicle at a known time, taking into account its speed.

11. Vehicle with a tracking device, comprising a first sensor and a second sensor and with at least one computing instance, characterized by the fact that u) the second sensor is configured as an imaging sensor and the first sensor and the second sensor are configured to perform at least the process steps c), e) and h) according to one of the preceding claims, v) the at least one computing instance is part of a computing environment, wherein the computing environment is configured to perform at least the process steps d), f), g), i) and j) according to one of the preceding claims.

12. Computer program product, containing program instructions that are executable by a computing environment, such that at least the process steps d), f), g), i) and j) of the method according to one of claims 1 - 10 are executed.

13. A computer-readable storage medium containing data which is stored as data records on the storage medium, such that the data records make the computer program product according to the last preceding claim executable.

Images