Automated sensor alignment

Abstract

Procedure (100), in which - a position of the first kind of a reference object (16, 18) relative to a given reference point (22) is determined (102); - by means of at least one sensor (10, 12) of a rail-bound vehicle (14) a position of the second kind of the reference object (16, 18) within a detection range (20) of this at least one sensor (10, 12) is detected (104); - as a reference object (16, 18) a platform edge or a platform marking which is set up for the purpose of operating the said rail-bound vehicle (14) is provided (106); - based on a comparison of the determined (102) position of the first kind and the recorded (104) position of the second kind, an orientation of the detection range (20) of the at least one sensor (10, 12) of the rail-bound vehicle (14) relative to the aforementioned reference point (22) is determined (108) and - an orientation of a detection area (20) of each of at least two sensors (10, 12) of the rail-bound vehicle (14) relative to the specified reference point (22) is determined by means of the same reference object (16, 18) (108).

Description

[0001] The invention relates to a method for determining an orientation of a detection area of ​​a sensor, a rail-bound vehicle by means of which the method can be carried out, a computer program and a computer-readable medium.

[0002] In order to determine which section of a room lies within the detection range of a sensor, the orientation of the detection range relative to a reference point is determined. Furthermore, multiple sensors are usually provided whose detection ranges at least partially overlap. In this way, the quality of recorded sensor data can be improved and / or additional information regarding observed objects can be obtained that goes beyond the sensor data itself. For example, the objects can be classified, localized and / or identified in this way using computer-implemented algorithms. For this purpose, however, the orientation of the detection ranges of the multiple sensors relative to one another must be determined. This makes it possible to fuse data recorded by each of the multiple sensors in such a way that a common virtual detection range can be provided.This procedure is therefore known under the term sensor data function.

[0003] To determine the alignment of the detection range of each of the multiple sensors relative to a reference point, it has previously been necessary to create a defined test environment. In such a test environment, so-called test marks are provided. The position of the test marks relative to the reference point is known. The test marks can also be used to determine whether multiple sensors are detecting the same object or whether the detected objects are different. The information thus obtained is then used to determine how the individual detection ranges of the multiple sensors are aligned relative to one another and relative to the reference point.

[0004] A large number of sensors are provided, particularly for rail-bound vehicles. In one application, sensors in optical cameras are used to monitor passenger boarding and alighting processes. For example, a gap between a platform edge and the vehicle can be monitored in this way to ensure high operational reliability. In order to determine the alignment of a field of view of such cameras relative to the vehicle and relative to each other, test tracks are currently prepared with test markers that are intended to be detected by the optical cameras. This process is associated with a high level of personnel and time expenditure. Furthermore, an interruption in service has been necessary to determine the alignment. Therefore, the vehicles must be taken out of service and are not available for operational use.

[0005] WO 2019 / 154565 A1 describes a method for calibrating a detection device, a counting method, a computer program product, and a detection device for a passenger transport vehicle. The method comprises detecting a spatial area using an image acquisition unit of the detection device. Furthermore, image information is generated by the image acquisition unit based on the detection, wherein the image information represents the spatial area. The method further comprises detecting a prominent area of ​​the spatial area using an image recognition device based on the image information and calibrating the detection device based on the prominent area.

[0006] DE 10 2020 112 289 A1 describes a method for calibrating an environmental sensor of a vehicle, comprising the steps of: receiving sensor data from the environmental sensor, wherein the sensor data describe an orientation of the environmental sensor relative to a body of the vehicle, comparing the sensor data with reference sensor data, determining body data, wherein the body data describe an orientation of the body of the vehicle relative to a roadway, comparing the body data with reference body data, and checking an orientation of the environmental sensor relative to the roadway based on the comparison of the sensor data with the reference sensor data and based on the comparison of the body data with the reference body data.

[0007] The object of the invention is to facilitate the determination of an orientation of a detection area of ​​a sensor.

[0008] This object is achieved by the features of the method according to independent claim 1.

[0009] Furthermore, the invention is based on the object of specifying a device for carrying out the method.

[0010] This problem is solved by a rail-bound vehicle having the features of the subordinate claim.

[0011] Furthermore, it is an object of the invention to provide a computer program and a computer-readable medium.

[0012] These objects are achieved by a computer program according to the features of the independent computer program claim 8 and by a computer-readable medium according to the features of the independent claim 9.

[0013] Advantageous further training is the subject of dependent subclaims.

[0014] In the method according to the invention, a position of a first type of reference object relative to a predetermined reference point is determined. Furthermore, a position of a second type of the reference object within a detection range of this at least one sensor is detected by means of at least one sensor of a vehicle. A part of said vehicle is provided as the reference object. Alternatively or additionally, a part of an infrastructure facility that is set up for the purpose of operating said vehicle is provided as the reference object. Furthermore, the method according to the invention provides that, based on a comparison of the determined position of the first type and the detected position of the second type, an orientation of the detection range of the at least one sensor of the vehicle relative to the aforementioned reference point is determined.

[0015] Using the aforementioned reference objects, a convenient and automated determination of the alignment of the detection range of at least one sensor can be carried out. In particular, the alignment of the detection range can be determined independently of a prepared test track. In this case, vehicle-specific or stationary parts of an infrastructure facility can be used to determine the alignment of the detection range. In a preferred application, this enables the detection range to be aligned without a dedicated interruption in operation. This increases the frequency of alignment determination, thus improving the accuracy and reliability of the data acquired by the sensor and the information obtained therefrom. The vehicle does not have to be taken out of service to correct an alignment error.Instead, an alignment error can be carried out, for example, at a terminal station without requiring any additional service interruption.

[0016] According to the invention, an alignment of a detection range of each of at least two sensors of the vehicle is determined relative to the predetermined reference point. This makes it possible to determine the extent to which the two sensors have an overlapping detection range. Such a determination of the alignment of the respective detection ranges provides the basis for sensor data fusion, by means of which data can be acquired in a detection range that is expanded compared to the independent detection range of a single sensor. The method can therefore be applied in a variety of ways. In particular, the method is not limited to the alignment of a predetermined sensor type. Instead, different sensor types can be aligned relative to one another, such as an optical camera relative to an infrared camera or a radar device.

[0017] Furthermore, the orientation of the detection range of each of the at least two sensors of the vehicle relative to the predetermined reference point is determined using the same reference object. In this way, the orientation of the detection ranges of the at least two sensors relative to one another can be precisely determined. In this process, it can also be determined whether the objects detected by the at least two sensors are the same object or different objects. Furthermore, this enables results to be determined with high accuracy using data fusion. This allows, for example, a reliable localization, classification, and / or identification of an object located in the detection range.

[0018] Furthermore, it is planned to use a platform edge or platform marking as a reference object. This allows sensors that are part of a rail-bound vehicle to be aligned reliably and cost-effectively. A signaling system of the infrastructure facility is expediently provided as the reference object.

[0019] Preferably, a one-dimensional or two-dimensional ultrasonic sensor array is provided as the at least one sensor. Alternatively or additionally, the at least one sensor is preferably provided as part of an optical camera, a radar device, a lidar device, an infrared camera, or a time-of-flight camera. Determining the orientation of detection areas of such sensors is generally associated with considerable effort. This effort can be easily avoided using the present method. Instead, a cost-effective and reliable determination of the orientation of the detection areas of such sensors can be performed.

[0020] Furthermore, an advantageous development of the method provides that the orientation of the detection area of ​​the at least one sensor relative to the reference point is determined using a computer-implemented algorithm. Preferably, the orientation of the detection area of ​​each of the at least two sensors is determined using the computer-implemented algorithm. This enables an automated determination of the orientation of the detection areas relative to the reference point.

[0021] Preferably, the computer-implemented algorithm is a machine learning algorithm. For example, the computer-implemented algorithm is based on artificial neural networks. Particularly preferably, the computer-implemented algorithm is based on a deep neural network. In particular, a deep-learning algorithm is provided. Reference objects can be easily and reliably identified using the computer-implemented algorithm. This enables not only a rapid and reliable determination of the orientation of the detection areas when the vehicle is stationary, but also, in a preferred application, a determination of the orientation of the detection areas when the vehicle is moving.

[0022] It is further proposed that the orientation of the detection range of at least one sensor of a rail-bound vehicle be determined relative to the aforementioned reference point. This can provide a high level of operational reliability for the rail-bound vehicle. For example, this makes it possible to determine the orientation of a detection range of multiple sensors of the rail-bound vehicle regularly and cost-effectively, without requiring the rail-bound vehicle to be placed in a predefined test environment.

[0023] The method according to the invention can be carried out by means of the vehicle according to the invention.

[0024] The vehicle according to the invention has at least one sensor. The at least one sensor is, in particular, a sensor of the type described above. Furthermore, the vehicle according to the invention has a data processing device configured to carry out the method according to the invention. Said data processing device can be, for example, a computer, a microcontroller, a processor, or another programmable hardware component. The data processing device is expediently configured to read, write, transmit, and / or manage data. This data processing device can be part of a decentralized data processing device of the vehicle, as is the case, for example, in the case of edge computing.Thus, several interconnected data processing devices of the vehicle can be jointly configured to carry out the method according to the invention. Furthermore, the data processing device can be used to determine the orientation of a detection area of ​​the at least one sensor relative to a predetermined reference point. This eliminates the need for a complex method for determining the orientation of the detection area using a prepared test environment. Instead, a simple and automated determination of the orientation of the detection area of ​​the at least one sensor relative to the predetermined reference point can be performed.

[0025] An advantageous development of the vehicle provides for it to have at least two sensors whose detection zones at least partially overlap. In the case of overlapping detection zones, a precise determination of the relative orientation of the individual detection zones is required. This allows objects to be identified, classified, and / or localized with high accuracy. Data fusion is preferably performed for this purpose. This also makes it possible to provide a large, contiguous detection zone.

[0026] The vehicle is designed as a rail-bound vehicle. This allows for a cost-effective determination of the alignment of detection zones of multiple sensors on the rail-bound vehicle using standardized parts of an infrastructure facility, such as a platform edge or platform markings. Furthermore, edges or corners of the rail-bound vehicle body can be used to determine the alignment of the detection zones of the rail-bound vehicle's sensors.

[0027] Particularly preferably, the at least one sensor is part of an optical camera of the vehicle. This eliminates the need to introduce test marks into the camera's field of view for the purpose of determining the orientation of the optical camera's detection range. Instead, vehicle-specific components or components of the aforementioned infrastructure facility can be used in a simple manner to determine the orientation of a field of view and thus of the optical camera's detection range relative to a predetermined reference point. This makes it possible to classify, localize, and / or identify objects with high accuracy using the optical camera. Reference objects can thus be easily detected and used to determine the orientation of corresponding detection ranges.

[0028] Furthermore, the invention provides a computer program which, when executed, causes the data processing device of the vehicle according to the invention to carry out the method according to the invention.

[0029] The invention also provides a computer-readable medium. This medium contains instructions that cause the data processing device of the vehicle according to the invention to carry out the method according to the invention. The aforementioned computer-readable medium can be, for example, a CD-ROM, a DVD, a USB or flash memory, or a non-physical medium, such as a data stream and / or a digital carrier signal.

[0030] The properties, features, and advantages of the invention described above, as well as the manner in which they are achieved, are explained in more detail in conjunction with the following description of the figures. Where appropriate, the same reference numerals are used in the figures for the same or corresponding elements of the invention. The description of the figures and their variations serve to illustrate the invention and do not limit the invention to the combinations of features specified therein, including with regard to functional features. Furthermore, all features specified in the description of the figures can be considered in isolation and combined as appropriate with the features of any claim.

[0031] They show: Fig. 1 shows an embodiment of a vehicle according to the invention in a schematic representation and an illustration of an example of the method according to the invention; Fig. 2 a further illustration of the example of the related Fig. 1 described method in the form of a schematic flow diagram.

[0032] Fig. Figure 1 shows a schematic representation of a rail-bound vehicle 14. This rail-bound vehicle 14 has several sensors 10, 12. Furthermore, Fig. 1 shows an example of a method 100 in which an orientation of a detection area 20 of the plurality of sensors 10, 12 is determined 108.

[0033] The example of the method 100 is illustrated here using an exemplary section of a rail-bound vehicle 14 located at a platform 26. For the sake of clarity, two optical cameras 10, 12 of the rail-bound vehicle 14 are provided as the plurality of sensors 10, 12. Alternatively or additionally, a one- or two-dimensional ultrasonic sensor array can be provided as the plurality of sensors 10, 12. Furthermore, the plurality of sensors 10, 12 can be provided as part of a radar device, a lidar device, an infrared camera, or a time-of-flight camera. This allows the example of the method 100 to be applied in a variety of ways. In particular, the method 100 is not limited to a predetermined sensor type. Rather, different sensor types can be aligned relative to one another, such as an optical camera relative to a lidar device.

[0034] The two optical cameras 10, 12 of the rail-bound vehicle 14 are, for example, spaced apart from each other at different end sections of the Fig. 1. The two optical cameras 10, 12 are arranged such that their detection areas 20 at least partially overlap. The platform 26 with an associated platform edge and a platform marking 18 is located in the common detection area 20 of the two optical cameras 10, 12. Such platform markings 18 generally serve as an orientation aid and are intended to enable barrier-free use of the platform 26. For this purpose, the platform markings 18 are already distinguished from surrounding areas of the platform 26 by an optical contrast. Furthermore, the vehicle doors 28 of the rail-bound vehicle 14 are located in the common detection area 20 of the two aforementioned optical cameras 10, 12. This makes it possible to optically monitor an intermediate space between the platform 26 and the vehicle 14 using the two aforementioned optical cameras 10, 12.

[0035] In order to be able to compare and jointly use the image data acquired by means of the two optical cameras 10, 12, it is necessary to determine 108 an alignment of a detection area 20 of each of the two optical cameras 10, 12 relative to a reference point 22 and relative to each other. The reference point 22 is, in the present case, for example, at a predetermined end section of the Fig. 1 shown carriage section of the rail-bound vehicle 14.

[0036] For the purpose of determining 108 the orientation of the detection range 20 of each of the two optical cameras 10, 12, a position of the first type of two reference objects 16, 18 relative to the predetermined reference point 22 is first determined 102. In the present exemplary embodiment, an edge of the vehicle door 28 is provided 106 as a first reference object 16, which is located in the common section of the detection range 20 of the two aforementioned optical cameras 10, 12. As a result, a part of the vehicle itself can be used as the reference object 16. In the present case, the platform marking 18 on the platform 26 is provided 106 as a further reference object 18. A usually standardized platform marking 18 can thus be used as a reliable reference object 18.In addition, a platform edge of the platform 26 and / or another edge or a corner of a part of the rail-bound vehicle 14 can be provided 106 as a reference object 16, 18. This makes it possible to dispense with test marks in a complex predefined test environment.

[0037] Furthermore, the example of the method 100 provides that by means of each of the two optical cameras 10, 12, a position of a second type of the reference objects 16, 18 is detected 104 relative to a respectively associated detection area 20. As a result, by means of the two optical cameras 10, 12, the two reference objects 16, 18 are each located within a respectively associated detection area 20. Based on a comparison of the determined 102 position of the first type and the detected 104 position of the second type, an orientation of a detection area 20 of each of the two optical cameras 10, 12 is then determined 108 independently of one another relative to the aforementioned reference point 22. The presently described example of the method 100 provides that the same reference objects 16, 18 are used for this purpose.In this way, an alignment of the detection areas 20 of the two optical cameras 10, 12 relative to one another is determined 108 by comparing the 104 second-type positions detected in relation to a respective detection area 20 assigned to the optical cameras 10, 12. This makes it possible to precisely determine and delimit the common detection area 20 of the two optical cameras 10, 12. This allows sensor data fusion to be performed, based on which objects can be classified, localized, and / or identified with high accuracy.

[0038] For the purpose of implementing the example of method 100, the rail-bound vehicle 14 comprises, for example, a data processing device 24. In this way, a determination 108 of the orientation of the detection areas 20 of the two aforementioned optical cameras 10, 12 relative to the aforementioned reference point 22 and relative to each other can be carried out in a simple, automated manner.

[0039] In a preferred embodiment, the orientation of the detection areas 20 of the two optical cameras 10, 12 is determined 108 using a computer-implemented algorithm. By way of example, the computer-implemented algorithm may be a machine learning algorithm or a deep learning algorithm. The computer-implemented algorithm is preferably based on artificial neural networks. This allows reference objects to be identified and located automatically. The method 100 for determining the orientation of the detection areas 20 can thereby be significantly accelerated and simplified.

[0040] Fig. 2 illustrates this in connection with Fig. 1 described example of the method 100 in the form of a schematic flow diagram.

[0041] Although the invention has been illustrated and described in detail by the preferred embodiment and its variations, the invention is not limited to the disclosed examples and other variations may be derived therefrom by those skilled in the art without departing from the scope of the invention.

[0042] Regardless of the grammatical gender of a particular term, persons with male, female or other gender identities are included.

Claims

[1] Procedure (100) in which - a position of a first type of reference object (16, 18) relative to a predetermined reference point (22) is determined (102); - by means of at least one sensor (10, 12) of a rail-bound vehicle (14), a position of a second type of the reference object (16, 18) within a detection range (20) of this at least one sensor (10, 12) is detected (104); - a platform edge or a platform marking which is designed for the purpose of operating said rail-bound vehicle (14) is provided as a reference object (16, 18) (106); - on the basis of a comparison of the determined (102) position of the first type and the detected (104) position of the second type, an orientation of the detection area (20) of the at least one sensor (10, 12) of the rail-bound vehicle (14) relative to the aforementioned reference point (22) is determined (108) and - an orientation of a detection area (20) of each of at least two sensors (10, 12) of the rail-bound vehicle (14) relative to the predetermined reference point (22) is determined (108) by means of the same reference object (16, 18). [2] Method (100) according to claim 1, wherein an ultrasonic sensor and / or an infrared sensor is provided as the at least two sensors (10, 12) or the at least two sensors (10, 12) are each provided as part of an optical camera, a radar device, a lidar device or a time-of-flight camera. [3] Method (100) according to one of the preceding claims, wherein the orientation of the detection area (20) of each of the at least two sensors (10, 12) relative to the reference point (22) is determined by means of a computer-implemented algorithm (108). [4] Method (100) according to claim 3, wherein a machine learning algorithm or a deep learning algorithm is provided as the computer-implemented algorithm. [5] Rail-bound vehicle (14) comprising: - at least two sensors (10, 12); - a data processing device (24) which is configured to carry out the method (100) according to one of the preceding claims and by means of which an orientation of a detection area (20) of each of the at least two sensors (10, 12) relative to a predetermined reference point (22) can be determined. [6] Rail-bound vehicle (14) according to claim 5, characterized by that the detection areas (20) of the at least two sensors (10, 12) at least partially overlap. [7] Rail-bound vehicle (14) according to one of claims 5 or 6, characterized bythat the at least two sensors (10, 12) are each part of an optical camera of the rail-bound vehicle (14). [8] Computer program which, when executed, causes the data processing device (24) of the rail-bound vehicle (14) according to one of claims 5 to 7 to carry out the method (100) according to one of claims 1 to 4. [9] Computer-readable medium comprising instructions which cause the data processing device (24) of the rail-bound vehicle (14) according to one of claims 5 to 7 to carry out the method (100) according to one of claims 1 to 4.

Images