Method for data processing on machine using database storing definition data of landmark objects along predetermined path of machine, associated data processing device and computer program

Abstract

On a machine (1) adapted to move on a predetermined path (21, 22), a database stores definition data of landmark objects along the path and associated with its 3D geographic coordinates, and a capture block (18) provides an image of a scene in front of the machine (1), said capture block comprising a remote detection block adapted to create an image from received echoes, calculating the direction and the distance of the echo object relative to the machine according to the echo; wherein the steps are repeated: capturing at least one image from the remote detection block; identifying a landmark object (50) in the image according to landmark object definition data in a database; determining the direction and distance of the identified landmark object relative to the machine (1) using the image from the remote detection block; a current 3D position of the machine (1) is determined from the determined direction and distance relative to the machine and the 3D geographic coordinates associated with the identified landmark objects in the database.

Description

Technical Field

[0001] The invention belongs to the field of machines, such as trains, adapted to move on at least one predetermined path, which machines perform autonomous tasks such as autonomous localization or obstacle detection. Background Art

[0002] For positioning applications, odometers with discrete positioning enable autonomous train positioning. Tags or transponders are used to perform discrete positioning to correct the odometer. However, tags and transponders are physical items that need to be installed and maintained. GNSS signals are not available everywhere: there is no GNSS signal reception in tunnels or in subway lines where most of the tracks are underground.

[0003] For obstacle detection applications, when the train position is uncertain, facilities very close to the train clearance may be mistaken as obstacles, such as station platforms.

[0004] Therefore, a solution is needed to alleviate these disadvantages. Summary of the invention

[0005] To this end, according to a first aspect, the invention describes a data processing method implemented by an electronic data processing device on a machine adapted to move on at least one predetermined path; said data processing device comprising:

[0006] a database storing definition data of landmark objects along the predetermined path, the definition data of each landmark object being associated with its 3D geographic coordinates;

[0007] a capture block providing an image of a scene in front of the machine, the capture block comprising at least one remote detection block adapted to transmit waves towards a path in front of the machine, for receiving echoes of the transmitted waves from at least one echo object, for creating an image from the received echoes, and for calculating a direction and a distance of the echo object relative to the machine from the waves and the echoes;

[0008] The method comprises repeating the following steps:

[0009] a / capturing a set of images of the current scene by the capture block, including at least one image from the remote detection block;

[0010] b / identifying landmark objects in the image according to landmark object definition data in a database;

[0011] c / Using the image from the remote detection block, determine the direction and distance of the identified landmark object relative to the machine;

[0012] d / Determining the current 3D position of the machine based on said determined direction and distance relative to the machine and the 3D geographic coordinates associated with the identified landmark objects in the database.

[0013] The present invention uses physical items or simple geometric figures that exist along a predetermined path (usually a track) and obtains the absolute position from these physical items.

[0014] In an embodiment, the method will also include at least one of the following characteristics:

[0015] - Based on the rough estimated position of the machine, in step b, perform the following steps:

[0016] determining, among the landmark objects defined in the database, a subset of landmark objects that are at most a threshold distance from the rough estimated position of the machine;

[0017] performing recognition of the landmark object by excluding the landmark objects outside the subset and considering only the determined subset as including the landmark object to be recognized;

[0018] - the capture block also includes a camera, and the set of images of the current scene includes images from the camera and images from the remote detection block, recognition of landmark objects is performed in the images from the camera; and

[0019] performing a projection of the image from the long-range detection block into a reference of the video image in order to match the representation of the identified landmark object in both images;

[0020] enabling determination of a direction and distance of the identified landmark object relative to the machine based on matched representations of the identified object in the image from the remote detection block;

[0021] - the machine is a railway machine, and the landmark objects include objects among: QR code, station platform, sign board, flood gate, geometric figure, signal;

[0022] - the landmark objects include objects with respective associated geometric forms, wherein the database stores the correspondence between the geometric forms and the corresponding numbers, and the method comprises the following steps:

[0023] identifying successive identified geometric objects along the path;

[0024] inferring a sequence code consisting of consecutive numbers corresponding to the identified consecutive geometric shape objects;

[0025] The database stores the sequence code associated with the 3D geographic coordinates of at least one of the continuous geometric shape objects from which the sequence code is generated, and according to the inferred sequence code, extracts from the database the 3D geographic coordinates of at least one of the continuous geometric shape objects associated with the inferred sequence code;

[0026] The determination of the current 3D position of the machine is a function of the determined direction and distance of at least one geometric object relative to the machine and the extracted 3D geographic coordinates;

[0027] -The method comprises the following steps:

[0028] - calculating an estimated current position of the machine based on the 3D position determined in step d at the previous position and an estimate of the movement of the machine between the previous position and the current position;

[0029] - compare the current 3D position determined by step d with the calculated estimated current position;

[0030] - assessing whether the remote detection block is reliable based on the comparison to determine whether the remote detection sensor is functioning properly;

[0031] - the identified landmark object is a circulation signal, and at least one of the following steps is performed:

[0032] - the recognition of said running signal triggers a command to an onboard signal reading block for reading the current aspect of the signal;

[0033] - the database stores alternative appearances of the operating signal in association with the operating signal, the current appearance of the signal detected by the onboard signal reading block being compared with said alternative appearances of the identification signal such as those stored in the database; and the detected current appearance is discarded based on the comparison of the detected current appearance with said possible appearances.

[0034] According to another aspect, the invention describes a computer program suitable for being stored in a memory of an electronic data processing device, and also comprising a microcomputer, said computer program comprising instructions for implementing the steps of the method according to the first aspect of the invention when executed on the microcomputer.

[0035] According to another aspect, the invention describes a data processing device intended to be placed on a machine suitable for moving on at least one predetermined path; the data processing device comprising:

[0036] a database storing definition data of landmark objects along the predetermined path, the definition data of each landmark object being associated with its 3D geographic coordinates;

[0037] a capture block providing an image of a scene in front of the machine, the capture block comprising at least one remote detection block adapted to transmit waves towards a path in front of the machine, for receiving echoes of the transmitted waves from at least one echo object, for creating an image from the received echoes, and for calculating a direction and a distance of the echo object relative to the machine from the waves and the echoes;

[0038] in:

[0039] the capture block being adapted to capture a set of images of a current scene, the set of images including at least one image from the remote detection block; the data processing device being adapted to identify landmark objects in the images based on landmark object definition data in the database, and to determine the direction and distance of the identified landmark objects relative to the machine using the images from the remote detection block;

[0040] The data processing device is further adapted to determine a current 3D position of the machine based on said determined direction and distance relative to the machine and 3D geographical coordinates associated with identified landmark objects in the database.

[0041] In an embodiment, such an apparatus is adapted to identify landmark objects in said image based on a roughly estimated position of the machine and in accordance with landmark object definition data in a database to determine, among the landmark objects defined in the database, a subset of landmark objects that are at most a threshold distance from said roughly estimated position of the machine;

[0042] The identification of landmark objects by the data processing device is performed excluding landmark objects outside the subset and considering only the determined subset as including the landmark objects to be identified. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The invention will be better understood since the accompanying drawings are given by way of example and other characteristics, details and advantages will emerge better on reading the following description, which is not limiting.

[0044] [ Figure 1 ] Figure 1 is a schematic diagram of a train 1 running on a track;

[0045] [ Figure 2 ] Figure 2 is a schematic diagram of a processing device in an embodiment of the present invention;

[0046] [ Figure 3 ] Figure 3 The steps of the method for processing data for executing train positioning in an embodiment of the present invention are shown;

[0047] [ Figure 4 ] Figure 4The calculation of train positioning according to an embodiment of the present invention is shown;

[0048] [ Figure 5 ] Figure 5 is an illustration of a landmark object of a station platform type used in embodiments of the present invention;

[0049] [ Figure 6 ] Figure 6 is an illustration of a landmark object of a flood gate used in an embodiment of the present invention;

[0050] [ Figure 7 ] Figure 7 is a diagram of landmark object information for a flood gate used in an embodiment of the present invention;

[0051] [ Figure 8 ] Figure 8 is an illustration of a geometric landmark object used in embodiments of the present invention;

[0052] [ Fig. 9 ] Fig. 9 is a diagram of detection of a geometric landmark object in an embodiment of the present invention;

[0053] [ Fig.10 ] Fig.10 is a diagram illustrating detection of geometric landmark objects in an embodiment of the present invention.

[0054] The same reference numerals may be used in different figures when they represent the same or equivalent elements. DETAILED DESCRIPTION

[0055] The description of the invention is carried out below with reference to an embodiment having a train 1. The train 1 is adapted to run on a railway track network comprising a plurality of tracks. Figure 1 A schematic diagram showing a train 1 running on a track comprising rails 21, 22. The train 1 is driven manually or autonomously.

[0056] The train 1 comprises, for example, a locomotive facing a further part of the track to be taken by the train and pulling some of the vehicles of the train.

[0057] Figure 2 An electronic processing device 10 is shown on board the train 1. The processing device 10 comprises a database 11, a position block 12 and an image capture block 18 comprising a LIDAR block 13.

[0058] The position block 12 is adapted to determine the position of the train 1 in each time period T1 (eg T1 is in the range of 50 to 500 ms). The position block 12 provides, for example, the absolute position of the train 1 on the earth, such as its 3D absolute coordinates.

[0059] A LIDAR (“Laser Imaging Detection and Ranging”) block 13 is mounted on the front of the train locomotive, facing the track section ahead of the train 1, such as Figure 3 shown.

[0060] As is known, the LIDAR block 13 comprises at least one laser source adapted to emit laser pulses, and comprises a sensor. When encountering objects, the emitted laser pulses are reflected by these objects: some of these echoes are received back by the LiDAR block 13 and captured by the sensor. Based on the measurements of the received echoes by the sensor, the LiDAR block 13 is adapted to measure the travel time of the laser echoes, and to calculate the distance from the source to each reflecting object, and the direction of the object compared to a reference axis (e.g. the longitudinal axis of the train locomotive). Thus, a 3D picture of the objects in the field of view of the LIDAR block 13 can be obtained in a time period T2 (e.g. T2 is in the range of 50 to 500 ms).

[0061] The laser source is mounted and the emission of the LIDAR is tuned in such a way that the LIDAR field of view covers all objects to be detected along the track that are close to the track without any intrusion.

[0062] The scanning speed of the LIDAR block 13 affects the number of points and echoes measured. The choice of optics and scanner greatly affects the resolution and range of the LiDAR system. The range of the LIDAR is the length of the area in front of the train that can be monitored by the laser waves.

[0063] In one embodiment, the database 11 stores data disclosing the track network topology, making it possible to know the 3D coordinates of each point of the railway network track, or at least an accurate approximation.

[0064] For example, each track is represented by the right side of the track and the middle axis between the link rails:

[0065] - the intermediate axis has been segmented into successive parts, each part having been modeled by a segment; the successive segments are stored in a database, for example by identifiers linking them, the form and the 3D geographical coordinates of the endpoints being known and stored in the database 11; and / or - the database 11 stores the 3D coordinates of a set of points along the intermediate axis, the succession of these points being indicated in the database 11, for example by identifiers linking them, and the distance between each point and the next point being equal to or less than a predetermined distance d; for example d being in the range of 0 to 5000 m; d being for example chosen to be equal to 1 meter (m); for example, the embodiment is considered in the following: Figure 5 A given plane P, perpendicular to the plane of the tracks, a point 20 in the middle axis between track 21 and track 22 of the track is shown.

[0066] The database 11 also stores data defining landmark objects present along the track. These landmark objects are physical objects suitable for LIDAR, i.e. suitable for reflecting incident LIDAR waves. Examples of landmark object types are: signal elements (with traffic lights), e.g. 50_2, reference Figure 1 ; Pylons, such as 50_1; KM columns, such as 50_3, which indicate the kilometers from the KM column to the reference point; flood gates; supports with QR codes; station platforms; sign boards, including mileage boards and other signs; elements with typical geometric figures.

[0067] The database 11 stores, for each defined landmark object, its type, its ID, its coordinates, other attributes such as landmark link information, the direction in which the landmark can be detected, the content that can be read from the landmark, and the like.

[0068] In an embodiment, characteristic features of landmark objects are stored: 3D coordinates (eg, latitude, longitude) of object vertices in a geo-referenced globe linked to the earth, enabling landmark objects to be identified and therefore recognized from received LIDAR returns.

[0069] Figure 3 1 shows the steps of a method 100 for processing data to perform train positioning in an embodiment of the present invention.

[0070] In a preliminary step 100_1 , definition data are determined and stored in the database 11 , the definition data comprising the 3D geographical coordinates of characteristic points of the landmark object 50 along the entire trajectory (from the departure point to the final destination point).

[0071] For example, a landmark object may be modeled by a 3D standard geometric volume and / or surface associated with geometric parameters including its characteristic dimensions and coordinates of characteristic vertices in the modeled form (e.g., obtained via a GNSS receiver or by using a mobile topology associated with an inertial measurement unit).

[0072] position

[0073] The exact location of the train is unknown.

[0074] In the operating mode of the processing device 100 , in step 100_2 , at least a LIDAR image is captured in each period T2 .

[0075] In step 100_3, in each cycle T1, the positioning block 12 analyzes an image in the image set including at least the last captured LIDAR image, which in an embodiment is the last captured LIDAR image, and determines one or more subsets of points in the image corresponding to the corresponding one or more objects as the echo source. Based on at least the subset of points in the image compared with the landmark object information (feature vertices, dimensions), the positioning block 12 searches for the landmark object in the database 11, finds it and determines all possible IDs of the landmark represented in the image, because the landmarks can be almost the same and difficult to distinguish. Then, the train moves until the next landmark. Based on the detection and the distance to the first landmark, the ID of the landmark is further screened. This process continues until a unique landmark ID can be identified. The train position is then determined based on the landmark. At points where the train often starts cold, landmarks with readable content, such as QR codes, can be installed. In this case, the ID of the first detected landmark can be read and the position can be determined.

[0076] In an embodiment, based on the last position determined by repetition of the previous process and knowing the maximum / minimum speed (fixed or determined from a roughly estimated speed), a subset of landmark objects that are at most / at least a first threshold distance (equal to the maximum / minimum speed multiplied by T1) from the last position of the train is extracted from the database 11, and the identification step is performed by comparing the landmark objects in the image only with the subset (and not with landmark objects outside the subset) to identify the landmark objects.

[0077] In another embodiment, the positioning block 12 is adapted to perform a rough estimate of the current train position (for example, using an odometer or an inertial measurement unit) in parallel with the process 100, and knowing the maximum position error relative to this estimate, extracting from the database 11 a subset of landmark objects that are at most a second threshold distance from the last position of the train, and performing an identification step by comparing the landmark objects in the image only with the subset (and not with landmark objects outside the subset) in order to identify the landmark objects.

[0078] Once a landmark object is identified in the LIDAR image, in step 100_4, the location block 12 determines the relative distance and direction of the identified object relative to the train based on the LIDAR information associated with the landmark object.

[0079] For example, refer to Figure 4 , taking into account the identified landmark object 50 , the distance d as well as the angles α and β (with respect to the median axis between the trajectory tracks 21 , 22 ) are determined.

[0080] Furthermore, the location block 12 extracts the 3D coordinates of the identified landmark object 50 from the database 11. Figure 4, these coordinates are (x, y, z). And from these extracted coordinates combined with relative distance and direction angle, the discrete and absolute position of the train expressed in 3D coordinates is calculated.

[0081] In an embodiment, the processing device 10 further comprises a camera 17 located on the front side of the train 1, e.g. with (substantially) the same field of view central axis as the LIDAR 13. The camera 17 is adapted to capture periodic images of the scene in front of the train 1. By using an already well developed detection model via artificial intelligence, it is easier to use the camera to distinguish between the types of detected objects.

[0082] In such an embodiment, in step 100_2, the camera captures video images added to the current set of images at each time period T3. And in step 100_3, for example, recognition of landmark objects is performed into the video images. And in order to ensure that the landmark objects considered in the LIDAR images in step 100_4 are the same as the landmark objects recognized in the video images, a geometric transformation of the 3D LIDAR images of the set of images into 2D video images is performed; then a matching of the recognized landmark object image points is performed from the video images to the transformed 3D LIDAR images; thereby indicating which LIDAR points are to be considered to obtain distance and direction information.

[0083] In an embodiment, the QR code is constructed using a material with high reflectivity for better detection by LiDAR.

[0084] The database 11 includes the following information of the QR code object:

[0085] -QR code ID,

[0086] -The 3D geographic coordinates of the center point of the QR code

[0087] - Link information.

[0088] The QR code itself contains the same information.

[0089] The QR code object can be recognized by the LiDAR block 13 by detecting the LIDAR image of the four QR code vertices and then determining the center point from these vertices using a deterministic algorithm.

[0090] The content of the QR code can be read by the camera block 17. By looking it up in the database 11, the ID and position of the QR code can be determined, so that the train position can be calculated based on the absolute position of the QR code and the range of the LIDAR.

[0091] refer to Figure 5, any station platform 80 is defined as key vertices 1 and 2 (i.e., the top vertices along the track) and the corresponding track segments. Therefore, in one embodiment, the information of the landmark object of type station platform contained in the database 11 includes:

[0092] -Station platform ID,

[0093] - track segment ID,

[0094] - 3D geographic coordinates of vertex 1_1,

[0095] - 3D geographic coordinates of vertex 2_1.

[0096] The station platform can be detected by the LiDAR block 13 using a deterministic algorithm so that two vertices can be identified in step 100_3. Using the absolute positions of the two vertices from the database 11, the train position can be calculated.

[0097] The marker board can be a kilometer post speed limit board, or any other board along the track made of a highly reflective material. The database 11 includes the following information of the marker board type landmark object:

[0098] -Marker board ID

[0099] - 3D geographic coordinates of the center point of the marker board.

[0100] LiDAR can detect the marker plate using a deterministic algorithm by detecting the vertices to identify the center point. Based on the absolute position of the marker plate and the distance measurement from the LiDAR, the train position can be calculated.

[0101] Flood gates are commonly used to prevent water from flowing into stations. Figure 6 It means flood gate. Figure 7 Shows Figure 6 Considered properties of the floodgate, which is considered as a rectangle. The top two vertices are defined as 1_2 and 2_2. The two sides of the floodgate are considered as two landmarks with different coordinates of the vertices, distinguished by detecting the direction of the floodgate.

[0102] The information of flood control gate objects in the database 11 includes:

[0103] -Flood Gate ID

[0104] - Track segment ID

[0105] - The direction of the flood gate is detected,

[0106] - 3D geographic coordinates of vertex 1_2,

[0107] -3D geographic coordinates of vertex 2_2.

[0108] The flood gates can be detected by LiDAR using a deterministic algorithm so that the two vertices 1_2, 2_2 can be identified. Using the absolute positions of the two vertices from the database 11, the train position can be calculated. By knowing this information, the detected flood gates will not be considered as obstacles that intrude on the train clearance. The detection of flood gates can also monitor the LiDAR behavior.

[0109] Geometric objects are not natural landmarks, they should be installed additionally.

[0110] Objects in the form of rectangles and isosceles right triangles with different orientations may be considered geometric objects.

[0111] In the case of using 5 geometric figures, each figure can represent a number, for example 0 to 4.

[0112] refer to Figure 8 , Figure 60_i, i=0 to 4, corresponding to the number i.

[0113] Further to the geometric object detection performed by LIDAR 13, the localization block 12 infers the corresponding number (the correspondence between each geometric figure and its associated number has been previously stored in the database 11).

[0114] A series of patterns detected continuously by the positioning block 12 constitutes the ID.

[0115] The information in the database 11 for a series of geometric figures comprises an identifier of the series, associated with the 3D geographical coordinates of each object constituting the figures of the series (or at least the figures of an object, for example the figure corresponding to the last object encountered in a given direction).

[0116] This series of numbers constitutes a serial code. It can be considered as a 5-ary number, each object of which represents a number from 0 to 4 as a digit of the 5-ary code. If one object is used, it can represent 5 identifiers of length 1, and if, for example, a sequence of 10 objects is used, it can represent 9765625 (5^10) identifiers of length 10. These identifiers are indicated in sequence by the trajectory and position of the object in the database. Once a complete series is detected, an identifier is obtained, which is an index in the database, making it possible to find the geographical location of the object that generated the sequence. Therefore, based on this geographical location, the position of the train can be calculated.

[0117] For example, if a train continuously crosses its track:

[0118] - an object that hosts a geometry of type 60_1, then

[0119] - an object that hosts a geometry of type 60_3, then

[0120] - an object that hosts a geometry of type 60_1, then

[0121] - an object that hosts a geometry of type 60_1, then

[0122] - an object that hosts a geometry of type 60_3, then

[0123] - an object carrying a geometric figure of type 60_0, the location block 12 infers the identifier "131130", and then the location block 12 searches the database 11 for the geographical location information stored corresponding to the identifier: for example, the information includes the 3D coordinates of each of the objects encountered, or the 3D coordinates of an object (for example, the last object encountered: the object detected here carries the graphic 60_0).

[0124] In this case, LiDAR is therefore used to detect the type of graphic, from which the associated number can be inferred. For example:

[0125] like Fig. 9 As shown, the geometric figure appears on the LiDAR point cloud image. In one embodiment, the LiDAR point cloud is divided into X (here X=16) identical segments, such as the shadow segment 70. Regarding the reflection points, only four segments at the four vertices of the detected shape are used. If only three vertices are detected based on the relative positions of the three vertices and therefore based on the corresponding numbers 0, 1, 2, 3 or 4, the direction of the triangle can be determined. If all four segments have reflection points on them, it can be determined that it is a rectangle.

[0126] In various embodiments, for example Figure 8 The geometric objects of type 60_i represented in are arranged one after another along the track, or they can be grouped at one position of the track, as in Fig.10 As shown, in two examples of a series of geometric objects, both "encode" the identifier "131130" (if the convention is preserved in the case of grouped objects, reading from top to bottom starting from the left, and then reading from top to bottom on the right).

[0127] The signal and its appearance (green light in the first position of the lamp, red light in the second position of the lamp) can be detected by the camera 17. The signal is associated with any landmarks (having a fixed special distance vector from each other, such as 0 or different from 0), which can also be detected only by LiDAR by detecting the landmarks with a deterministic algorithm.

[0128] The information of the signal in the database should be:

[0129] - Signal ID,

[0130] - Associated landmark ID (if applicable), the signal can be associated with a landmark that can be detected by LiDAR using a deterministic algorithm to

[0131] Gives an accurate measurement of the signal.

[0132] - Spatial difference information (coordinates of the vector between the signal and the associated landmark), which can be determined by detecting and ranging the landmarks with LiDAR

[0133] The distance measurement of the fixed signal.

[0134] - track segment ID,

[0135] -Signal location, determine the train location, use LiDAR ranging to calculate the location for review, improve the safety level -Possible signal appearance, use artificial intelligence to detect signal appearance, and this information gives the range of detection results to improve artificial intelligence

[0136] The integrity of the algorithm.

[0137] - Signal link information (gives information about the next signal to be detected).

[0138] Every period T, the set of steps 100_2 to 100_4 is repeated.

[0139] In the embodiment under consideration, an additional action 100_5 is optionally performed. For example, based on the determined position or based on the train speed estimated at least based on the determined position, an emergency action, such as emergency braking, is triggered. In an embodiment, the additional action 100_5 involves obstacle detection and / or auxiliary signal reading functions and / or monitoring LIDAR behavior, as described below.

[0140] Obstacle Detection

[0141] In one embodiment, the processing device 10 comprises an obstacle detection block 14 adapted to detect obstacles around the train 1 and trigger emergency actions such as emergency braking based on such detection.

[0142] The obstacle detection is performed, for example, using LIDAR images and / or using video images.

[0143] In an embodiment, the invention provides for comparing the estimate of obstacles with a ground truth defined by landmark information stored in the database 11 .

[0144] For example, in step 100_5, using the landmark information in the database 11, the obstacle detection discards from the detected obstacles the forward landmark objects that would be wrongly detected as obstacles due to being very close to the train or due to train position uncertainty or due to sensor errors. The steps would be, for example:

[0145] - During obstacle detection, there are slight errors in the train position and / or sensor attitude. Errors are amplified over long distances.

[0146] - Look up in the database 11 at the location of the obstacle to determine if there is a platform or flood gate, or any other object defined in the database 11 close to the track.

[0147] - If no, then the obstacle detection module knows it is an obstacle and not anything close to the track but out of the train's clearance.

[0148] If yes, the obstacle is marked as a potential false alarm for further processing by the obstacle detection module, for example to make a determination at a closer position to the obstacle.

[0149] Therefore, station platforms or flood gates are not considered obstacles even if they are very close to the track and could therefore be easily falsely detected as obstacles.

[0150] Facilitates signal reading capabilities

[0151] In one embodiment, the processing device 10 comprises a signal reading block 15 adapted to determine the signal given by a traffic signal device such as 50_2, for example by determining the position and color of the emitted light determined by image processing on the video image provided by the camera block 17.

[0152] The landmark information (e.g., position, possible appearance) of the signal in the database 11 may provide the possibility of reducing the computational load, for example, because the signal position is known from the database 11, the detection of the signal is only performed from positions shorter than a certain distance to the signal (depending on the detection range, e.g., 200 m), and as a consistency check reference to improve integrity, whereas in the absence of any such information, the algorithm in the signal reading block 15 would need to look for the signal to detect all the time, which consumes a lot of computing power. If the signal appearance is read, the computer vision needs to recognize the color of the signal (there may be polarized light causing the color to be read incorrectly). The signal landmark information provides a reference (possible appearance) to check the reading result, for example, if the signal appearance is detected as an appearance that is not in the list of possible appearances, it is known that the detection is wrong.

[0153] In step 100_5, the steps implemented are, for example:

[0154] - Fusion of LiDAR data and camera data to identify LiDAR detected signals on camera images - Identification of signal appearance from camera images

[0155] - Look up in the database 11 whether the identified signal appearance is possible. If yes, then use this appearance, if not, then discard it.

[0156] Abandon the recognition result.

[0157] Monitoring LIDAR behavior

[0158] In one embodiment, the processing device 10 comprises a LIDAR monitoring block 16 adapted to monitor the behavior of the LIDAR block 13 and compare landmarks detected in the LIDAR images with landmarks in the database 11 .

[0159] In step 100_5, the steps implemented are, for example:

[0160] - Search database 11 for landmarks ahead in the field of view

[0161] - using the previous accurate position and the odometer including the odometer error to calculate the current estimated position of the train with a tolerant error - if a landmark in the database 11 is successfully detected and the position of the train calculated from the landmark is within the tolerant error from the mileage travelled

[0162] If the position is within tolerance, the LiDAR is considered to be working properly.

[0163] - If no, then the LiDAR detection results cannot be relied upon.

[0164] In an embodiment, the processing module 10 performs only one or more of the mentioned processes using the landmarks in the database 11 between positioning, obstacle detection, auxiliary signal reading functions and monitoring LIDAR behavior.

[0165] In an embodiment, the processing device 10 comprises a microprocessor and a memory comprising instructions which, when executed by the microprocessor, implement one or more of steps 100_1 to 100_5. Alternatively, at least some of the steps may be implemented by dedicated hardware (typically digital integrated circuits), dedicated (ASIC) or based on programmable logic (e.g., FPGA / field programmable gate array).

[0166] Due to the constraints of LiDAR performance, the wider the FoV (field of view), the lower the point density. Therefore, it may not be possible to find a single LiDAR to cover both the range and width. Therefore, multiple LiDARs can be used. In one embodiment, two or more LiDARs are used for long distance (field of view FoV1) and short distance (field of view FoV2) respectively, instead of only one LiDAR, to cover the entire range, such as Figure 4Benefits include both coverage width and range, and both / all LiDARs can be utilized to detect FoV overlapping spaces to provide independent detection chains to facilitate safe demonstrations.

[0167] Of course, many landmark objects detected by LiDAR (and camera if camera 17 is also used) may be considered in database 11 .

[0168] The invention has been disclosed above using a LIDAR block.Other technologies may be used instead of LIDAR, such as RADAR or SONAR technology, or any suitable technology that detects echoes of waves generated on the train.

[0169] The invention has been disclosed above in relation to trains, but it is more generally applicable with respect to any machine movable along any of a set of known tracks, such machines being therefore for example subways, railways, ships, airplanes, drones, with or without automatic and autonomous driving.

Claims

1. A data processing method implemented by an electronic data processing device (10) on a machine (1) adapted to move on at least one predetermined path (21, 22); The data processing device comprises: A database (11) storing definition data of landmark objects along the predetermined path, wherein the definition data of each landmark object is associated with the 3D geographic coordinates of each landmark object; a capture block (18) providing an image of a scene in front of the machine (1), the capture block comprising at least a remote detection block (13) adapted to transmit waves towards a path in front of the machine, for receiving echoes of the transmitted waves from at least one echo object, for creating an image from the received echoes, and for calculating the direction and distance of the echo object relative to the machine from the waves and the echoes; The method comprises the following steps which are repeated: -a / capturing a set of images of the current scene by the capture block (18), the set of images including at least one image from the remote detection block (13); -b / identifying landmark objects (50) in the image according to landmark object definition data in the database (11); -c / using said image from said remote detection block (13), determining the direction and distance of the identified landmark object relative to said machine (1); -d / Determine the current 3D position of the machine (1) based on the determined direction and distance relative to the machine and the 3D geographical coordinates associated with the identified landmark object in the database (11).

2. The method according to claim 1, wherein: Based on the roughly estimated position of the machine (1), the following steps are performed in step b: - among the landmark objects (50) defined in the database (11), determining a subset of the landmark objects that are at most a threshold distance from the roughly estimated position of the machine (1); - said identification of landmark objects (50) is performed excluding landmark objects outside the determined subset, only considering said subset as including said landmark objects to be identified.

3. The method according to claim 1 or 2, wherein: The capture block (18) further comprises a camera (17), and the set of images of the current scene comprises images from the camera (17) and images from the remote detection block (13), - performing said identification of said landmark object (50) in the image from said camera (17); as well as - performing a projection of the image coming from the remote detection block (13) into a reference of the video image in order to match the representation of the identified landmark objects (50) in the two images; - said determination of said direction and distance of said identified landmark object (50) relative to said machine (1) is effected based on matched representations of said identified objects in images from said remote detection block (13).

4. A method according to any of the preceding claims, wherein: The machine (1) is a railway machine, and the landmark objects (50) include objects among the following: QR code, station platform, sign board, flood gate, geometric figure, signal.

5. A method according to any of the preceding claims, wherein: The landmark object (50) comprises an object with respective associated geometric shapes, wherein the database stores the correspondence between the geometric shapes and the corresponding numbers, comprising the following steps: - identifying geometric objects along said path (21, 22); - inferring a sequence code consisting of consecutive numbers corresponding to the identified consecutive geometric objects; - the database (11) stores a sequence code associated with the 3D geographic coordinates of at least one of the continuous geometric shape objects from which the sequence code is generated, and extracts the 3D geographic coordinates of at least one of the continuous geometric shape objects associated with the inferred sequence code from the database (11) according to the inferred sequence code; - said determination of said current 3D position of said machine (1) is based on the determined direction and distance of said at least one geometrical object relative to said machine, and the extracted 3D geographical coordinates.

6. A method according to any of the preceding claims, comprising the steps of: - calculating an estimated current position of the machine (1) based on the 3D position determined in step d at a previous position and an estimate of the movement of the machine between the previous position and the current position; - comparing the current 3D position determined by step d with the calculated estimated current position; - Evaluating whether the remote detection block (13) is reliable based on the comparison to determine whether the remote detection sensor is functioning properly.

7. A method according to any of the preceding claims, wherein: The identified landmark object (50) is an operation signal, and at least one of the following steps is performed: - the recognition of said running signal triggers a command to the on-board signal reading block (15) for reading the current appearance of the signal; - the database (11) stores alternative appearances of the operating signal in association with the operating signal, the current appearance of the signal detected by the on-board signal reading block (15) being compared with the alternative appearances of the identified signal such as those stored in the database; and the detected current appearance is discarded based on the comparison of the detected current appearance with the possible appearances.

8. A computer program suitable for being stored in a memory of an electronic data processing device (10) and comprising a microcomputer, said computer program comprising instructions which, when executed on said microcomputer, implement the steps of the method according to one of the preceding claims.

9. A data processing device (10) intended to be placed on a machine (1) adapted to move on at least one predetermined path (21, 22); the data processing device comprising: A database (11) storing definition data of landmark objects along the predetermined path, wherein the definition data of each landmark object is associated with the 3D geographic coordinates of each landmark object; a capture block (18) providing an image of a scene in front of the machine (1), the capture block comprising at least a remote detection block (13) adapted to transmit waves towards a path in front of the machine, for receiving echoes of the transmitted waves from at least one echo object, for creating an image from the received echoes, and for calculating the direction and distance of the echo object relative to the machine from the waves and the echoes; in: A capture block (18) is adapted to capture a set of images of a current scene, said set of images comprising at least one image from said remote detection block (13); The data processing device is adapted to identify a landmark object (50) in the image based on landmark object definition data in the database (11), and to determine the direction and distance of the identified landmark object relative to the machine (1) using the image from the remote detection block (13); The data processing device is further adapted to determine the current 3D position of the machine (1) based on the determined direction and distance relative to the machine and the 3D geographical coordinates associated with the identified landmark objects in the database (11).

10. A data processing device (10) according to claim 9, adapted to identify landmark objects (50) in the image based on a roughly estimated position of the machine (1) and according to landmark object definition data in the database (11), to determine, among the landmark objects (50) defined in the database (11), a subset of the landmark objects which are at most a threshold distance from the roughly estimated position of the machine (1); The identification of landmark objects (50) is performed by the data processing device (10) excluding landmark objects outside the determined subset and considering only the subset as including the landmark objects to be identified.

Images