Preceded by data processing on board a machine and using a database in which object definition data are stored, constituting reference points along a predetermined path of the machine.

Abstract

According to the invention, on board a machine (1) adapted to move along a predetermined path (21, 22), a database contains object definition data constituting reference points along the path and associated with its 3D geographic coordinates and an acquisition block (18) provides images of the scene in front of the machine (1), said acquisition block comprising a remote sensing block adapted to create an image from received echoes and to calculate, from said echoes, a direction and a distance, relative to the machine, of said reflecting object; these steps being repeated: acquire at least one image from the remote sensing block; identify in said image an object constituting a reference point (50) according to the object definition data constituting reference points in the database;Using the image from the remote sensing unit, determine a direction and distance, relative to the machine (1), of the object constituting an identified reference point; determine the current 3D location of the machine (1) as a function of said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting an identified reference point in the database. Figure for the abstract: Fig. 1;

Description

Title of the invention: Method for processing data on board a machine and using a database in which are recorded data defining objects constituting reference points along a predetermined path of the machine, associated data processing device and computer program Technical field

[0001] The invention relates to a machine adapted to move on at least one predetermined path, for example a train, performing tasks autonomously, such as autonomous location or obstacle detection. Prior art

[0002] For a localization application, a discontinuous localization odometer is capable of performing autonomous localization of the train. Markers and beacons are used to perform the discontinuous localization in order to correct the odometer. However, the markers and beacons are physical elements that must be installed and maintained. GNSS signals cannot be used everywhere: GNSS signals are not received in tunnels, nor on metro lines whose tracks are mostly underground.

[0003] For an obstacle detection application, if there is uncertainty as to the location of the train, a structure, for example a station platform, located very close to the congestion contour of the train may be mistakenly considered as an obstacle.

[0004] A solution that overcomes these drawbacks is therefore necessary. Summary of the invention

[0005] The solution proposed by the present invention comprises, according to a first aspect, a data processing method implemented by an electronic data processing device on board a machine adapted to move on at least one predetermined path; said data processing device comprising: a database in which are recorded data defining objects constituting landmarks along the predetermined path, the data defining each object constituting a landmark being associated with its 3D geographical coordinates; an acquisition block providing images of the scene in front of the machine, said acquisition block comprising at least one remote sensing block adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves from at least one reflecting object, creating an image from the received echoes and calculating, from said waves and said echoes, a direction and a distance, relative to the machine, of said reflecting object; said method comprising the following iterated steps: a / acquire, by means of the acquisition block, a set of images of the current scene including at least one image coming from the remote sensing block; b / identify in said image an object constituting a landmark based on the data defining objects constituting landmarks in the database; c / using the image from the remote sensing unit, determine a direction and a distance, relative to the machine, of the object constituting an identified reference point; d / determining the current 3D location of the machine based on said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database.

[0006] The invention uses existing physical elements or simple geometric figures along the predetermined path, commonly the railway track, and obtains an absolute location from these physical elements.

[0007] In some embodiments, such a method will also include at least one of the following features:

[0008] - based on an approximately estimated location of the machine, the following steps are completed in step b: determine, among the objects constituting landmarks defined in the database, a subset of the objects constituting landmarks located at most at a threshold distance from said location estimated approximately from the machine; the identification of an object constituting a landmark being accomplished by considering only the subset determined as comprising the object constituting a landmark to be identified, excluding objects constituting landmarks not included in the subset;

[0009] - the acquisition block further comprises a camera and the set of images of the current scene includes an image from the camera and an image from the remote sensing block, the identification of the object constituting a landmark is carried out on the image from the camera; and a projection of the image coming from the remote sensing block into the reference system of the video image is carried out in order to bring into coincidence the representation of the object constituting a reference point identified on the two images; determining the direction and distance, relative to the machine, of the object constituting an identified landmark is carried out on the basis of the representation made in coincidence of the identified object on the image coming from the remote sensing block;

[0010] - the machine is a railway machine and said objects constituting points of landmarks include objects such as: 2D codes (QR codes), station platforms, traffic signs, flood barriers, geometric figures, signals;

[0011] - the objects constituting reference points comprise objects each having a respective geometric shape associated, the database containing a correspondence between the geometric shape, comprising the following steps: identify successive geometrically shaped objects along the path; deduce a serial code composed of the successive numbers corresponding to successive geometrically shaped objects; based on the inferred series code, extracting from the database the 3D geographic coordinates of at least one of the successive geometrically shaped objects associated with the inferred series code; determining the current 3D location of the machine based on the determined direction and distance of the at least one geometrically shaped object from the machine and the extracted 3D geographic coordinates;

[0012] - the method comprises the following steps: - calculating an estimated current location of the machine based on a 3D location determined in step d at a previous location and an estimate of a movement of the machine between the previous location and the current location; - compare the current 3D location determined by step d with the calculated estimated current location; - assess whether the remote sensing unit is reliable based on said comparison in order to determine whether the remote sensing sensor is functioning correctly;

[0013] the object constituting an identified reference point is a traffic signal, and at least one of the following steps is implemented: - the identification of said traffic signal triggers the transmission of an instruction to an on-board signal reading block so that it reads the current aspect of the signal; - the database containing, in association with the traffic signal, the alternative aspects of the traffic signal, the current aspect of the signal detected by an on-board signal reading block is compared with said alternative aspects of the identified signal as recorded in the database; and the current detected aspect is eliminated based on a comparison of the current detected aspect with said possible aspects.

[0014] According to another aspect, the invention relates to a computer program adapted to be recorded in the memory of an electronic data processing device and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, cause the steps of a method according to the first aspect of the invention to be implemented.

[0015] According to another aspect, the invention relates to a data processing device intended to be placed on board a machine adapted to move on at least one predetermined path; said data processing device comprising: a database in which definition data of objects constituting landmarks along the predetermined path are recorded, the definition data of each object constituting a landmark being associated with its 3D geographic coordinates; an acquisition block providing images of the scene in front of the machine, said acquisition block comprising at least one remote sensing block adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves coming from at least one reflective object, to create an image from the received echoes and to calculate, from said waves and said echoes, a direction and a distance, relative to the machine, of said reflective object; in which: the acquisition block is adapted to acquire a set of images of the current scene comprising at least one image coming from the remote sensing block; the data processing device being adapted to identify on said image an object constituting a reference point as a function of data defining objects constituting reference points in the database and, using the image coming from the remote sensing block, to determine a direction and a distance, relative to the machine, of the object constituting an identified reference point; the data processing device being further capable of determining the current 3D location of the machine as a function of said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database.

[0016] In certain embodiments, such a device is adapted to, on the basis of an approximately estimated location of the machine and to identify on said image an object constituting a landmark according to data defining objects constituting landmarks in the database, determine, among the objects constituting landmarks defined in the database, a subset of the objects constituting landmarks located at most at a threshold distance from said approximately estimated location of the machine; the identification of an object constituting a landmark being implemented by the data processing device considering only the subset determined as comprising the object constituting a reference point to be identified, excluding objects constituting reference points not included in the subset. Brief description of the drawings:

[0017] The invention will be better understood and other characteristics, details and advantages will appear more clearly on reading the following description, given in a non-limiting manner, and thanks to the attached figures, provided by way of example.

[0018] [Fig-1] [Fig.l] is a schematic view of train 1 running on a railway track.

[0019] [Fig.2] [Fig.2] is a schematic view of a processing device in a embodiment of the invention.

[0020] [Fig.3] [Fig.3] represents the steps of a data processing method in an embodiment of the invention accomplishing train localization.

[0021] [Fig.4] [Fig.4] illustrates the calculation of the train location according to a realization mode lization of the invention.

[0022] [Fig.5] [Fig.5] is an illustration of an object constituting a landmark of 2D code type used in one embodiment of the invention.

[0023] [Fig.6] [Fig.6] is an illustration of an object constituting a landmark of station platform type used in one embodiment of the invention.

[0024] [Fig.7] [Fig.7] is an illustration of an object constituting a landmark of flood barrier type used in one embodiment of the invention.

[0025] [Fig.8] [Fig.8] is an illustration of information of an object constituting a point flood barrier type marker used in one embodiment of the invention.

[0026] [Fig.9] [Fig.9] is an illustration of objects constituting geo landmarks metrics used in one embodiment of the invention.

[0027] [Fig. 10] [Fig. 10] is an illustration of the detection of an object constituting a geometric reference point in one embodiment of the invention.

[0028] Like references may be used in different figures when they indicate the same or comparable elements. Description of the embodiments

[0029] The description of the invention is given below in relation to an embodiment with a train 1. The train 1 is adapted to run on a railway network comprising a plurality of railway tracks. [Fig.l] represents a schematic view of the train 1 running on a railway track comprising the rails 21, 22. The train 1 is driven manually or autonomously.

[0030] Train 1 comprises, for example, a locomotive facing the front part of the railway track which will be traveled by the train and pulling wagons of the train.

[0031] [Fig.2] represents an electronic processing device 10 on board the train 1. The processing device 10 comprises a database 11, a location block 12 and an image acquisition block 18 comprising a lidar block 13.

[0032] The location block 12 is adapted to determine the location of the train 1 every time interval of duration Tl (for example Tl is in the range from 50 to 500 ms). The location block 12 provides, for example, the absolute location of the train 1 on Earth, for example its absolute 3D coordinates.

[0033] The lidar (Laser Imaging Detection and Ranging) unit 13 is installed on the front of the train's locomotive, and faces the portion of the railway track in front of the train 1, as illustrated in [Fig.3].

[0034] The lidar unit 13, as is known, comprises at least one laser source adapted to emit laser pulses and comprises sensors. Emitted laser pulses, when they encounter objects, are reflected by these objects: some of these echoes are received by the lidar unit 13 and picked up by the sensors. On the basis of the received echoes measured by the sensors, the lidar unit 13 is adapted to measure the travel time of the laser echoes and to calculate the distance from the source to each reflecting object and also the direction of this object relative to a reference axis, for example the longitudinal axis of the locomotive of the train. A 3D image of the objects in the field of view of the lidar unit 13 can thus be obtained according to an interval T2 (for example T2 is in the range from 50 to 500 ms).

[0035] The laser source is installed - and the lidar emission is adjusted - so that the lidar field of view covers all objects to be detected along the railway track and which are in the vicinity of the railway track without intrusion.

[0036] The scanning speed of the lidar unit 13 influences the number of points and echoes that are measured. The choice of optics and scanner has a strong influence on the resolution and range of the lidar system. The range of the lidar is the length of the area in front of the train that can be monitored by means of laser waves.

[0037] In one embodiment, the database 11 contains data revealing the topology of the railway network, making it possible to know the 3D coordinates of each point of the railway track or at least a precise approximation.

[0038] For example, each railway track being represented by the median axis between the straight and connecting rails of the railway track: - the median axis having been segmented into successive portions, each portion has been modeled by a segment; the succession of segments is recorded in the database, for example by chaining their identifier, the shape and the 3D geographic coordinates of the end are known and recorded in the database 11; and / or - database 11 contains the 3D coordinates of a set of points along of the median axis, the succession of these points being indicated in the database 11, for example by chaining their identifier, and the distance between each point and the following point being equal to or less than a predetermined distance d; for example, d is included in the range from 0 to 5000 m; d is, for example, chosen so that it is equal to 1 meter (m); for example, this embodiment is considered below: [Fig.6] illustrates, in a given plane P perpendicular to a plane of the railway, the point 20 in the median axis between the rail 21 and the rail 22 of the railway.

[0039] The database 11 also contains data defining landmark objects that are present along the railway. These landmark objects are physical objects that are suitable for lidar, i.e. suitable for reflecting incident lidar waves. Examples of types of landmark objects are: a signaling element (with traffic lights), for example 50_2 with reference to [Fig.l]; pylons, for example 50_l; kilometer signs such as 50_3, which indicate a number of kilometers separating the kilometer sign from a reference point; flood barriers; supports with 2D codes; station platforms; traffic signs, including mileage signs and other indicators; elements having typical geometric figures.

[0040] The database 11 contains, for each object constituting a defined landmark, its type, its identifier, its coordinates, other attributes such as landmark link information, a direction in which the landmark can be detected, content that can be read at the landmark, etc.

[0041] In one embodiment, characteristic attributes of the objects constituting landmarks are recorded: the 3D coordinates of the vertices of the objects in a global geographic reference system linked to the Earth (for example a latitude and a longitude), allowing the objects constituting landmarks to be recognized, and therefore identified, according to the lidar echoes received.

[0042] [Fig.3] represents the steps of a data processing method 100 in one embodiment of the invention accomplishing the localization of the train.

[0043] In a preliminary step 100_l, the definition data, including the 3D geographical coordinates of the characteristic point(s) of the objects constituting reference points 50 along the entire railway track (from the starting point to the final destination point) are determined and are recorded in the database 11.

[0044] For example, objects constituting landmarks can be modeled by standard 3D geometric volumes and / or surfaces associated with geometric parameters including their characteristic dimensions and the coordinates (for example obtained by means of a GNSS receiver or by using mobile topography associated with an inertial unit) of the characteristic vertices of the shape of modeling.

[0045] Location

[0046] The exact location of the train is not known.

[0047] In the operating mode of the processing device 100, in step 100_2, at least one lidar image is acquired according to an interval T2.

[0048] During a step 100_3, after each interval T1, the localization block 12 analyzes an image of a set of images comprising at least the last acquired lidar image, said image being, in one embodiment, the last acquired lidar image, and determines one or more subsets of points in the image corresponding to one or more respective objects which are echo sources. Based at least on said subset of points on said image compared with information of objects constituting landmarks (characteristic vertices, dimensions, etc.), the localization block 12 searches for the object constituting a landmark in the database 11, finds it and determines all possible identifiers of this landmark represented on the image, given that the landmarks can be almost identical. The train then moves to the next landmark.Based on the detection and distance to the first landmark, the landmark IDs are further filtered. This process continues until a single landmark can be identified. The train's location is then determined based on this landmark. At points where trains often make a cold start, landmarks with readable content can be installed, for example a 2D code. In this case, the ID of the first detected landmark can be read and the location can be determined.

[0049] In one embodiment, based on the last location determined with the repetition of the preceding method, and knowing a maximum / minimum speed (fixed or determined from an approximately estimated speed), a subset of the objects constituting landmarks located at most / at least at a first threshold distance (equal to the maximum / minimum speed multiplied by T1) from said last location of the train is extracted from the database 11, and the identification step is performed by comparing the object constituting a landmark on the image only to said subset (and not to the objects constituting landmarks not included in the subset) in order to identify said object constituting a landmark.

[0050] In another embodiment, the location block 12 is adapted to carry out, in parallel with the method 100, an approximate estimation of the current location of the train (for example by means of an odometer or an inertial unit) and, knowing a maximum location error associated with this estimation, a subset of the objects constituting reference points located at most a second threshold distance from said last train location is extracted from the database 11, and the identification step is carried out by comparing the object constituting a landmark on the image only to said subset (and not to the objects constituting landmarks not included in the subset) in order to identify said object constituting a landmark.

[0051] Once an object constituting a reference point has been identified on the lidar image, during a step 100_4, the location block 12 determines, from the lidar information associated with this object constituting a reference point, the relative distance and direction of the identified object with respect to the train.

[0052] For example, with reference to [Fig.4], considering the object constituting an identified reference point 50, the distance d and the angles a and [3 (relative to the median axis between the rails 21, 22 of the railway) are determined.

[0053] Furthermore, the location block 12 extracts from the database 11 the 3D coordinates of the object constituting an identified reference point 50. With reference to [Fig.4], these coordinates are (x, y, z). And, from these extracted coordinates combined with the distance and the relative direction angles, the discrete and absolute location of the train, expressed in 3D coordinates, is calculated.

[0054] In one embodiment, the processing device 10 further comprises a video camera 17 placed on the front face of the train 1, for example with (approximately) the same median axis of field of vision as the lidar 13. The video camera 17 is adapted to acquire periodic images of the scene in front of the train 1. It is easier to use a camera to distinguish the type of detected objects, by employing detection models already highly developed through artificial intelligence.

[0055] In such an embodiment, in step 100_2, the video camera acquires a video image every time interval of duration T3, which is added to the current set of images. And in step 100_3, for example, the identification of landmark objects is performed on the video image. And, in order to ensure that the landmark object considered on the lidar image in step 100_4 is the same as the landmark object identified on the video image, a geometric transformation of the 3D lidar image of the set of images is performed to produce a 2D video image; and a matching of image points of the identified landmark object is then performed, from the video image to the transformed 3D lidar image, this indicating the lidar points to be considered for obtaining the distance and direction information.

[0056] In some embodiments, the 2D codes are framed with materials having high reflectivity to be better detected by the lidar.

[0057] The database 11 includes the following information for the object in the form 2D code, with reference to [Fig.5]: - 2D code identifier, - 3D geographic coordinates of the central point of the 2D code, - link information.

[0058] The 2D code itself contains the same information.

[0059] The lidar block 13 can identify a 2D code type object by detecting, on the lidar images, the 4 vertices of the 2D code, then by determining the central point from said vertices using deterministic algorithms.

[0060] The content of the 2D code can be read by the camera block 17. By searching the database 11, the identifier and location of the 2D code can be determined, such that the location of the train can be calculated from the absolute location of the 2D code and the distance from the lidar.

[0061] With reference to [Fig.6], any station platform 80 is defined by the key vertices 1_1 and 2_1 (i.e., the upper end vertices along the railway track) and the corresponding railway track segment. Therefore, the information of the objects constituting landmarks of the station platform type recorded in the database 11 comprises, in one embodiment: - identifier of the station platform, - identifier of the railway track segment, - 3D geographical coordinates of the vertex 1_1, - 3D geographic coordinates of summit 2_1.

[0062] The station platforms can be detected by the lidar block 13 using deterministic algorithms, such that the two vertices can be identified in step 100_3. With the absolute location of the two vertices from the database 11, the location of the train can be calculated.

[0063] A traffic sign may be a kilometer sign, a speed limit sign or any other sign along the railway composed of a material with high reflectivity. The database 11 includes the following information for objects constituting landmarks of the traffic sign type: - identifier of the traffic sign, - 3D geographic coordinates of the central point of the traffic sign.

[0064] Traffic signs can be detected by lidar using deterministic algorithms by detecting vertices to identify the center point. The location of the train can be calculated from the absolute location of the traffic sign and the distance from the lidar.

[0065] Flood barriers are commonly used means to prevent water from flowing into a station. [Fig.7] shows a flood barrier and [Fig.8] illustrates the considered characteristics of the flood barrier. flood of [Fig.7] interpreted as a rectangle. The two upper extreme vertices are defined as 1_2 and 2_2. The two sides of the flood barrier are considered as two landmarks with different vertex coordinates, distinguished by the direction in which the flood barrier is detected.

[0066] The information in the database 11 for flood barrier type objects includes: flood barrier ID, railway segment ID, direction in which it is detected, 3D geographic coordinates of vertex 1_2, 3D geographic coordinates of vertex 2_2.

[0067] Flood barriers can be detected by the lidar using deterministic algorithms, so that the two vertices 1_2, 2_2 can be identified. With the absolute location of the two vertices from the database 11, the location of the train can be calculated. With this information, the detected flood barrier will not be considered as an obstacle intruding into the congestion contour of the train. Flood barrier detection can also monitor the behavior of the lidar.

[0068] Objects in the form of geometric figures are not natural landmarks, they must be installed.

[0069] Objects in the form of rectangles and right-angled isosceles triangles with different directions can be the objects in the form of geometric figures considered. Each figure can represent a number, from 0 to 4 when 5 geometric figures are used.

[0070] With reference to [Fig.9], figure 60_i, 1 = 0 to 4, corresponds to the number i.

[0071] By detecting the geometric object by means of the lidar 13, the corresponding number is then deduced by the localization block 12 from the database 11 associating the number with the object in the form of a figure. A series of the figures forms an identifier. In this case, the lidar image is therefore used to detect the identifier.

[0072] A geometric figure appears in the lidar point cloud image shown in [Fig. 10]. In one embodiment, the lidar point cloud is divided into X (here, X = 16) identical pieces, such as the hatched piece 70. Only the four pieces at the four vertices of the detected shape are used for the reflected points. The direction of the triangle can be determined if only three vertices are detected based on the relative position of the three vertices, and thus the corresponding number 0, 1, 2, 3, or 4. If all four pieces have reflected points, it can be determined to be a rectangle.

[0073] The information in the database 11 for the geometric figures includes, for example, the number associated with each figure and the information in the database 11 for a series of geometric figures includes: - series identifier, - 3D geographic coordinates of each object in the form of a figure in the series, - formed series code (different series have different series codes).

[0074] Signals and their appearance (green light in a first position of the light signal, red light in a second position of the light signal) can be detected by the camera 17. A signal is associated with any of the landmarks (with a fixed special distance vector between them, e.g. 0 or different from 0) which can also be detected only by the lidar by detecting the landmark using deterministic algorithms.

[0075] The information for the signals in the database will be: - signal identifier, - associated landmark identifier (if applicable), the signal can be associated with a landmark that can be detected by the lidar using deterministic algorithms, to give an accurate distance to the signal, - spatial difference information (vector coordinates between the signal and the associated landmark), by detecting and measuring the distance of the landmark with the lidar, the signal distance can be determined, - identifier of the railway segment, - signal location, to determine the location of the train in order to perform a check with the location calculated from the lidar distance measurement, to improve the safety level - possible aspect of the signal, the aspect of the signal is detected by artificial intelligence, and this information gives a scope of the detection result to improve the integrity of the artificial intelligence algorithm, - signal binding information (to provide information regarding the next signals to be detected).

[0076] The set of steps 100_2 to 100_4 is repeated every time interval T.

[0077] In the embodiment under consideration, additional actions 100_5 are optionally performed. For example, based on the determined location or based on a train speed estimated at least on the basis of the determined location, an emergency action is triggered, for example emergency braking. In some embodiments, the additional actions 100_5 are related to obstacle detection and / or assistance with the signal reading function and / or monitoring the behavior of the lidar, as described below. Obstacle detection

[0078] In one embodiment, the processing device 10 comprises an obstacle detection block 14 adapted to detect obstacles around the train 1 and to trigger, on the basis of such detection, emergency actions such as emergency braking.

[0079] Obstacle detection is performed, for example, using lidar images and / or using video images.

[0080] In one embodiment, the invention comprises comparing the obstacle estimate with the actual on-site situation defined by the landmark information recorded in the database 11.

[0081] For example, in step 100_5, using the landmark information in the database 11, obstacle detection eliminates, from the detected obstacles, landmark objects located ahead that would have been erroneously detected as obstacles due to being very close to the train or due to uncertainty about the train's location or due to sensor errors. The steps would be, for example: - During the obstacle detection process, there is a slight error in the train location and / or sensor orientation. The error increases at long distances. - A search is performed in database 11 for the location of the obstacle to determine if there is a wharf or flood barrier, or anything else defined in database 11 in the vicinity of the railway. - If this is not the case, the obstacle detection module knows that it is an obstacle and not any object located near the railway track but outside the train's congestion contour. - If so, this obstacle is marked as a potential false alarm to be processed further by the obstacle detection module, e.g. to be determined at a location closer to the obstacle. Thus, even if station platforms or flood barriers are located very close to the railway line and can therefore easily be detected, erroneously, as obstacles, they will not be considered as obstacles. Signal reading function support

[0082] A signal is also a landmark. In one embodiment, the processing device 10 comprises a signal reading block 15 adapted to identify the signal provided by traffic signaling equipment such as 50_2, for example by determining the location and color of the light emitted by image processing on a video image provided by the camera block 17.

[0083] The signal landmark information in the database 11, e.g., location, possible aspects, can reduce the computational load, e.g., signal detection is only performed from a location within a certain distance of the signal (depending on the detection range, e.g., 200 m) since the signal location is known from the database 11, and as a cross-check reference to improve integrity, whereas, without this information, the algorithm in the signal reading block 15 would have to constantly search for signals to detect, which consumes considerable computing power. If the aspect of a signal is read, the computer vision must identify the color of the signal (there may be polarized light resulting in a misreading of the color).The signal landmark information provides a reference (possible aspects) to check the reading result, for example if the signal aspect is detected as an aspect not included in the list of possible aspects, it is known that the detection is wrong.

[0084] In step 100_5, the steps performed are, for example: - combination of lidar data and camera data, to identify, on the camera image, the signal that the lidar detects, - identification of the signal appearance from the camera image, - searches database 11 to determine if the signal aspect identified is possible, if so, this aspect can be used and, if not, the identification result is discarded. Monitoring lidar behavior

[0085] In one embodiment, the processing device 10 comprises a lidar monitoring block 16 adapted to monitor the behavior of the lidar block 13, by comparing the landmarks detected in the lidar images and the landmarks in the database 11.

[0086] In step 100_5, the steps performed are, for example: - search in database 11 of landmarks located in the visual field ahead, - use of accurate previous locations and odometry data, including odometry errors, to calculate the estimated current location of the train within a margin of error, - if the landmark in the database 11 is correctly detected and the location of the train calculated from this landmark is within the location tolerance limits resulting from the odometry data, it is considered that the lidar is working correctly, - otherwise, the lidar detection result is not reliable.

[0087] In some embodiments, the processing module 10 only performs one or more of the methods mentioned using landmarks in the database 11 among localization, obstacle detection, assistance with the signal reading function and monitoring of the behavior of the lidar.

[0088] In one embodiment, the processing device 10 comprises a microprocessor and a memory comprising instructions which, when executed by the microprocessor, cause one or more of the steps 100_1 to 100_5 to be performed. Alternatively, at least some of the steps may be performed by specialized hardware, commonly a digital integrated circuit, either specific (ASIC) or based on programmable logic (e.g., FPGA).

[0089] Due to performance restrictions of lidars, the wider the field of view (FoV), the lower the point density. Therefore, it may not be possible to find a lidar that covers both distance and width. Therefore, multiple lidars can be used. In one embodiment, at least two lidars are used, instead of one, respectively for long distance (FoV1 field of view) and short distance (FoV2 field of view) to cover the total distance, as illustrated in [Fig. 4]. Advantages include that both width and distance are covered and that the FoV overlap space can be sensed with both / all lidars so as to have independent sensing chains to facilitate the demonstration of the safety level.

[0090] Of course, many objects constituting landmarks can be taken into account in the database 11, detected by lidar (and the camera when a camera 17 is also used).

[0091] The invention has been disclosed above with the use of a lidar unit. Other technologies may be used instead of lidar, for example radar or sonar technology, or any suitable technology using the detection of echoes of waves generated on board the train.

[0092] The invention has been disclosed above in relation to a train, but it can be used more generally with any machine adapted to move along any trajectory from among a set of known trajectories, such a machine being, for example, a metro, a tram, a boat, an airplane, a drone, with or without automatic and autonomous driving.

Claims

Claims

1. A method of data processing performed by an electronic data processing device (10) on board a machine (1) adapted to move along at least one predetermined path (21, 22); said data processing device comprising: a database (11) in which are recorded data defining objects constituting landmarks along the predetermined path, the data defining each object constituting a landmark being associated with its 3D geographical coordinates;an acquisition block (18) providing images of the scene in front of the machine (1), said acquisition block comprising at least one remote sensing block (13) adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves coming from at least one reflecting object, to create an image from the received echoes and to calculate, from said waves and said echoes, a direction and a distance, relative to the machine, of said reflecting object; said method comprising the following repeated steps: - a / acquire, by means of the acquisition block (18), a set of images of the current scene comprising at least one image coming from the remote sensing block (13); - b / identifying in said image an object constituting a reference point (50) as a function of the data defining objects constituting reference points in the database (11); - c / using the image from the remote sensing unit (13), determine a direction and a distance, relative to the machine (1), of the object constituting an identified reference point; - d / determining the current 3D location of the machine (1) based on said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database (11).

2. A method according to claim 1, wherein, based on an approximately estimated location of the machine (1), the following steps are performed in step b: - determining, from among the objects constituting landmarks (50) defined in the database (11), a subset of the objects constituting landmarks located at most at a threshold distance from said location estimated approximately from the machine (1); - the identification of an object constituting a landmark (50) being performed by considering only the subset determined as comprising the object constituting a landmark to be identified, excluding the objects constituting landmarks not included in the subset.

3. A method according to claim 1 or 2, wherein the acquisition unit (18) further comprises a camera (17) and the set of images of the current scene comprises an image from the camera (17) and an image from the remote sensing unit (13), - identification of the landmark object (50) is performed on the image from the camera (17); and - projection of the image from the remote sensing unit (13) into the reference system of the video image is performed in order to bring into coincidence the representation of the landmark object (50) identified on the two images; - determination of the direction and distance, relative to the machine (1), of the identified landmark object (50) is performed on the basis of the brought into coincidence representation of the identified object on the image from the remote sensing unit (13).

4. A method according to any preceding claim, wherein the machine (1) is a railway machine and said objects constituting landmarks (50) comprise objects from among: 2D codes, station platforms, traffic signs, flood barriers, geometric figures, signals.

5. A method according to any preceding claim, wherein the objects constituting landmarks (50) comprise objects each having a respective associated geometric shape, the database containing a correspondence between the geometric shape, comprising the following steps: - identifying successive identified geometric shape objects along the path (21, 22); - deducing a serial code composed of the successive numbers corresponding to the successive geometric shape objects; - based on the deduced serial code, extracting from the database (11) the 3D geographic coordinates of at least one of the successive geometric shape objects associated with the deduced serial code; - determining the current 3D location of the machine (1) based on the determined direction and distance of the at least one geometric shape object from the machine and the extracted 3D geographic coordinates.

6. A method according to any preceding claim, comprising the following steps: - calculating an estimated current location of the machine (1) based on a 3D location determined in step d at a previous location and an estimation of a movement of the machine between the previous location and the current location; - comparing the current 3D location determined by step d with the calculated estimated current location; - evaluating whether the remote sensing block (13) is reliable based on said comparison to determine whether the remote sensing sensor is functioning properly.

7. A method according to any preceding claim, wherein the identified landmark object (50) is a traffic signal, and at least one of the following steps is performed: - the identification of said traffic signal triggers the transmission of an instruction to an on-board signal reading block (15) to read the current aspect of the signal; - the database (11) containing, in association with the traffic signal, alternative aspects of the traffic signal, the current aspect of the signal detected by an on-board signal reading block (15) is compared with said alternative aspects of the identified signal as as recorded in the database; and the current detected aspect is eliminated based on a comparison of the current detected aspect with said possible aspects.

8. A computer program adapted to be recorded in the memory of an electronic data processing device (10) and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, cause the steps of a method according to one of the preceding claims to be carried out.

9. Data processing device (10) intended to be placed on board a machine (1) adapted to move on at least one predetermined path (21, 22); said data processing device comprising: a database (11) in which are recorded data defining objects constituting landmarks along the predetermined path, the data defining each object constituting a landmark being associated with its 3D geographical coordinates;an acquisition block (18) providing images of the scene in front of the machine (1), said acquisition block comprising at least one remote sensing block (13) adapted to emit waves towards the path extending in front of the machine, to receive echoes of the emitted waves coming from at least one reflective object, to create an image from the received echoes and to calculate, from said waves and said echoes, a direction and a distance, relative to the machine, of said reflective object; wherein: the acquisition block (18) is adapted to acquire a set of images of the current scene comprising at least one image coming from the remote sensing block (13);the data processing device being adapted to identify on said image an object constituting a landmark (50) as a function of data defining objects constituting landmarks in the database (11) and, using the image coming from the remote sensing block (13), to determine a direction and a distance, relative to the machine (1), of the object constituting an identified landmark; the data processing device being further suitable for determining the current 3D location of the machine (1) as a function of said direction and distance determined relative to the machine and of; 3D geographic coordinates associated with the object constituting a landmark identified in the database (11).

10. A data processing device (10) according to claim 9, capable, on the basis of an approximately estimated location of the machine (1) and for identifying on said image an object constituting a landmark (50) according to data defining objects constituting landmarks in the database (11), of determining, from among the objects constituting landmarks (50) defined in the database (11), a subset of the objects constituting landmarks located at most at a threshold distance from said approximately estimated location of the machine (1); the identification of an object constituting a landmark (50) being accomplished by the data processing device (10) by considering only the determined subset as comprising the object constituting a landmark to be identified, excluding the objects constituting landmarks not included in the subset.

Images