Preceded by data processing on board a machine and using a database in which object definition data are stored, constituting reference points along a predetermined path of the machine.

Abstract

According to the invention, on board a machine (1) adapted to move along a predetermined path (21, 22), a database contains object definition data constituting reference points along the path and associated with its 3D geographic coordinates and an acquisition block (18) provides images of the scene in front of the machine (1), said acquisition block comprising a remote sensing block adapted to create an image from received echoes and to calculate, from said echoes, a direction and a distance, relative to the machine, of said reflecting object; these steps being repeated: acquire at least one image from the remote sensing block; identify in said image an object constituting a reference point (50) according to the object definition data constituting reference points in the database;Using the image from the remote sensing unit, determine a direction and distance, relative to the machine (1), of the object constituting an identified reference point; determine the current 3D location of the machine (1) as a function of said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting an identified reference point in the database. Figure for the abstract: Fig. 1;

Description

Title of the invention: Method for processing data on board a machine and using a database in which object definition data constituting reference points along a predetermined path of the machine are stored, associated data processing device and computer program. Technical field

[0001] The invention relates to a machine adapted to move along at least one predetermined path, for example a train, performing tasks autonomously, such as autonomous localization or obstacle detection. Previous technique

[0002] For a localization application, a discontinuous localization odometer is capable of autonomously localizing the train. Beacons and markers are used to perform discontinuous localization in order to correct the pedometer. However, the beacons and markers are physical elements that must be installed and maintained. GNSS signals cannot be used everywhere: GNSS signals are not received in tunnels, nor on subway lines, most of whose tracks are underground.

[0003] For an obstacle detection application, if there is uncertainty as to the location of the train, a structure, for example a station platform, located very close to the train's congestion contour may be mistakenly considered an obstacle.

[0004] A solution that overcomes these drawbacks is therefore necessary. Summary of the invention

[0005] The solution proposed by the present invention comprises, according to a first aspect, a data processing method implemented by an electronic data processing device on board a machine adapted to move along at least one predetermined path; said data processing device comprising: a database in which are recorded data defining objects constituting reference points along the predetermined path, the data defining each object constituting a reference point being associated with its 3D geographic coordinates; an acquisition block providing images of the scene in front of the machine, said acquisition block comprising at least one remote sensing block adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes from waves emitted from at least one reflective object, to create an image from the echoes received and to calculate, from said waves and echoes, a direction and a distance, relative to the machine, from said reflective object; said process includes the following iterative steps: a / acquire, via the acquisition block, a set of images of the current scene including at least one image from the remote sensing block; b / identify in said image an object constituting a reference point according to the definition data of objects constituting reference points in the database; c / using the image from the remote sensing unit, determine a direction and a distance, relative to the machine, of the object constituting an identified reference point; d / determine the current 3D location of the machine based on the direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database.

[0006] The invention uses existing physical elements or simple geometric figures along the predetermined path, commonly the railway track, and obtains an absolute location from these physical elements.

[0007] In certain embodiments, such a method will also include at least one of the following features:

[0008] - based on an approximate estimated location of the machine, the The following steps are completed in step b: determine, among the objects constituting reference points defined in the database, a subset of the objects constituting reference points located at most at a threshold distance from said location estimated approximately from the machine; the identification of an object constituting a reference point being accomplished by considering only the determined subset as including the object constituting a reference point to be identified, excluding the objects constituting reference points not included in the subset;

[0009] - the acquisition unit further comprises a camera and the image set of the The current scene includes an image from the camera and an image from the remote sensing unit; the identification of the object constituting a reference point is performed on the image from the camera; and a projection of the image from the remote sensing block into the video image reference system is carried out in order to make the representation of the object constituting an identified reference point coincide on the two images; the determination of the direction and distance, relative to the machine, of the object constituting an identified reference point is carried out on the basis of the representation made to coincide with the identified object on the image coming from the remote sensing block;

[0010] - the machine is a railway machine and said objects constituting points of landmarks include objects such as: 2D codes (QR codes), railway platforms, traffic signs, flood barriers, geometric figures, signals;

[0011] - the objects constituting reference points comprise objects each having a respective geometric shape associated, the database containing a correspondence between the geometric shape, comprising the following steps: identify successive objects with identified geometric shape along the path; deduce a serial code composed of successive numbers corresponding to successive objects with geometric shape; based on the deduced serial code, extract from the database the 3D geographic coordinates of at least one of the successive geometrically shaped objects associated with the deduced serial code; the determination of the machine's current 3D location is based on the determined direction and distance of at least one geometrically shaped object relative to the machine and the extracted 3D geographic coordinates;

[0012] - the process comprises the following steps: - calculate an estimated current location of the machine based on a 3D location determined in step d at a previous location and an estimate of a displacement of the machine between the previous location and the current location; - compare the current 3D location determined by step d with the current estimated location calculated; - evaluate whether the remote sensing unit is reliable based on said comparison in order to determine whether the remote sensing sensor is functioning correctly;

[0013] The object constituting an identified landmark is a traffic signal, and at least one of the following steps is implemented: - the identification of said traffic signal triggers the transmission of an instruction to an on-board signal reading block so that it reads the current aspect of the signal; - the database containing, in association with the traffic signal, the alternative aspects of the traffic signal, the current aspect of the signal detected by an on-board signal reading block is compared with said alternative aspects of the identified signal as recorded in the database; and the current detected aspect is eliminated based on a comparison of the current detected aspect with said possible aspects.

[0014] According to another aspect, the invention relates to a computer program adapted to be stored in the memory of an electronic data processing device and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, lead to the implementation of the steps of a process according to the first aspect of the invention.

[0015] According to another aspect, the invention relates to a data processing device intended to be placed on board a machine adapted to move along at least one predetermined path; said data processing device comprising: a database in which are recorded data defining objects constituting reference points along the predetermined path, the data defining each object constituting a reference point being associated with its 3D geographic coordinates; an acquisition block providing images of the scene in front of the machine, said acquisition block comprising at least one remote sensing block adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves from at least one reflective object, to create an image from the received echoes and to calculate, from said waves and echoes, a direction and a distance, relative to the machine, from said reflective object; in which: the acquisition block is adapted to acquire a set of images of the current scene including at least one image from the remote sensing block; the data processing device being adapted to identify on said image an object constituting a reference point according to data defining objects constituting reference points in the database and, using the image from the remote sensing block, to determine a direction and a distance, relative to the machine, of the object constituting an identified reference point; the data processing device being also capable of determining the current 3D location of the machine based on said direction and distance determined relative to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database.

[0016] In certain embodiments, such a device is adapted to, on the basis of an approximately estimated location of the machine and to identify on said image an object constituting a reference point according to data defining objects constituting reference points in the database, determine, among the objects constituting reference points defined in the database, a subset of the objects constituting reference points located at most at a threshold distance of said approximately estimated location of the machine; The identification of an object constituting a reference point is implemented by the data processing system by considering only the subset determined to include the object constituting a reference point to be identified, excluding objects constituting reference points not included in the subset. Brief description of the drawings:

[0017] The invention will be better understood and other features, details and advantages will become more apparent from the following description, which is not exhaustive, and from the accompanying figures, provided by way of example.

[0018] [Fig-1] The [Fig.1] is a schematic view of train 1 travelling on a railway track.

[0019] [Fig.2] Fig.2 is a schematic view of a processing device in a method of embodiment of the invention.

[0020] [Fig. 3] Fig. 3 represents the steps of a data processing procedure in an embodiment of the invention performing the train localization.

[0021] [Fig.4] Fig.4 illustrates the calculation of the train's location according to a method of realization of the invention.

[0022] [Fig. 5] Fig. 5 is an illustration of an object constituting a reference point of type 2D code used in an embodiment of the invention.

[0023] [Fig.6] Fig.6 is an illustration of an object constituting a reference point of type of railway platform used in one embodiment of the invention.

[0024] [Fig.7] Fig.7 is an illustration of an object constituting a reference point of flood barrier type used in one embodiment of the invention.

[0025] [Fig.8] Fig.8 is an illustration of information about an object constituting a point of a flood barrier type marker used in one embodiment of the invention.

[0026] [Fig.9] Fig.9 is an illustration of objects constituting reference points geometric shapes used in an embodiment of the invention.

[0027] [Fig. 10] The [Fig. 10] is an illustration of the detection of an object constituting a geometric reference point in an embodiment of the invention.

[0028] Identical references may be used in different figures when they indicate identical or comparable elements. Description of the implementation methods

[0029] The invention is described below with respect to an embodiment with a train 1. The train 1 is adapted to run on a railway network comprising a plurality of tracks. Figure 1 shows a schematic view of the train 1 running on a track comprising rails 21, 22. The train 1 is operated manually or autonomously.

[0030] Train 1 includes, for example, a locomotive facing the front part of the railway track that will be travelled by the train and pulling wagons of the train.

[0031] Fig. 2 represents an electronic processing device 10 on board train 1. The processing device 10 includes a database 11, a localization block 12 and an image acquisition block 18 including a lidar block 13.

[0032] The localization block 12 is adapted to determine the location of train 1 at each time interval of duration Tl (for example, Tl is in the range from 50 to 500 ms). The localization block 12 provides, for example, the absolute location of train 1 on Earth, for example its absolute 3D coordinates.

[0033] The lidar block (from the English "Laser Imaging Detection and Ranging") 13 is installed on the front of the train locomotive, and faces the part of the railway track in front of the train 1, as illustrated in [Fig.3].

[0034] The lidar block 13, as is known, comprises at least one laser source adapted to emit laser pulses and includes sensors. Emitted laser pulses, when they encounter objects, are reflected by these objects: some of these echoes are received by the lidar block 13 and captured by the sensors. Based on the received echoes measured by the sensors, the lidar block 13 is adapted to measure the travel time of the laser echoes and to calculate the distance from the source to each reflecting object and also the direction of this object relative to a reference axis, for example, the longitudinal axis of the train locomotive. A 3D image of the objects in the field of view of the lidar block 13 can thus be obtained over an interval T2 (for example, T2 is in the range of 50 to 500 ms).

[0035] The laser source is installed - and the lidar emission is adjusted - so that the lidar field of view covers all objects to be detected along the railway and which are in close proximity to the railway without intrusion.

[0036] The scanning speed of the lidar block 13 influences the number of points and echoes that are measured. The choice of optics and scanner has a significant influence on the resolution and range of the lidar system. The lidar range is the length of the area in front of the train that can be monitored by means of the laser waves.

[0037] In one embodiment, the database 11 contains data revealing the topology of the railway network, allowing the 3D coordinates of each point of the railway track to be known, or at least a precise approximation.

[0038] For example, each railway track being represented by the median axis between the straight and connecting rails of the railway track: - the median axis having been segmented into successive portions, each portion was modeled by a segment; the succession of segments is recorded in the database, for example by linking their identifier, shape and coordinates 3D geographic endpoints are known and recorded in database 11; and / or - the database 11 contains the 3D coordinates of a set of points along the median axis, the succession of these points being indicated in the database 11, for example by chaining their identifier, and the distance between each point and the next point being equal to or less than a predetermined distance d; for example, d is in the range from 0 to 5000 m; d is, for example, chosen so that it is equal to 1 meter (m); for example, this embodiment is considered below: [Fig.6] illustrates, in a given plane P perpendicular to a plane of the railway, the point 20 in the median axis between rail 21 and rail 22 of the railway.

[0039] Database 11 also contains data defining reference point objects that are present along the railway. These reference point objects are physical objects that are lidar-compatible, i.e., capable of reflecting incident lidar waves. Examples of reference point object types are: a signal element (with traffic lights), for example 50_2 with reference to [Fig. 1]; pylons, for example 50_1; kilometer markers such as 50_3, which indicate the number of kilometers separating the kilometer marker from a reference point; flood barriers; supports with 2D codes; station platforms; traffic signs, including mile markers and other indicators; and features with typical geometric shapes.

[0040] The database 11 contains, for each object constituting a defined reference point, its type, its identifier, its coordinates, other attributes such as reference point linking information, a direction in which the reference point can be detected, content that can be read at the reference point, etc.

[0041] In one embodiment, characteristic attributes of the objects constituting reference points are recorded: the 3D coordinates of the vertices of the objects in a global geographic reference system linked to the Earth (for example a latitude and a longitude), allowing the objects constituting reference points to be recognized, and therefore identified, according to the lidar echoes received.

[0042] Figure 3 represents the steps of a data processing method 100 in a embodiment of the invention performing train localization.

[0043] In a preliminary step 100_l, the definition data, including the 3D geographic coordinates of the characteristic point(s) of the objects constituting reference points 50 along the entire railway line (from the starting point to the final destination point) are determined and recorded in the database 11.

[0044] For example, the objects constituting reference points can be modeled by standard 3D geometric volumes and / or surfaces associated with parameters geometric including their characteristic dimensions and the coordinates (e.g. obtained via a GNSS receiver or using moving topography associated with an inertial measurement unit) of the characteristic vertices of the modeling shape.

[0045] Location

[0046] The exact location of the train is not known.

[0047] In the operating mode of the processing device 100, at step 100_2, at least one lidar image is acquired according to an interval T2.

[0048] During a step 100_3, after each interval T1, the localization block 12 analyzes an image from a set of images comprising at least the last acquired lidar image, said image being, in one embodiment, the last acquired lidar image, and determines one or more subsets of points in the image corresponding to one or more respective objects that are echo sources. Based on at least said subset of points on said image compared with information on objects constituting reference points (characteristic vertices, dimensions, etc.), the localization block 12 searches for the object constituting a reference point in the database 11, finds it, and determines all possible identifiers of this reference point represented on the image, given that the reference points can be almost identical. The train then moves to the next reference point.Based on the detection and distance to the first reference point, the reference point identifiers are further filtered. This process continues until a single reference point can be identified. The train's location is then determined based on this reference point. At locations where trains frequently perform cold starts, reference points with readable content, such as a 2D code, can be installed. In this case, the identifier of the first detected reference point can be read, and the location can be determined.

[0049] In one embodiment, based on the last location determined by repeating the previous process, and knowing a maximum / minimum speed (fixed or determined from an approximate estimated speed), a subset of the objects constituting reference points located at most / at least at a first threshold distance (equal to the maximum / minimum speed multiplied by Tl) from said last location of the train is extracted from the database 11, and the identification step is carried out by comparing the object constituting a reference point on the image only to said subset (and not to the objects constituting reference points not included in the subset) in order to identify said object constituting a reference point.

[0050] In another embodiment, the localization block 12 is adapted to perform, in parallel with the method 100, an approximate estimation of the current location of the train (for example by means of an odometer or an inertial measurement unit) and, knowing a maximum location error associated with this estimation, a subset of the objects constituting reference points located at most one second threshold distance from said last location of the train is extracted from database 11, and the identification step is carried out by comparing the object constituting a reference point on the image only to said subset (and not to objects constituting reference points not included in the subset) in order to identify said object constituting a reference point.

[0051] Once an object constituting a reference point has been identified on the lidar image, during a step 100_4, the localization block 12 determines, from the lidar information associated with this object constituting a reference point, the relative distance and direction of the identified object with respect to the train.

[0052] For example, with reference to [Fig.4], considering the object constituting a reference point identified as 50, the distance d and the angles a and [3 (with respect to the median axis between the rails 21, 22 of the railway) are determined.

[0053] Furthermore, the location block 12 extracts from the database 11 the 3D coordinates of the object constituting an identified reference point 50. With reference to [Fig. 4], these coordinates are (x, y, z). And, from these extracted coordinates combined with the relative distance and direction angles, the discrete and absolute location of the train, expressed in 3D coordinates, is calculated.

[0054] In one embodiment, the processing device 10 further comprises a video camera 17 placed on the front face of the train 1, for example with (approximately) the same median axis of field of view as the lidar 13. The video camera 17 is adapted to acquire periodic images of the scene in front of the train 1. It is easier to use a camera to distinguish the type of objects detected, by employing detection models already highly developed through artificial intelligence.

[0055] In such an embodiment, in step 100_2, the video camera acquires a video image every time interval of duration T3, which is added to the current set of images. And in step 100_3, for example, the identification of objects constituting reference points is performed on the video image. And, in order to ensure that the object constituting a reference point considered on the lidar image in step 100_4 is the same as the object constituting a reference point identified on the video image, a geometric transformation of the 3D lidar image of the image set is performed to produce a 2D video image; and a coincidence of image points of the object constituting an identified reference point is then performed, from the video image to the transformed 3D lidar image, this indicating the lidar points to be considered to obtain the distance and direction information.

[0056] In some embodiments, the 2D codes are framed with materials exhibiting high reflectivity to be better detected by the lidar.

[0057] Database 11 includes the following information for the object in the form of 2D code, with reference to [Fig.5]: - 2D code identifier, - 3D geographic coordinates of the central point of the 2D code, - linking information.

[0058] The 2D code itself contains the same information.

[0059] The lidar block 13 can identify a 2D code type object by detecting, on lidar images, the 4 vertices of the 2D code, and then determining the central point from said vertices using deterministic algorithms.

[0060] The content of the 2D code can be read by the camera block 17. By searching in the database 11, the identifier and location of the 2D code can be determined, so that the location of the train can be calculated from the absolute location of the 2D code and the distance from the lidar.

[0061] With reference to [Fig. 6], every station platform 80 is defined by the key vertices 1_1 and 2_1 (i.e., the upper end vertices along the railway track) and the corresponding railway track segment. Therefore, the information for objects constituting station platform type reference points stored in the database 11 includes, in one embodiment: - station platform identifier, - railway track segment identifier, - 3D geographic coordinates of vertex 1_1, - 3D geographic coordinates of vertex 2_1.

[0062] The station platforms can be detected by the lidar block 13 using deterministic algorithms, so that the two vertices can be identified at step 100_3. With the absolute location of the two vertices from the database 11, the location of the train can be calculated.

[0063] A signpost can be a kilometer marker, a speed limit sign, or any other sign along the railway line made of a highly reflective material. Database 11 includes the following information for objects constituting signpost-type landmarks: - signpost identifier, - 3D geographic coordinates of the signpost's center point.

[0064] Traffic signs can be detected by lidar using deterministic algorithms by detecting the vertices in order to identify the central point. The train's location can be calculated from the absolute location of the signal sign and the distance from the lidar.

[0065] Flood barriers are commonly used to prevent water from flowing into a station. Figure 7 represents a flood barrier, and Figure 8 illustrates the considered characteristics of the flood barrier in Figure 7, interpreted as a rectangle. The two uppermost vertices are defined as 1_2 and 2_2. The two sides of the flood barrier are considered as two reference points with different vertex coordinates, distinguished by the direction in which the flood barrier is detected.

[0066] The information in database 11 for flood barrier type objects includes: - Flood barrier identifier, - identifier of the railway track segment, - direction in which it is detected, - 3D geographic coordinates of vertex 1_2, - 3D geographic coordinates of vertex 2_2.

[0067] Flood barriers can be detected by lidar using deterministic algorithms, such that the two vertices 1_2, 2_2 can be identified. With the absolute location of the two vertices from database 11, the train's location can be calculated. With this information, the detected flood barrier will not be considered an obstacle intruding into the train's clearance contour. Flood barrier detection can also monitor the lidar's behavior.

[0068] Objects in the form of geometric figures are not natural reference points; they must be installed.

[0069] Objects in the form of rectangles and right isosceles triangles with different directions can be the objects in the form of geometric figures considered. Each figure can represent a number, from 0 to 4 when 5 geometric figures are used.

[0070] With reference to [Fig.9], figure 60_i, 1 = 0 to 4, corresponds to number i.

[0071] By detecting the geometric object by means of the lidar 13, the corresponding number is The localization block 12 then deduces the number from the database 11, associating it with the object in the form of a figure. A series of figures forms an identifier. In this case, the lidar image is therefore used to detect the identifier.

[0072] A geometric figure appears on the lidar point cloud image shown in [Fig. 10]. In one embodiment, the lidar point cloud is divided into X (here, X = 16) identical pieces, such as the hatched piece 70. Only the four pieces The four vertices of the detected shape are used for the reflected points. The direction of the triangle can be determined if only three vertices are detected based on the relative position of the three vertices, and therefore the corresponding number 0, 1, 2, 3, or 4. If all four pieces have reflected points, it can be determined that it is a rectangle.

[0073] The information in database 11 for geometric figures includes, for example, the number associated with each figure and the information in database 11 for a series of geometric figures includes: - series identifier, - 3D geographic coordinates of each object in the form of a figure in the series, - formed series code (different series have different series codes).

[0074] Signals and their appearance (green light in a first position of the light signal, red light in a second position of the light signal) can be detected by the camera 17. A signal is associated with any one of the reference points (with a special fixed distance vector between them, for example of 0 or different from 0) which can also be detected only by the lidar by detecting the reference point using deterministic algorithms.

[0075] The information for the signals in the database will be: - signal identifier, - identifier of the associated reference point (if applicable), the signal can be associated with a reference point that can be detected by the lidar using deterministic algorithms, to give an accurate distance to the signal, - spatial difference information (coordinates of the vector between the signal and the associated reference point), by detecting and measuring the distance of the reference point with the lidar, the distance to the signal can be determined, - identifier of the railway track segment, - signal location, to determine the train's location in order to perform a check with the location calculated from the lidar distance measurement, to improve the safety level - Possible aspect of the signal; the aspect of the signal is detected by artificial intelligence, and this information provides a scope for the detection result to improve the integrity of the artificial intelligence algorithm. - signal link information (to provide information about the next signals to be detected).

[0076] The set of steps 100_2 to 100_4 is repeated every time interval T.

[0077] In the embodiment considered, additional actions 100_5 are performed optionally. For example, based on the determined location or, based on an estimated train speed determined at least at the specified location, an emergency action is triggered, for example, emergency braking. In some embodiments, the additional actions 100_5 are related to obstacle detection and / or assistance with the signal reading function and / or monitoring of lidar behavior, as described below. Obstacle detection

[0078] In one embodiment, the processing device 10 includes an obstacle detection block 14 adapted to detect obstacles around the train 1 and to trigger, on the basis of such detection, emergency actions such as emergency braking.

[0079] Obstacle detection is carried out, for example, using lidar images and / or using video images.

[0080] In one embodiment, the invention includes comparing the obstacle estimation with the actual situation on site defined by the reference point information recorded in the database 11.

[0081] For example, in step 100_5, using the landmark information in database 11, obstacle detection eliminates, from among the detected obstacles, objects constituting landmarks located ahead that would have been erroneously detected as obstacles because they are very close to the train, or because of uncertainty regarding the train's location, or due to sensor errors. The steps would be, for example: - During the obstacle detection process, there is a slight error relative to the train's location and / or the sensor's orientation. The error increases over long distances. - A search is carried out in database 11 for the location of the obstacle in order to determine if there is a dock or flood barrier, or anything else defined in database 11 in the vicinity of the railway. - If this is not the case, the obstacle detection module knows that it is an obstacle and not just any object located near the railway but outside the train's congestion perimeter. - If so, this obstacle is marked as a potential false alarm to be dealt with later by the obstacle detection module, e.g. to be determined at a location closer to the obstacle. Thus, even if railway platforms or flood barriers are located very close to the railway line and can therefore easily be mistakenly detected as obstacles, they will not be considered as obstacles. Signal reading function assistance

[0082] A signal is also a reference point. In one embodiment, the processing device 10 includes a signal reading block 15 adapted to identify the signal provided by traffic signaling equipment such as 50_2, for example by determining the location and color of the light emitted by image processing on a video image provided by the camera block 17.

[0083] The signal reference point information in database 11, e.g., location, possible aspects, can reduce the computational load. For example, signal detection is only performed from a location within a certain distance of the signal (depending on the detection range, e.g., 200 m) since the signal location is known from database 11. This information also serves as a concordance check reference to improve integrity. Without this information, the algorithm in the signal reading block 15 would have to constantly search for signals to detect, which consumes considerable computing power. If the aspect of a signal is read, the computer vision must identify the signal's color (polarized light may be present, leading to an erroneous color reading).The signal's landmark information provides a reference (possible aspects) to verify the reading result; for example, if the signal's aspect is detected as one not included in the list of possible aspects, the detection is known to be erroneous.

[0084] At step 100_5, the steps completed are, for example: - combination of lidar and camera data, to identify, in the camera image, the signal that the lidar detects, - identification of the signal's aspect ratio from the camera image, - search in database 11 to determine if the signal aspect If identified, this aspect can be used; if not, the identification result is eliminated. Lidar behavior monitoring

[0085] In one embodiment, the processing device 10 includes a lidar monitoring block 16 adapted to monitor the behavior of the lidar block 13, by comparing the landmarks detected in the lidar images and the landmarks in the database 11.

[0086] At step 100_5, the steps completed are, for example: - search in database 11 for landmarks located in the forward field of vision, - using precise previous locations and odometry data, including odometry errors, to calculate the estimated current location of the train with a margin of error, - if the reference point in database 11 is correctly detected and the train location calculated from this reference point is within the location tolerance limits resulting from the odometry data, the lidar is considered to be functioning correctly, - otherwise, the lidar detection result is not reliable.

[0087] In some embodiments, the processing module 10 performs only one or more of the processes mentioned using reference points in the database 11, including localization, obstacle detection, assistance with the signal reading function, and monitoring of lidar behavior.

[0088] In one embodiment, the processing device 10 comprises a microprocessor and a memory comprising instructions which, when executed by the microprocessor, lead to the completion of one or more of the steps 100_1 to 100_5. Alternatively, at least some of the steps can be performed by specialized hardware, commonly a digital integrated circuit, either specific (ASIC) or based on programmable logic (e.g. FPGA).

[0089] Due to lidar performance limitations, the wider the field of view (FoV), the lower the point density. Consequently, it may not be possible to find a lidar that covers both distance and width. Therefore, multiple lidars can be used. In one embodiment, at least two lidars are used, instead of just one, for long-range (FoV1) and short-range (FoV2) respectively, in order to cover the total distance, as illustrated in [Fig. 4]. The advantages include the fact that both width and distance are covered and that the FoV overlap space can be detected with both / all lidars so as to have independent detection chains to facilitate the demonstration of the safety level.

[0090] Of course, many objects constituting reference points can be taken into account in the database 11, detected by lidar (and the camera when a camera 17 is also used).

[0091] The invention has been disclosed above with the use of a lidar block. Other technologies may be used instead of lidar, for example radar or sonar technology, or any suitable technology using the detection of echoes of waves generated on board the train.

[0092] The invention has been disclosed above in relation to a train, but it can be used more generally with any machine adapted to move along any trajectory among a set of known trajectories, such a machine being, for example, a metro, a tram, a boat, an airplane, a drone, with or without automatic and autonomous driving.

Claims

1. Demands Data processing method carried out by an electronic data processing device (10) on board a machine (1) adapted to move along at least one predetermined path (21, 22); said data processing device comprising: a database (11) in which are recorded data defining objects constituting reference points along the predetermined path, the definition data of each object constituting a reference point being associated with its 3D geographic coordinates; an acquisition block (18) providing images of the scene in front of the machine (1), said acquisition block comprising at least one remote sensing block (13) adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves from at least one reflective object, to create an image from the received echoes and to calculate, from said waves and said echoes, a direction and a distance, relative to the machine, from said reflective object; said process comprising the following repeated steps: - a / acquire a set of images of the current scene provided by the acquisition block (18), including at least one image from the remote sensing block (13); - b / identify in said image an object constituting a reference point (50) according to the definition data of objects constituting reference points in the database (il); - c / using the image from the remote sensing block (1 3), determine a direction and a distance, relative to the machine (1), of the object constituting an identified reference point; - d / determine the current 3D location of the machine (1) as a function of said direction and distance determined in relation to the machine and the 3D geographic coordinates associated with the object constituting a reference point identified in the database (11); according to which the method is characterized in that the objects constituting reference points (50) comprise objects each having a respective associated geometric shape, the database containing a correspondence between the geometric shape and a respective number, said method comprises the following steps: identifying successive objects with identified geometric shape along the path (21, 22); deduce a serial code composed of successive numbers corresponding to successive geometrically shaped objects; on the basis of the deduced serial code, extract from the database (11) the 3D geographic coordinates of at least one of the successive geometrically shaped objects associated with the deduced serial code; the determination of the current 3D location of the machine (1) is based on the direction and distance determined of at least one object with a geometric shape relative to the machine and the extracted 3D geographic coordinates.

2. A method according to claim 1, wherein, based on an approximately estimated location of the machine (1), the following steps are performed in step b: - determine, among the objects constituting reference points (50) defined in the database (11), a subset of the objects constituting reference points located at most at a threshold distance from said location estimated approximately from the machine (1); - the identification of an object constituting a reference point (50 ) being accomplished by considering only the determined subset as including the object constituting a reference point to be identified, excluding objects constituting reference points not included in the subset.

3. A method according to claim 1 or 2, wherein the acquisition block (18) further comprises a camera (17) and the image set of the current scene comprises an image from the camera (17) and an image from the remote sensing block (13), - the identification of the object constituting a reference point (50) is carried out on the image coming from the camera (17); and

4.

5.

6. - a projection of the image from the remote sensing block (13) into the reference system of the video image is accomplished in order to make the representation of the object constituting a reference point (50) identified on the two images coincide; - the determination of the direction and distance, with respect to the machine (1), of the object constituting an identified reference point (50) is accomplished on the basis of the representation put into coincidence of the identified object on the image coming from the remote sensing block (13). A method according to any one of the preceding claims, wherein the machine (1) is a railway machine and said objects constituting reference points (50) comprise objects including: 2D codes, station platforms, traffic signs, flood barriers, geometric figures, signals. A method according to any one of the preceding claims, comprising the following steps: - calculate an estimated current location of the machine (1) based on a 3D location determined in step d at a previous location and an estimate of a displacement of the machine between the previous location and the current location; - compare the current 3D location determined by step d with the current estimated location calculated; - evaluate whether the remote sensing block (13) is reliable on the basis of said comparison in order to determine whether the remote sensing sensor is functioning correctly. A method according to any one of the preceding claims, wherein the object constituting an identified reference point (50) is a traffic signal, and at least one of the following steps is performed: - the identification of said traffic signal triggers the transmission of an instruction to an on-board signal reading block (15) so that it reads the current aspect of the signal; - the database (11) containing, in association with the traffic signal, the alternative aspects of the traffic signal, the current aspect of the signal detected by an on-board signal reading unit (15) is compared with said alternative aspects of the identified signal as recorded in the database; and the currently detected aspect is eliminated based on a comparison of the currently detected aspect with said possible aspects.

7. A computer program adapted to be stored in the memory of an electronic data processing device (10) and further comprising a microcomputer, said computer program comprising instructions which, when executed on the microcomputer, result in the completion of the steps of a process according to any one of the preceding claims.

8. Data processing device (10) intended to be placed on board a machine (1) adapted to move along at least one predetermined path (21, 22); said data processing device comprising: a database (11) in which are recorded data defining objects constituting reference points along the predetermined path, the definition data of each object constituting a reference point being associated with its 3D geographic coordinates;an acquisition block (18) providing images of the scene in front of the machine (1), said acquisition block comprising at least one remote sensing block (13) adapted to emit waves in the direction of the path extending in front of the machine, to receive echoes of the emitted waves from at least one reflective object, to create an image from the received echoes and to calculate, from said waves and said echoes, a direction and a distance, relative to the machine, from said reflective object; in which: the acquisition block (18) is adapted to acquire a set of images of the current scene comprising at least one image from the remote sensing block (13);the data processing device being adapted to identify on said image an object constituting a reference point (50) according to data defining objects constituting reference points in the database (11) and, using the image from the remote sensing block (13), to determine a direction and a distance, relative to the machine (1), of the object constituting an identified reference point;

9. the data processing device being also capable of determining the current 3D location of the machine (1) as a function of said direction and distance determined in relation to the machine and of the 3D geographical coordinates associated with the object constituting a reference point identified in the database (11); said device is characterized in that, the objects constituting reference points (50) comprise objects each having a respective associated geometric shape, the database containing a correspondence between the geometric shape and a respective number, said device is adapted to identify successive objects with identified geometric shapes along the path (21, 22), to deduce a serial code composed of successive numbers corresponding to successive objects with geometric shapes, to, on the basis of the deduced serial code, extract from the database (11) the 3D geographic coordinates of at least one of the successive objects with geometric shapes associated with the deduced serial code; the determination of the current 3D location of the machine (1) being carried out as a function of the direction and distance determined of at least one object with geometric shape relative to the machine and the extracted 3D geographic coordinates. Data processing device (10) according to claim 8, proper, on the basis of an approximately estimated location of the machine (1) and to identify on said image an object constituting a reference point (50) according to definition data of objects constituting reference points in the database (11), to determine, among the objects constituting reference points (50) defined in the database (11), a subset of the objects constituting reference points located at most at a threshold distance of said approximately estimated location of the machine (1); the identification of an object constituting a reference point (50) being accomplished by the data processing device (10) by considering only the subset determined as including the object constituting a reference point to be identified, excluding objects constituting reference points not included in the subset.

Images