Method for detecting obstacles on a track, computer program and associated obstacle detection device

Abstract

The invention relates to a method (2) for detecting obstacles on a track, the method being implemented by computer and comprising, for at least one current image representative of the track, the steps: implementation (32) of an open object detection model to: detect the presence or absence of an object in a region of interest of the current image; and, if so, assign the detected object to one of: a class of known non-critical objects, a class of known critical objects, and a class of unknown objects; if a class assigned to the object detected by the open object detection model is a class of known critical objects or a class of unknown objects, generation (36) of an alert signal indicating the presence of an obstacle on the track. Figure 1

Description

Title of the invention: Method for detecting obstacles on a track, computer program and associated obstacle detection device. Technical field

[0001] The present invention relates to a method for detecting obstacles on a track.

[0002] The invention also relates to a computer program and a device implementing such a method.

[0003] The invention applies to the field of automatic vehicle driving, in particular the automatic driving of a railway vehicle. State of the art

[0004] With a view to automating driving, in particular driving a railway vehicle, the safety question arises of managing obstacles that may be on the vehicle's path.

[0005] To answer such a question, it has been proposed to use an automatic obstacle detection method, in which images of the track are provided as input to a classification model trained to recognize obstacles.

[0006] However, such a method does not give complete satisfaction.

[0007] Indeed, obstacle detection is a function that operates in an open environment, as the track is not separated from the rest of the world by a constant fence or a permanent tunnel (as in the case of a subway). Consequently, the classes of obstacles that the vehicle may encounter are unpredictable, and the number of potential obstacles is infinite. Therefore, exhaustive training of the classification model is not feasible, and safe operation cannot be guaranteed in this way.

[0008] Furthermore, an artificial intelligence model, such as a classification model, is a complex mathematical function that is generally unintelligible to a human. As a result, a decision made by the classification model cannot be interpreted, which is detrimental to safe operation, particularly when a decision made by the classification model is inconsistent with the actual situation.

[0009] One object of the present invention is to remedy at least one of the drawbacks of the prior art.

[0010] Another object of the invention is to provide a method for detecting obstacles on a track which is able to detect any type of obstacle likely to present a danger.

[0011] Another object of the invention is to propose an obstacle detection method capable of producing indicative information of the decision-making process implemented. Description of the invention

[0012] To this end, the invention relates to a method of the aforementioned type, the method being implemented by computer and comprising, for at least one current image representative of the track, the following steps: • Implementation of an open object detection model for: • detect the presence or absence of an object in a region of interest in the current image; and • if applicable, assign the detected object to one of the following: a class of known non-critical objects, a class of known critical objects and a class of unknown objects; • if a class assigned to the object detected by the open object detection model is a class of known critical objects or a class of unknown objects, generation of an alert signal indicating the presence of an obstacle on the track.

[0013] Indeed, thanks to the implementation of the open object detection model, rapid detection of any object present on the track is achieved.

[0014] Furthermore, thanks to the implementation of such a model, a class for the detected object is immediately obtained: it is then either a known object class (non-critical or critical), or a class grouping unknown objects, considered, by default, as critical. In other words, the fact that the detected object does not belong to one of the known non-critical object classes creates a suspicion of criticality, generating an alert signal calling for further processing, such as stopping the vehicle.

[0015] Thus, exhaustive knowledge of every object that may be on the track is not required, and safe operation is possible.

[0016] Advantageously, the process according to the invention has one or more of the following characteristics, taken individually or in any technically feasible combination:

[0017] The method includes, prior to the step of implementing the open object detection model, a step of identifying the part of the current image corresponding to the lane, the identified part forming the region of interest;

[0018] The method further comprises, in the event of detection of an object by the open object detection model: • a calculation of a confidence bounding box, corresponding to the part of the current image in which the object detection, by the open object detection model, has a predetermined classification confidence level; and / or • a calculation of an open detection explainability zone, from an open detection explainability map indicative of the contribution of each image element in the detection of the object; Object detection is considered consolidated if: • the detected object is included in the bounding box of trust; and / or • proximity, according to a predetermined proximity metric, between the bounding box associated with the detected object and the calculated open detection explainability zone is greater than a predetermined open detection proximity threshold;

[0019] The process further comprises the steps: • implementation of an anomaly detection model to detect the presence or absence, in the region of interest of the current image, of an element forming an anomaly; • upon detection of an anomaly, implementation of a classification algorithm to determine whether or not the detected anomaly belongs to a critical anomaly class, by implementing at least one predetermined process, the anomaly being considered as belonging to a critical anomaly class if the classification algorithm determines that the anomaly does not belong to any of at least one predetermined non-critical known object classes; and • if an object is detected by the open object detection model, and an anomaly is detected by the anomaly detection model, comparison of at least one feature of the detected object to at least one corresponding feature of the detected anomaly; and • association of an attribute to the detected object, based on a comparison result;

[0020] if the anomaly detected by the anomaly detection model belongs to a critical anomaly class and corresponds to the object detected by the open object detection model, the attribute associated with the detected object is representative of a degree of confidence in the detection of said object as an obstacle;

[0021] The method further comprises, after the anomaly detection model implementation step, an anomaly detection consolidation step comprising: • a calculation of a confidence zone corresponding to the part of the current image in which the anomaly detection, by the anomaly detection model, has a predetermined level of detection confidence; and / or • a calculation of a detection explainability zone, from a detection explainability map indicative of the contribution of each image element in the detection of the anomaly; The detection of the anomaly is considered consolidated if: • the detected anomaly is included within the calculated confidence zone; and / or • proximity, according to a predetermined proximity metric, between the detected anomaly and the calculated detection explainability zone is greater than a predetermined detection proximity threshold;

[0022] The classification algorithm is a classification model, the method further comprising, after implementation of the classification algorithm, a classification consolidation step comprising: • a calculation of a set of confidence classes, corresponding to the classes for which a classification of the anomaly detected in one of said confidence classes, by the classification model, presents a predetermined level of classification confidence; and / or • a calculation of a classification explainability zone, from a classification explainability map indicative of the contribution of each element of the image in the classification of the detected anomaly; The classification of the detected anomaly is considered consolidated if: • the class assigned to the detected anomaly belongs to the set of confidence classes; and / or • a proximity, with regard to a predetermined proximity metric, between the detected anomaly and the calculated classification explainability zone is greater than a predetermined classification proximity threshold;

[0023] The method includes, if a class assigned by the open object detection model is a class of critical known objects or a class of unknown objects: • a determination, based on the current image, of the safety level of the detected object; and • an inclusion of the determined safety level in the alert signal.

[0024] According to another aspect of the invention, a computer program is proposed comprising executable instructions which, when executed by computer, implement the steps of the process as defined above.

[0025] The computer program can be in any computer language, such as for example in machine language, in C, C++, JAVA, Python, etc.

[0026] According to another aspect of the invention, a computer device is proposed for obstacle detection on a track, configured so as to, for at least one current image representative of the track: • Implement an open object detection model to: • detect the presence or absence of an object in a region of interest in the current image; and • if applicable, assign the detected object to one of the following: a class of known non-critical objects, a class of known critical objects, and a class of unknown objects; and • if a class assigned to the object detected by the open object detection model is a critical known object class or an unknown object class, generate an alert signal indicating the presence of an obstacle on the path.

[0027] The device according to the invention can be any type of device such as a server, a computer, a tablet, a calculator, a processor, a computer chip, programmed to implement the method according to the invention, for example by executing the computer program according to the invention. Brief description of the figures

[0028] The invention will be better understood upon reading the following description, given solely by way of non-limiting example and made with reference to the accompanying drawings in which:

[0029] [Fig.1] is a flowchart of an obstacle detection method on a track according to the invention;

[0030] [Fig.2] is a schematic representation of a device implementing the process of [Fig.1];

[0031] [Fig.3] is a schematic top view of a railway track for which the process of [Fig.1] is implemented;

[0032] [Fig.4] is a schematic representation of a road including an obstacle, as seen by a camera mounted on board a vehicle moving on the road.

[0033] It is understood that the embodiments described below are in no way limiting. In particular, variants of the invention may be conceived comprising only a selection of the features described below, isolated from the other features described, if this selection of features is sufficient to confer a technical advantage or to differentiate the invention from the prior art. This selection includes at least one preferably functional feature without structural details, or with only a portion of the structural details if this portion alone is sufficient to confer a technical advantage or to differentiate the invention from the prior art.

[0034] In particular, all the variants and all the embodiments described are combinable with each other if nothing prevents this combination from a technical point of view.

[0035] In the figures and in the rest of the description, elements common to several figures retain the same reference. Detailed description

[0036] An obstacle detection method 2 according to the invention is illustrated by [Fig. 1]. More specifically, method 2 is an obstacle detection method on a track 4 ([Fig. 3]).

[0037] For the purposes of this invention, "way" means any space intended to be traversed by a vehicle 6, in particular a motor vehicle or a railway vehicle.

[0038] Such a space corresponds, preferably, to a layout intended to receive the vehicle, increased by a dynamic dimension 12 of the vehicle and, possibly, by an additional safety margin 14 for the protection of the operators.

[0039] For example, and as illustrated by [Fig.3], in the case of a railway track 8 intended to be travelled by the vehicle 6 (in this case, a railway vehicle), the track 4 is defined as the space delimited by rails 10 of the railway track 8 and, preferably, laterally augmented by a footprint of the vehicle 6 (dynamic gauge 12), as well as by the safety margin 14.

[0040] The detection method 2 is capable of being implemented by any computer device 16 ([Fig.2]) comprising a memory 18 and a processor 20 connected together.

[0041] More specifically, the memory 18 is configured to store a detection program 22 according to the invention. Furthermore, the processor 20 is configured to execute instructions from the detection program 22 in order to implement the detection method 2.

[0042] The memory 18 is further configured to store, in a corresponding memory location 24, at least one image, for example an image from a video stream representative of a scene observed by a camera 26 ([Fig.3]).

[0043] In particular, the camera 26 is placed at the front of the vehicle 6, so as to acquire a scene located downstream of the vehicle 6, following the direction of movement of the vehicle 6.

[0044] As previously stated, the computer device 16 is configured to implement the detection method 2.

[0045] More specifically, the computer device 16 is configured to implement the detection method 2 for each current image from the camera 26 that is representative of a current operational situation. The current image is, in particular, available in the memory location 24.

[0046] As illustrated by [Fig.1], the detection method 2 includes a step 32 of implementing an open object detection model (called the "open detection step") and a step 36 of generating an alert signal (called the "alert step").

[0047] Advantageously, the detection method 2 further includes an optional open detection consolidation step 34, subsequent to the open detection step 32.

[0048] Advantageously, the detection method 2 further includes an optional channel identification step 30 (called the "identification step"), prior to the open detection step 32.

[0049] Advantageously, the detection method 2 further includes an optional step 38 of implementing an anomaly detection model (called the "anomaly detection step") and an optional step 42 of classifying an anomaly (called the "classification step") subsequent to the anomaly detection step 38.

[0050] In this case, the detection process 2 advantageously includes an optional anomaly detection consolidation step 40, subsequent to the anomaly detection step 38 and / or an optional classification consolidation step 44, subsequent to the classification step 42.

[0051] Advantageously still, in the case of the implementation of the anomaly detection steps 38 and classification steps 42, the detection process 2 further includes an optional fusion step 46, subsequent to the open detection step 32 and the classification step 42.

[0052] Identification step 30

[0053] Advantageously, the computer device 16 is configured so as to identify, during the identification step, the part of the current image that corresponds to channel 4. In this case, the identified part forms a region of interest 50.

[0054] In particular, to identify lane 4, the computer device 16 is configured to execute any suitable algorithm known to the person skilled in the art, all or part of the current image forming an input to the algorithm.

[0055] For example, in the case of rail transport, track 4 is a railway track, and the computer device 16 is configured to execute the Ego-Path algorithm in order to identify track 4.

[0056] Such an algorithm is described by Thomas Laurent in the digital preprint "Train Ego-Path Detection on Railway Tracks Using End-to-End Deep Leaming", referenced arXiv:2363.13094.

[0057] Alternatively, the region of interest 50 is identified by any known means, automated or manual.

[0058] Implementing such a step is advantageous, insofar as it helps to limit the spatial domain in which an anomaly will be sought. From this In this way, objects present in the current image, but not likely to interfere with vehicle 6, are not taken into account.

[0059] Open detection step 32

[0060] In addition, the computer device 16 is configured to, during the open detection step, provide the current image as input to an open object detection model.

[0061] Advantageously, the current image is augmented by the result of the segmentation implemented during the identification step 30 (i.e. the current image associated with an indicative mask of the detected channel 4).

[0062] Such a feature is advantageous insofar as knowledge of the identified limits of lane 4 avoids the detection of objects appearing on the image but not present on the lane and, consequently, not likely to constitute any danger from the point of the vehicle.

[0063] For the purposes of this invention, "open object detection model" means a machine learning model configured to detect objects beyond predefined objects during the model's training. In other words, an open object detection model is any model configured to perform exhaustive detection by design.

[0064] In particular, unlike very frequently used detectors (such as YOLO), which are limited in their detections to the categories of their training datasets, an open-set object detection model is configured to detect objects beyond the classes on which it was trained. This can be done by also detecting unknown objects (as in the case of an "open-set abject detector" model capable of implementing a so-called "Open Set Object Detection" method) or by other means, such as language (as in the case of the YOLO-Open World model).

[0065] For example, the open object detection model is an Open Set object detector (“open-set object detector”).

[0066] Such an Open Set object detector is, for example, described by Jiaming Han et al., in the digital preprint "Expanding Low-Density Latent Regions for Open-Set Object Detection", referenced arXiv:2203.14911.

[0067] Preferably, the open object detection model has been previously trained on the basis of a first training dataset comprising a plurality of images representing a very large number of common objects (humans, cars, trains, animals, etc.) and / or objects likely to form obstacles on a track, or even obstacles actually encountered (whether struck or not). In this case, each image is associated with a label indicating the class to which the object represented in said image belongs.

[0068] Furthermore, the open object detection model is configured to output, from the current image provided at its input: • a bounding box corresponding to an object 52 detected in the region of interest 50; and • a class associated with said bounding box, and indicative of a class of object 52 detected within said bounding box, namely: • a class of known non-critical objects, from the first training dataset; • a class of known critical objects, derived from the first training dataset; or • a class of unknown objects, among at least one homogeneous class of unknown objects (with respect to a predetermined homogeneity metric), considered critical by default.

[0069] For example, at least one class of known non-critical objects is associated with a constituent object of track 4. In the case of a railway track, such an object is, in particular, a rail, a sleeper, or the ballast.

[0070] According to another example, at least one non-critical known object class is associated with an object that may be located on track 4, but does not present a hazard. In the case of a railway track, such a non-critical known object is, for example, a beacon, such as a KVB beacon (for "Speed ​​Control by Beacons"), an ETCS beacon (for "European Train Control System"), or any other beacon. Such a critical class is defined as a class grouping together any element that has no legitimate reason to be on a track (in particular a railway track), as opposed to non-critical objects on the track known to the infrastructure manager.

[0071] According to another example, at least one class of known critical object is associated with an object that is likely to be on lane 4, and which presents a risk to the vehicle (such as another vehicle, an animal, a tree) or for which the vehicle is likely to present a danger (such as a person or an animal).

[0072] Preferably, the computer device 16 is further configured to assign a level of safety to an object belonging to a critical class (in particular belonging to the class of unknown objects), called a "critical object".

[0073] For example, the computer device 16 is configured to determine whether the critical object is dangerous (e.g., a car) or whether it has no safety impact (e.g., a shoe).

[0074] The determination of the level of safety will be briefly described.

[0075] According to a first example, the computer device 16 is configured to calculate a distance, in a predetermined latent classification space, between the detected object and at least one representative cluster of a known hazardous object. In addition, the computer device 16 is configured to determine that the critical object is hazardous if said calculated distance is between a first distinction threshold and a second predetermined threshold higher than the first distinction threshold.

[0076] According to another example, the computer device 16 is configured to evaluate, from the current image, a physical characteristic of the detected critical object (in particular, its size). Furthermore, the computer device 16 is configured to determine that the detected object is a hazardous critical object based on the evaluated physical characteristic (for example, if its size exceeds a predetermined size limit).

[0077] Open detection consolidation step 34

[0078] Preferably, in the case where the open object detection model detects an object 52 in the current image, the computer device 16 is configured to implement the open detection consolidation step 34 in order to consolidate the open detection of said object 52 (i.e. to verify that said open detection is certifiable with regard to at least one predetermined criterion).

[0079] In particular (but not exclusively), the computer device 16 is configured to implement step 34 if, at the end of the open detection step 32, the detected object belongs to the class of unknown objects or to the class of critical known objects.

[0080] More specifically, the computer device 16 is configured to calculate a bounding box of trust.

[0081] Such a bounding box of confidence corresponds to the part of the current image in which the detection of the object 52, by the open object detection model, presents a predetermined open detection confidence level (from a prior qualification of the open object detection model).

[0082] For example, the predetermined open detection confidence level is 95%.

[0083] In this case, the computer device 16 is configured to determine that the open detection of the object is consolidated if the detected object 52 is included in the bounding box of trust

[0084] Alternatively, or in a complementary manner, the computer device 16 is configured to calculate an open detection explainability zone, from an open detection explainability map relating to the detection of object 52 (in the current image) by the open object detection model.

[0085] For the purposes of this invention, "explainability map" means a map indicating the contribution of each element of the image (pixels, attributes, etc.) to a prediction (in this case, the detection of object 52). For example, such an explainability map may correspond to one of the following maps, known to those skilled in the art under their English names: saliency map or contribution map.

[0086] For example, in order to obtain the open detection explainability zone, the computer device 16 is configured to implement a process comprising: • an extraction, from the output of the open object detection model, of the open detection explainability map relating to the detection of object 52 (in the current image) by the open object detection model; and • a calculation, from the extracted open detection explainability map, of the corresponding open detection explainability zone.

[0087] In this case, the open detection explainability zone is defined as the part of the open detection explainability map for which the contribution to the detection of object 52, by the open object detection model, is greater than a predetermined open detection contribution threshold.

[0088] Alternatively, the open detection explainability zone is an output of the open object detection model.

[0089] Furthermore, the detection of object 52 by the open object detection model is considered consolidated if a proximity score, with respect to a predetermined proximity metric, between the bounding box associated with the detected object 52 and the calculated open detection explainability zone is greater than a predetermined open detection proximity threshold.

[0090] For example, such a proximity score results from the implementation of a predetermined similarity measure, such as the structural similarity index measure, or SSIM (from the English "Structural Similarity Index Measure").

[0091] According to another example, such a proximity score is a value that is all the greater the closer the object 52 and the pixels associated with the detection explainability zone are in the image, for example all the greater the smaller the distance between, on the one hand, a barycenter of the bounding box associated with the detected object 52 and, on the other hand, a pixel of the current image associated with a barycenter of the detection explainability zone.

[0092] Such a detection consolidation step is advantageous, insofar as, by producing explanatory data of the decision-making process during open detection, it allows for possible certification (or approval) of the computer device 16, which is a prerequisite in security function.

[0093] Anomaly detection step 38

[0094] Advantageously, the computer device 16 is configured to implement, during the anomaly detection step 38, an anomaly detection model to detect the presence or absence, in the identified region of interest 50, of an element forming an anomaly.

[0095] By "anomaly", it is understood, for the purposes of the present invention, as an object (i.e. a set of pixels), defined as a deviation from a previously learned statistical representation of normality, called "normal appearance" of channel 4.

[0096] For example, in the case of a railway, such a normal appearance corresponds to a succession of sleepers arranged between two rails and resting on ballast.

[0097] Preferably, the anomaly detection model was trained on the basis of a second training dataset comprising a plurality of images, each representative of a lane considered to be normal.

[0098] For example, the anomaly detection model comprises three software blocks in series, the first block presenting an encoder architecture, the second block presenting a decoder architecture, and the third block presenting a discriminator architecture.

[0099] During the training of such anomaly detection model, said three building blocks operate in a hybrid manner.

[0100] More specifically, in a first phase of training, the encoder and decoder function together as a variational auto-encoder (or VAE), responsible for roughly reconstructing normality. In this case, the VAE learns to grasp the essence of normality by simplifying (using its encoder) and reconstructing (using its decoder) the normal images provided at its input.

[0101] Furthermore, in a second phase of training, the decoder and the discriminator operate jointly as a generative adversarial network (or GAN), where the decoder acts as an image generator, responsible for making the reconstructed images more accurate and realistic than in the first phase. In this case, the GAN refines the quality of the previous reconstruction, so as to make the image more realistic and comparable with the initial real image.

[0102] In this case, during the inference phase (i.e. the processing of the current image by the trained detection model), only the encoder and the decoder are implemented.

[0103] When, during the anomaly detection step 38, the region of interest 50 contains an anomaly, the anomaly detection model seeks to bring the current image as close as possible to that of a normal situation. The image thus reconstructed then differs significantly from the initial current image in the vicinity of the anomaly, which makes it possible to locate said anomaly in the current image.

[0104] The anomaly detection model is not an object detector: this results in a complementarity between, on the one hand, step 32 (positive detection of an object on the track) and, on the other hand, the anomaly detection steps 38 and classification steps 42 described below (negative detection of a deviation from a normal situation), which is advantageous.

[0105] Anomaly detection consolidation step 40

[0106] Advantageously, in the event that an anomaly is detected, the computer device 16 is configured to implement the anomaly detection consolidation step 40 in order to consolidate the anomaly detection (i.e. to verify that said detection is certifiable with regard to at least one predetermined criterion).

[0107] More specifically, the computer device 16 is configured to calculate a confidence zone (corresponding to a segmentation of the image).

[0108] Such a confidence zone corresponds to the part of the current image in which the detection of the anomaly, by the anomaly detection model, presents a predetermined level of detection confidence (from a prior qualification of the anomaly detection model).

[0109] For example, the predetermined detection confidence level is 95%.

[0110] In this case, the computer device 16 is configured to determine that the anomaly detection is consolidated if the detected anomaly is included in the confidence zone.

[0111] Alternatively, or in a complementary manner, the computer device 16 is configured to calculate a detection explainability zone, from a detection explainability map relating to the detection of the anomaly (in the current image) by the anomaly detection model.

[0112] In this case, the explainability map is an indicative map of the contribution of each element of the image (pixels, attributes, etc.) to the detection of the anomaly. As previously stated, such an explainability map is likely to correspond to a so-called "saliency map" or a so-called "contribution map," which are known to a person skilled in the art.

[0113] For example, in order to obtain the detection explainability zone, the computer device 16 is configured to implement a process comprising: • an extraction, from the output of the anomaly detection model, of the detection explainability map relating to the detection of the anomaly (in the current image) by the anomaly detection model; and • a calculation, from the extracted detection explainability map, of the corresponding detection explainability zone.

[0114] In this case, the detection explainability zone is defined as the part of the detection explainability map for which the contribution to the detection of the anomaly, by the anomaly detection model, is greater than a predetermined detection contribution threshold.

[0115] Alternatively, the detection explainability zone is an output of the anomaly detection model.

[0116] In addition, the anomaly detection is considered consolidated if a proximity score, with respect to a predetermined proximity metric, between the detected anomaly and the calculated detection explainability zone is greater than a predetermined detection proximity threshold.

[0117] Such a proximity score is, for example, obtained by using any of the functions described above in relation to the consolidation of open detection.

[0118] Such a detection consolidation step is advantageous, insofar as, by producing explanatory data of the decision-making process during detection, it allows for possible certification (or approval) of the computer device 16, which is a prerequisite in security function.

[0119] Classification step 42

[0120] In the event that the anomaly detection step 38 is implemented, and an anomaly is detected, the computer device 16 is configured to implement, during the classification step 42, at least one predetermined process to determine whether or not the detected anomaly belongs to a critical anomaly class.

[0121] Such a critical anomaly class is defined as a class grouping any element that has no legitimate reason to be on a track (in particular a railway track), as opposed to non-critical objects on the track known to the infrastructure manager.

[0122] More specifically, the computer device 16 is configured to implement a classification algorithm to determine whether the anomaly belongs to one of at least one predetermined non-critical known object class.

[0123] In this case, the anomaly is considered to belong to the critical anomaly class if the classification algorithm determines that the anomaly does not belong to any predetermined non-critical known object class.

[0124] For example, at least one non-critical known object class is associated with an object that is a constituent of lane 4. In another example, at least one non-critical known object class is associated with an object that is likely to be found on lane 4 but does not present a hazard. More generally, the non-critical known object classes implemented by the classification model are likely to be identical to the non-critical known object classes implemented by the open object detection model.

[0125] Examples of such known non-critical objects have been described previously, in relation to open detection step 32.

[0126] Preferably, to carry out such a classification, the computer device 16 is configured to implement an artificial intelligence model, and more specifically a classification model.

[0127] More specifically, such a classification model was previously trained on the basis of a third training dataset comprising a plurality of images, each representative of a non-critical object, and, for each image, a corresponding label indicating the class to which the non-critical object represented belongs.

[0128] In this case, for each image of the third training dataset provided as input to the classification model during its training, the corresponding label forms the expected output of the classification model.

[0129] In addition, the classification model is parameterized to assign, to one of at least one class relating to critical objects, any object (and in particular any detected anomaly) that would not correspond to one of the predetermined classes of known non-critical objects.

[0130] Alternatively, to perform such a classification, the computer device 16 is configured to compare the detected anomaly to each reference image in a predetermined reference image bank. In particular, each reference image is representative of a reference object belonging to a respective predetermined non-critical known object class.

[0131] In this case, the computer device 16 is configured to assign the detected anomaly to the class corresponding to the most similar reference image, if a comparison result is greater than a predetermined similarity threshold. Otherwise, it is determined that the detected anomaly belongs to the critical class.

[0132] Classification consolidation step 44

[0133] Advantageously, the computer device 16 is configured to implement the classification consolidation step 44 in order to consolidate the classification of the anomaly (i.e. to verify that said classification is certifiable with regard to at least one predetermined criterion).

[0134] In particular (but not exclusively), the computer device 16 is configured to implement step 44 if, at the end of the classification step 42, the detected anomaly is classified as critical.

[0135] More specifically, the computer device 16 is configured to calculate a set of confidence classes.

[0136] Such a set of confidence classes includes the class(es) (critical(s) and / or non-critical(s)) such that, if a classification result belongs to one of said classes, then said result has a predetermined classification confidence level (derived from a prior qualification of classification model).

[0137] For example, the predetermined classification confidence level is 95%.

[0138] In this case, the computer device 16 is configured to determine that the classification of the anomaly is consolidated if the class to which the detected anomaly is assigned belongs to the set of confidence classes.

[0139] Alternatively, or in a complementary manner, the computer device 16 is configured to calculate a classification explainability zone, from a classification explainability map relating to the classification, by the classification model, of the detected anomaly.

[0140] In this case, the classification explainability map is an indicative map of the contribution of each image element (pixels, attributes, etc.) to the classification of the anomaly. As previously stated, such a classification explainability map is likely to correspond to a so-called "saliency map" or a so-called "contribution map," which are known to a person skilled in the art.

[0141] For example, in order to obtain the classification explainability zone, the computer device 16 is configured to implement a process comprising: • an extraction, from the output of the classification model, of the detection explainability map relating to the classification of the anomaly (in the current image) by the classification model; and • a calculation, from the extracted classification explainability map, of the corresponding classification explainability zone.

[0142] In this case, the classification explainability zone is defined as the part of the classification explainability map for which the contribution to the classification of the anomaly, by the classification model, is greater than a predetermined classification contribution threshold.

[0143] Alternatively, the classification explainability zone is an output of the classification model.

[0144] In addition, the classification of the anomaly is considered to be consolidated if a proximity score between the detected anomaly and the calculated classification explainability zone is greater than a predetermined classification proximity threshold.

[0145] Such a proximity score is, for example, obtained by using any of the functions described above in relation to the consolidation of anomaly detection.

[0146] Such a detection consolidation step is advantageous, insofar as, by producing explanatory data of the decision-making process during classification, it allows for possible certification (or approval) of the computer device 16, which is a prerequisite in security function.

[0147] Merge step 46

[0148] Preferably, the computer device 16 is configured to compare, during the fusion step 46, the results obtained at the end of each of the open detection 32 and classification 42 steps.

[0149] In particular, the computer device 16 is configured to determine consistency between, on the one hand, the results of open detection, and, on the other hand, the results of anomaly detection and anomaly classification. In particular, the computer device 16 is configured to determine a correspondence between the detected object and the detected anomaly. Furthermore, the computer device 16 is configured to determine a correspondence between the critical (or non-critical) nature of the detected object and anomaly.

[0150] Preferably, the computer device 16 is configured to determine a proximity by location between the results provided by the open object detection model and the anomaly detection model. Such a calculation is based, in particular, on a calculation of the distance between the bounding box determined by the open object detection model and the anomaly detected by the detection model.

[0151] Alternatively, or in addition, the computer device 16 is configured to determine a proximity by appearance between the results provided by the open object detection model and the anomaly detection model. Such a calculation relies, in particular, on a calculation of visual similarity (shape, texture, color, etc.) between the contents of the bounding box determined by the open object detection model and the anomaly detected by the detection model.

[0152] Alternatively, or in addition, the computer device 16 is configured to determine a proximity by explainability between the results provided by the open object detection model and the anomaly detection model or the classification model. Such a calculation relies, in particular, on a similarity and / or proximity calculation by location between the explainability maps provided by the open object detection model and the anomaly detection model or the classification model.

[0153] Alternatively, or in a complementary manner, the computer device 16 is configured to determine the consistency of conforming predictions between the results provided by the open object detection model and the anomaly detection model. Such a calculation is based, in particular, on a calculation of the distance between the bounding box of confidence and the detection confidence zone, or on their overlap.

[0154] Preferably, in the case of consistent detections, and if the class associated with the bounding box determined by the open object detection model is the unknown object class, the computer device 16 is preferably configured to enrich the critical known object class with the detected anomaly. In this In this case, the anomaly is associated with a corresponding label (indicative of the type of object, or the nature of the anomaly).

[0155] Furthermore, the computer device 16 is configured to associate an attribute with the detected object, based on an output from the anomaly detection model and the classification algorithm. Such an attribute is, in particular, representative of a degree of confidence in the detection of the obstacle.

[0156] Such a degree of confidence is, in particular, all the greater when: • open detection is consolidated; • Anomaly detection is consolidated; • the anomaly classification is consolidated; and • the results of open detection and classification are consistent with each other.

[0157] Alert step 36

[0158] In the event that it is determined that the object 52 belongs to the critical class, the computer device 16 is configured to generate, during the alert step 36, an alert signal indicating the presence of an obstacle on the track 4.

[0159] Such a warning signal is, for example, a signal to a vehicle braking device, to command the vehicle to stop, or a signal to a vehicle computer or a centralized control station.

[0160] Preferably, if the fusion step is implemented, the computer device 16 is further configured to integrate, in the warning signal, the attribute representing the level of confidence in the detection of the obstacle.

[0161] Preferably, if the safety level of the critical object has been determined, the computer device 16 is further configured to include the determined safety level in the alert signal.

[0162] Operation

[0163] The operation of the computer device 16 will now be described with reference to [Fig.1].

[0164] The computer device 16 receives a current image, for example extracted from a video stream representative of a channel acquired by the camera 26.

[0165] Preferably, during the identification step 30, the computer device 16 identifies the part of the current image that corresponds to channel 4. The identified part forms the region of interest 50.

[0166] Then, during the open detection step 32, the computer device 16 provides, as input to the open object detection model, the current image, preferably augmented with the indication of the region of interest 50.

[0167] This results, in particular, if an object 52 is detected in the region of interest 50: • a bounding box corresponding to the detected object 52; and • a class associated with said bounding box, and indicative of a class of the object 52 detected in said bounding box, namely a class of known non-critical objects, a class of known critical objects, or a class of unknown objects (considered critical by default).

[0168] Then, preferably, the computer device 16 implements the open detection consolidation step 34 in order to consolidate or not the open detection of said object 52.

[0169] Preferably, during the anomaly detection step 38, the computer device 16 implements the anomaly detection model to detect the presence or absence, in the current image (and preferably in the identified region of interest 50), of an element forming an anomaly.

[0170] In the event that an anomaly is detected, the computer device 16 advantageously implements the anomaly detection consolidation step 40 in order to consolidate or not the anomaly detection.

[0171] In addition, preferably, in the case where an anomaly is detected, then, during the classification step 42, the computer device 16 determines whether or not the detected anomaly belongs to a critical class.

[0172] Then, the computer device 16 advantageously implements the classification consolidation step 44 in order to consolidate or not the classification of the anomaly.

[0173] Then, preferably during the fusion step 46, the computer device 16 compares the results obtained at the end of each of the open detection 32 and classification 42 steps, in particular to determine consistency between, on the one hand, the results of open detection, and, on the other hand, the results of anomaly detection and anomaly classification.

[0174] Furthermore, the computer device 16 associates an attribute with the detected object, based on an output from the anomaly detection model and the classification algorithm. Such an attribute is, in particular, representative of a degree of confidence in the detection of the obstacle.

[0175] Then, during the alert step 36, in the event that it is determined that the object belongs to the critical class, the computer device 16 generates an alert signal indicating the presence of an obstacle on the track 4.

[0176] Preferably, if the fusion step is implemented, the computer device 16 integrates, in the warning signal, the attribute representing the level of confidence in the detection of the obstacle.

[0177] Of course, the invention is not limited to the examples just described.

Claims

Demands

1. A method (2) for detecting obstacles on a lane (4), the method being computer-implemented and comprising, for at least one current image representative of the lane, the steps: • implementation (32) of an open object detection model to: • detect the presence or absence of an object in a region of interest (50) of the current image; and • if appropriate, assign the detected object (52) to one of: a class of known non-critical objects, a class of known critical objects and a class of unknown objects; • if a class assigned to the detected object (52) by the open object detection model is a class of known critical objects or a class of unknown objects, generation (36) of an alert signal indicating the presence of an obstacle on the lane (4).

2. Method according to claim 1, comprising, prior to the step of implementing the open object detection model, a step (30) of identifying the part of the current image corresponding to the channel (4), the identified part forming the region of interest (50).

3. A method according to claim 1 or 2, further comprising, in the case of detection of an object by the open object detection model: • a calculation of a confidence bounding box, corresponding to the part of the current image in which the detection of the object, by the open object detection model, has a predetermined level of classification confidence; and / or • a calculation of an open detection explainability zone, from an open detection explainability map indicative of the contribution of each element of the image in the detection of the object; • the detected object is included in the bounding box of confidence; and / or • a proximity, with respect to a predetermined proximity metric, between the bounding box associated with the detected object and the calculated open detection explainability zone is greater than a predetermined open detection proximity threshold.

4. A method according to any one of claims 1 to 3, further comprising the steps: • implementation (38) of an anomaly detection model to detect the presence or absence, in the region of interest (50) of the current image, of an element forming an anomaly; • in the event of an anomaly detection, implementation (42) of a classification algorithm to determine whether or not the detected anomaly belongs to a critical anomaly class, by implementing at least one predetermined processing, the anomaly being considered as belonging to a critical anomaly class if the classification algorithm determines that the anomaly does not belong to any of at least one predetermined non-critical known object class;and • if an object is detected by the open object detection model, and an anomaly is detected by the anomaly detection model, comparison of at least one feature of the detected object to at least one corresponding feature of the detected anomaly; and • association (46) of an attribute to the detected object, based on a result of the comparison.

5. A method according to claim 4, wherein, if the anomaly detected by the anomaly detection model belongs to a critical anomaly class and corresponds to the object detected by the open object detection model, the attribute associated with the detected object is representative of a degree of confidence in the detection of said object as an obstacle.

6.

7. A method according to claim 4 or 5, further comprising, after the anomaly detection model implementation step, an anomaly detection consolidation step (40) comprising: • a calculation of a confidence zone corresponding to the part of the current image in which the anomaly detection, by the anomaly detection model, has a predetermined level of detection confidence; and / or • a calculation of a detection explainability zone, from a detection explainability map indicative of the contribution of each image element in the detection of the anomaly; The detection of the anomaly is considered consolidated if: • the detected anomaly is included within the calculated confidence zone; and / or • a proximity, with regard to a predetermined proximity metric, between the detected anomaly and the calculated detection explainability zone is greater than a predetermined detection proximity threshold. A method according to any one of claims 4 to 6, wherein the classification algorithm is a classification model, the method further comprising, after implementation of the classification algorithm, a classification consolidation step (44) comprising: • a calculation of a set of confidence classes, corresponding to the classes for which a classification of the anomaly detected in one of said confidence classes, by the classification model, presents a predetermined level of classification confidence; and / or • a calculation of a classification explainability zone, from a classification explainability map indicative of the contribution of each element of the image in the classification of the detected anomaly; The classification of the detected anomaly is considered consolidated if: • the class assigned to the detected anomaly belongs to the set of confidence classes; and / or • a proximity, with regard to a predetermined proximity metric, between the detected anomaly and the calculated classification explainability zone is greater than a predetermined classification proximity threshold.

8. A method according to any one of claims 1 to 7, comprising, if a class assigned by the open object detection model is a class of known critical objects or a class of unknown objects: • a determination, from the current image, of a safety level for the detected object; and • an inclusion of the determined safety level in the alert signal.

9. A computer program comprising executable instructions which, when executed by a computer, implement the steps of the process according to any one of claims 1 to Q

10. O. Computer device (16) for obstacle detection on a lane (4), configured so that, for at least one current image representative of the lane (4): • implement (32) an open object detection model to: • detect the presence or absence of an object in a region of interest (50) of the current image; and • if so, assign the detected object (52) to one of: a class of known non-critical objects, a class of known critical objects and a class of unknown objects; and • if a class assigned to the detected object (52) by the open object detection model is a class of known critical objects or a class of unknown objects, generate (36) an alert signal indicating the presence of an obstacle on the lane (4).

Images