Method for training a machine learning model for object discovery in an image sequence

Abstract

Method for training a model (MDO, MDOEL) for discovering objects in an input image sequence (SI), the model comprising: an encoder (ENC, ENCEL) an attention module (ATT, ATTEL) configured to transform the first feature vector into a plurality of feature vectors called slots (S1, ..SK-1), a decoder (DEC) learning the attention maps (W1, ..WK-1) being supervised (404) by a set of binary masks for the discovery of moving objects produced by an external source, called pseudo-labels (PL), the pseudo-labels (PL) being filtered (FIL) by means of the steps of: determining (401) an attention map of the foreground of the image (Wfg), calculating (402) a confidence score from the average of the values ​​of the attention map (Wfg) of the foreground of the image at the positions of each moving object present in a pseudo-label, filtering (403) the moving objects of the pseudo-labels for which the confidence score is lower than a predefined threshold, Figure 4.

Description

Title of the invention: Method for training a machine learning model for the discovery of objects in a sequence of images

[0001] The invention relates to the field of object discovery in an image sequence. This is a computer vision task that aims to locate objects present in an image by producing object masks for each located object. An object mask is a binary image comprising values ​​at '1' at the locations of the object's pixels and values ​​at '0' elsewhere. There are as many masks as there are objects present in the scene captured by the image sequence.

[0002] The invention relates to a new method for discovering objects involving the implementation of a particular machine learning model. The invention relates in particular to the training of this model to carry out an object discovery task from a given sequence of images.

[0003] The invention applies in different fields which require the localization of objects in an image sequence, in particular but not exclusively: vision systems for autonomous driving, exploration of unknown environments, video surveillance systems, segmentation of active cells in medical data or even self-learning vision systems.

[0004] A general problem to be solved in the field of object discovery is to carry out this task in an unsupervised manner unlike the object detection task which requires annotated training data. An advantage associated with unsupervised training lies in the savings made on obtaining labeled data which is most often carried out by an operator.

[0005] However, the absence of annotated data conversely leads to greater difficulty in carrying out learning. One of the challenges encountered in unsupervised object discovery is the lack of a clear definition of what an object is.

[0006] References [1],[2] describe object discovery methods that aim to locate objects characterized by their movement. In other words, these methods are oriented towards the discovery of moving objects.

[0007] The methods described in references [1] and [2] propose to replace human annotation of the learning data by exploiting the movement information of the objects within the image sequence.

[0008] The choice of motion information has the advantage that this information can be estimated automatically and without human intervention (supervision). These approaches propose an object discovery model integrated into a pipeline composed of two main phases.

[0009] [Fig. 1a] illustrates the first phase which consists of learning to generate a set of binary object masks MO from images SI and associated optical flow maps FO. This task is carried out using an AI machine learning model. The optical flow map corresponds to a motion map in which the pixel values ​​describe the displacement of moving objects between two consecutive images. The AI ​​model is trained from synthetic data, without the need for human annotations.

[0010] [Fig.lb] illustrates the second phase which then consists of applying the trained model IA on real data SI' accompanied by an optical flow map FO' to generate object masks MO' which correspond to pseudo-labels because they can be noisy and / or incomplete. The noise can come from the imperfection of the motion map and is mainly manifested by the presence of random segments occupying the background of the image. Furthermore, the incompleteness is linked to the very use of the motion information, resulting in the absence of static objects in these pseudo-labels.

[0011] Another MGOD model is then trained to perform the discovery of DO objects from the image sequence SI' and the pseudo-labels MO'. The approach described in reference [3] is based on an architecture that implements an attention mechanism applied to slots. Each slot is associated with an attention map and the training of the model forces the sharing of the regions of the input image between several attention maps whose values ​​vary between 0 and 1. Each attention map activates a specific region (the values ​​of the pixels in this region are then close to 1) and attenuates the rest of the image (pixels close to 0). We then say that the attention of the model is oriented towards this activated region.

[0012] The MGOD model is trained by integrating the pseudo-labels of moving objects MO' into the learning architecture in the following way: some maps among the K attention maps are supervised (by an appropriate loss function) to contain the moving segments, while the other attention maps are left free without supervision. The observed model behavior is that at the supervised maps appear moving objects, and at the unsupervised maps appear either static objects that are visually similar to them, or random segments (noise).

[0013] The method described in Figures 1a and 1b has two limitations in particular.

[0014] A first problem is the lack of distinction between random segments corresponding to noise and useful segments corresponding to objects. This results from the absence of training supervision in object discovery. Thus, these methods are not very robust to noise, particularly related to camera movement.

[0015] A second problem is that this method mainly exploits motion information, which severely limits the localization of static objects in the sequence. The 'moving object to static object' extension offered by the so-called "slot-attention" architecture proposed in particular in reference [2] works by redirecting the model's attention to objects that resemble those already known to be moving. However, this method has its limitations. It does not guarantee that the model will detect a sufficient quantity of static objects, or even that it will detect them reliably. Detection depends entirely on the model's ability to judge whether a new static object sufficiently resembles the moving objects it already knows.

[0016] Reference [4] addresses, in a more recent approach, the first aforementioned problem by proposing a component for managing noise in the background of the image. This component consists of learning the separation between, on the one hand, all the objects in the scene that are activated in a map dedicated to the foreground of the image and noted W / g, and on the other hand, the background of the image that does not contain objects of interest (i.e. objects capable of moving). This background is activated in a map noted W among the K attention maps. By placing the background in this map and since all the attention maps are complementary to each other, this prevents random segments from appearing in the other K - 1 maps dedicated to the objects.

[0017] This approach, however, only allows partial management of noise in the image background. Indeed, the noise that appears in the image background, among the model outputs, has mainly two causes: the first cause is related to the slot-attention architecture, and to the fact that free attention maps can capture noise, and the second cause is related to the input pseudo-labels which, if noisy, propagate this noise to the model outputs. The approach in article [4] addresses the first cause by introducing an additional constraint that prevents empty attention maps from capturing noise. However, the second cause of this problem is not addressed.

[0018] The invention aims to overcome the limitations of the prior art by means of a method which provides a solution to the two problems discussed above.

[0019] The invention proposes to introduce automatic and robust filtering of the noise segments contained in the pseudo-labels at the input of the model from a confidence score calculation.

[0020] The invention also proposes to introduce a distillation learning approach to integrate static objects into the supervision of the attention maps of the model. Thus, even when the pseudo-labels come from the movement and therefore do not contain the static objects, they are completed by introducing a second source of supervision in the form of a master model and via distillation learning. The result of this component results in much better localization of objects, particularly static objects.

[0021] The proposed method for discovering objects in an image sequence is capable of filtering the input pseudo-labels resulting from the movement and of automatically improving itself over the course of training, by re-integrating its own results.

[0022] The invention makes it possible to resolve a technical obstacle linked to the noise present in the inputs of the object discovery models of the prior art.

[0023] Particularly in distillation training architectures, noise can be propagated in both the master and student models. In this type of scenario, the success or failure of distillation depends on the amount of noise present: if the noise segments are in the minority among the input pseudo-labels, the model is able to ignore them. This condition is not necessarily verified in real applications where noise can reach significant levels, leading to distillation failure.

[0024] Furthermore, this technical obstacle proves to be more pronounced when the basic model is based on the so-called 'slot attention' architecture. Indeed, this architecture is particularly interesting for the methods of the prior art referenced above because it allows attention to be extended to static objects, absent in the movement information. However, the 'slot attention' architecture also amplifies the noise received at the input. Indeed, the same mechanism that allows the 'moving objects to static objects' extension is responsible for the amplification of the noise received at the input. For example, the model can generate, at the level of the free attention maps, other random segments similar to the input noise. This noise therefore becomes critical and hinders the development of distillation applied to the discovery of objects by a 'slot attention' mechanism.

[0025] The subject of the invention is a method, implemented by computer, for training an automatic model for discovering objects in a sequence of input images, the model comprising: - an encoder to encode each image into a first vector of characteristics, - an attention module configured to transform the first feature vector into a plurality of feature vectors called slots, the state of a slot being determined from a similarity calculation between the first feature vector and each slot in its current state, each similarity calculation defining an attention map, - a decoder to decode all the slots so as to reconstruct a sequence of images corresponding to the input sequence of images, - the learning of attention maps being supervised by a set of binary masks for discovering moving objects produced by an external source, called pseudo-labels, such that each attention map is activated in an area corresponding to a distinct object contained in the pseudo-labels, an additional attention map is activated in an area corresponding to the background of the image, - the pseudo-labels being filtered by means of the steps of: i. determine an attention map of the foreground of the image, ii. calculate a confidence score from the average of the values ​​of the attention map from the foreground of the image to the positions of each moving object present in a pseudo-label, iii. filter mobile objects from pseudo-labels for which the confidence score is lower than a predefined threshold,

[0026] According to a particular aspect of the invention, the attention map of the foreground of the image is determined from the attention map of the background of the image of the attention module of the model.

[0027] According to a particular aspect of the invention, said model is a student model trained at least partially via a mechanism for transferring learning by distillation from a master model, the master model comprising an encoder and an attention module, the attention map of the foreground of the image of the student model being determined from the attention map of the background of the image of the attention module of the master model.

[0028] According to a particular aspect of the invention, the learning of the attention maps of the student model is supervised by the attention maps of the master model so that each attention map is activated in an area corresponding to a distinct object discovered in the attention maps of the master model.

[0029] According to a particular aspect of the invention, the learning of the attention maps of the student model comprises the steps of: - Binarize each attention map of the master model, - Determine the set of connected regions in the set of binarized attention maps, each connected region corresponding to a distinct discovered object

[0030] According to a particular aspect of the invention, the learning of the attention maps of the student model further comprises the steps of: - Calculate a confidence score for each distinct object discovered as being equal to the average value of the activations of said object in each attention map of the master model, - Filter objects whose confidence score is lower than a predetermined threshold

[0031] According to a particular aspect of the invention, the supervision of the attention maps of the student model is at least carried out by means of a first cross-entropy loss function applied between the attention maps of the student model and the objects determined from the attention maps of the master model weighted by their confidence score.

[0032] According to a particular aspect of the invention, the supervision of the attention maps of the model is at least carried out by means of a second cross-entropy loss function applied between the attention maps of the model and the objects of the pseudo-labels weighted by their confidence score.

[0033] According to a particular aspect of the invention, the pseudo-labels are obtained from the sequence of images and an associated optical flow sequence.

[0034] The invention also relates to a method, implemented by computer, for discovering objects in a sequence of images comprising the steps of: - Receiving a sequence of images, - Executing the automatic object discovery model trained using the method according to the invention for the sequence of images so as to generate at least one location mask of an object in the sequence of images, each location mask being obtained from an attention map.

[0035] The invention also relates to a computer program comprising instructions for executing the method according to the invention, when the program is executed by a processor.

[0036] The invention also relates to a recording medium readable by a processor on which is recorded a program comprising instructions for the execution of the method according to the invention, when the program is executed by a processor.

[0037] Other characteristics and advantages of the present invention will appear better on reading the description which follows in relation to the following appended drawings.

[0038] [Fig.la] represents a diagram illustrating a method of automatic learning of object masks from motion information according to the prior art,

[0039] [Fig.lb] represents a diagram illustrating a method of automatic learning for discovering objects from object masks obtained via the method of [Fig.la],

[0040] [Fig.2] represents a diagram illustrating the implementation of a method automatic learning of object discovery according to a first embodiment of the invention,

[0041] [Fig.3] represents a diagram illustrating a pseudo-label filtering step in the method of [Fig.2],

[0042] [Fig.4] represents a flowchart detailing the implementation of the filtering step,

[0043] [Fig.5] represents a diagram illustrating the implementation of a method machine learning for object discovery according to a second mode of rea- lization of the invention,

[0044] [Fig.6] represents a diagram illustrating the implementation of an automatic learning method for discovering objects according to a third embodiment of the invention,

[0045] [Fig.7] represents a flowchart detailing the implementation of a step of supervision of a student model by a master model, according to the third embodiment of the invention.

[0046] [Fig.2] shows, in a diagram, the method of training a model automatic learning for object discovery according to a first embodiment of the invention.

[0047] This first embodiment aims to solve the specific problem of the presence of noise in the pseudo-labels used for learning.

[0048] For this purpose, the MDO model receives as input a sequence of images SI and a set of pseudo-labels PL which correspond to binary masks of objects obtained, for example, by means of the method described in [Fig.1a]. These masks are imperfect due to the presence of noise. They aim to label the moving objects in the scene.

[0049] Without departing from the scope of the invention, the PL pseudo-labels can be obtained by other methods, for example human annotations or via other types of object discovery algorithms.

[0050] Each pseudo-label is supposed to correspond to a moving object present in the image sequence SI. For the reasons mentioned above, some masks may correspond to noise and not to objects.

[0051] The basic MDO model used corresponds to that described in references [1], [2] and [3] which is based on a “slot attention” type architecture. More precisely, this model comprises an ENC encoder configured to encode each image in a latent representation space so as to generate a vector of spatio-temporal characteristics descriptive of the content of the sequence. The ENC encoder is for example an artificial neural network such as a residual neural network or any other machine learning model capable of encoding a sequence of images into a set of spatio-temporal characteristics.

[0052] The MDO model also includes an ATT attention module which aims to transform a set of N spatio-temporal characteristics obtained at the output of the ENC encoder into K vectors called “slots” whose dimension is a hyper-parameter of the architecture. The ATT attention module is trained so that each slot describes an object or more generally a different area of ​​interest in the image sequence.

[0053] The ATT attention module implements an iterative attention mechanism that aims to learn a transformation function or mapping of N features into K slots, the coefficients of this function can be represented in the form of an attention map whose normalized values ​​vary between 0 and 1. Each attention map activates a different area of ​​the image.

[0054] In [Fig.2] we have represented Kl slots Si,...SK-i associated with Kl objects of the scene and an additional slot Sbg corresponding to the background of the scene. Each slot is associated with an attention map Wi,...WK-i,Wbg. The iterative attention mechanism aimed at training the attention module ATT is described in more detail in reference [3].

[0055] The slots Si,.. .SK-i, Sbg obtained at the final iteration are then supplied to a DEC decoder which performs a slot decoding operation to reconstruct a sequence of SR images. The DEC decoder is for example a convolutional neural network.

[0056] The MDO model is trained to minimize a loss function LMSE based on a distance or error criterion between the reconstructed sequence SR and the input sequence SL

[0057] Furthermore, the PL pseudo-labels are used to supervise certain maps among the Kl attention maps Wi,...WK-i so that each attention map is oriented towards a different object among the set of object masks that constitute the pseudo-labels. This principle, introduced in references [1] and [2], consists of supervising the training of the model by an external source characterizing the movement in the scene, i.e. the moving objects. The maps to be supervised are selected via a matching algorithm between the pseudo-labels and the content of the attention maps. This process is described in more detail in reference [1] and aims to orient each supervised attention map towards a different object among the set of object masks that constitute the pseudo-labels.

[0058] Thus, the attention module ATT is trained to generate Kl attention maps which are oriented towards distinct objects and an attention map oriented towards the background of the image. From each attention map obtained for an object, a location mask of the object can be derived by binarizing the activation values ​​of the map.

[0059] The invention aims to further improve the supervised training of the MDO model by adding a FIL filtering function of the pseudo-labels and by associating a confidence score with each object identified in the pseudo-labels in the supervision of the attention maps.

[0060] Figures 3 and 4 illustrate the implementation of this FIL filtering function which comprises several successive steps.

[0061] In step 401, a foreground attention map of the image Wfg is first determined from the background attention map of the image Wbg as the negative thereof: Wfg=l- Wbg. The foreground attention map theoretically contains all the objects present in the image, unlike the background attention map. the background which only contains the background of the image.

[0062] An example of a foreground attention map Wfg is given in [Fig.3] for an image I.

[0063] In [Fig.3] we also show an example of PL pseudo-labels comprising four masks of binary objects mb m2, m3, uq. The pseudo-labels being in binary form, they do not allow to distinguish an object from a noise segment.

[0064] In step 402 of the method, a confidence score is then calculated for each of the masks from the attention map of the foreground of the image Wfg which is composed of non-binary values, typically varying between 0 and 1, and which reflects the semantic content of the image. In other words, the objects identified in the pseudolabels coming from an external source are found in the attention map Wfg which is supposed to contain a representation of all the objects in the scene.

[0065] For each mask m;, i varying from 1 to M the number of pseudo-labels, the confidence score is calculated as being the average of the activations of the attention map W fg at the spatial positions of the map corresponding to the spatial positions of the mask for which the mask has a value of 1.

[0066] The confidence score can be represented by the following formula: [0°67] SCOremi = pr1— Q m^j)

[0068] 0 denotes the term-by-term product operator of two matrices.

[0069] The index j varies from 1 to the number of pixels N in the activation map Wfg which is of the same dimension as each object mask m;.

[0070] The confidence score can be calculated for each object because the attention map Wfg contains activations for all pixels in the scene since this map is calculated from the scene received as input.

[0071] In step 403, the calculated confidence score is compared to a predefined threshold p whose value is taken close to 1 to effectively filter the least reliable predictions, for example to a value of 0.9. The masks for which the confidence score is below this threshold are deleted from the set of pseudo-labels as illustrated in the example of [Fig.3]. In this example, the masks m2 and m3 are deleted because their confidence score is below the threshold p and more likely correspond to a noise segment.

[0072] Indeed, the Wfg attention map must normally have high activation values ​​for the areas of the image corresponding to objects. By comparing the average activation value of the areas of the Wfg attention map corresponding to the objects identified in the PL pseudo-labels to a threshold, this makes it possible to eliminate the objects which have low activation values, which is significant of the presence of noise.

[0073] The masks for which the confidence score is greater than or equal to the threshold p are on the other hand used to supervise, in step 404, the training of the attention module ATT by means of a loss function which aims to associate certain attention maps among the Kl attention maps Wi,...WK-i with one of the mobile objects present in the pseudo-labels after filtering.

[0074] For example, a binary cross-entropy loss function is used such that:

[0075] LBCE{m, W) = ^ï^=^ + scorem)m'(j)\og(W ) +(l-m'(j))log( W(z) )]

[0076] m' denotes an object mask retained after filtering and W denotes one of the Kl attention maps. In other words, each filtered object mask m' is used to supervise one of the attention maps W of the attention module. Other loss functions can be envisaged without departing from the scope of the invention.

[0077] The score of the filtered masks is taken into account in the calculation of the loss function so as to give greater weight to objects with higher confidence.

[0078] Thus, filtering the noise in the pseudo-labels makes it possible to make the model more robust to noise by effectively eliminating irrelevant segments in the model outputs.

[0079] Eliminating noise segments also makes it possible to apply a distillation mechanism for object discovery based on a slot attention model.

[0080] The foreground attention map can be determined by supervision using a binary cross-entropy loss function of the type:

[0081] Lfs / hg(m'fs, Wfs) = j)log(Wf JJ)) + aWfg(j)]

[0082] m'fg is the sum of all object masks m' after filtering,

[0083] a is a regularization coefficient.

[0084] For this, a second embodiment of the invention is illustrated in [Fig.5].

[0085] In this second embodiment, a mechanism for transferring learning by distillation is implemented on the basis of a general principle described in particular in reference [5].

[0086] The distillation process involves the use of two models of the same architecture: a master model MDOma and a student model MD0EL. Each of the two models thus comprises an encoder ENCel,ENCMa and an attention module ATTbl,ATTma as described previously. Only the decoder DEC of the student model is used in the distillation learning process. The master model comprises a similar decoder but it does not participate in any function or step during learning.

[0087] Each of the two models receives as input a different transformed version of the image sequence in order to avoid them producing the same results leading to a self-confirmation bias. The transformations applied to the image sequence SI are for example: transformations of intensities (colors) or geometric transformations such as a cropping or zooming operation. Geometric transformations are also applied identically to the PL pseudo-labels. Generally speaking, and as is known from the distillation principle, the transformations applied to the master model are of lower intensities and / or fewer in number than those applied to the student model because it is desirable that the master model benefits from an input closer to the original than the student model. For example, the same geometric transformations are applied to both models and to the PL pseudo-labels, on the other hand the visual transformations on the intensities are only applied to the inputs of the student model.

[0088] The weights of the student model are updated by a gradient back-propagation mechanism so as to reconstruct the sequence received as input to the student model SIel in the manner described above for the first embodiment. On the other hand, the weights of the master model are calculated as the moving average of the weights of the student model. The master model thus becomes a more stable version of the student model and produces more reliable results which are used to supervise the student model.

[0089] As detailed in reference [5], the distillation mechanism consists of transferring or distilling the knowledge of the master model into the student model. The type of supervision depends on the task being handled. In the case of supervised object detection, supervision consists of aligning the outputs of the student model with the most reliable predictions of the master model. An external dataset is used in this case to supervise the student model in order to avoid divergence of the master-student system. Distillation can be preceded by a preparation phase called "bum-in" during which the model is trained from external labels before being duplicated into a master model and a student model. The interest of this preparation phase is to provide an initialization that helps the two models converge towards a better solution.

[0090] In the case of the invention, the predictions of the master model MDOma are attention maps noted VL 3, ..., 3, Wbg_

[0091] In the second embodiment of the invention described in Figure 5, it is the attention map of the foreground = 1 - W bg. of the master model MDOma which is used to carry out the FIL filtering operation of the pseudo-labels.

[0092] An advantage of this embodiment lies in the benefit of the distillation mechanism by using the foreground attention map learned by the master model which is more accurate than that of the student model.

[0093] The FIL filtering method described in [Fig.4] remains identical.

[0094] [Fig.6] represents a third embodiment of the invention comprising in in addition to a SUP supervision module to connect the respective attention modules ATTel,ATTma of the two models MDOEL, MDOma.

[0095] The SUP supervision module implements the steps of the supervision method described in [Fig.7].

[0096] In step 701, the attention maps, WK s, of the master model are transformed into binary maps for example using the argmax operator. The values ​​of each attention map which correspond to local maxima are set to 1 while the other values ​​are set to 0. Kl binary masks are then obtained.

[0097] Each of these masks can contain several connected regions, that is to say sets of adjacent pixels having the value 1. In step 702, each connected region is considered as corresponding to a distinct object. When a mask comprises several objects, they are separated into as many distinct masks. Thus, the K1 binary masks are transformed into C object masks where C corresponds to the number of distinct objects.

[0098] The resulting set is then filtered based on the model confidence to retain only the most reliable master model predictions.

[0099] For this, in step 703, a confidence score is calculated for each object mask as the average of the activations of the attention map of the master model from which the object mask is extracted, at the spatial positions of the map corresponding to the spatial positions of the values ​​at 1 of the object mask.

[0100] The confidence score can be represented by the following formula, where c denotes an object mask:

[0101] scorec=^-^ 0 c(j)

[0102] In step 704, the object masks c whose confidence score is greater than a predefined threshold s are kept for the supervision of the attention maps of the student model, the other objects are deleted. Thus, only the most reliable predictions of the master model are retained. The threshold s has a value between 0 and 1, preferably a value close to 1 to effectively filter the least reliable predictions, for example a value equal to 0.9.

[0103] Finally, in step 705, the training of the ATTEL student attention module is supervised by means of a loss function which aims to associate each attention map Wb.. .W ki with one of the objects retained after filtering.

[0104] For example, a binary cross-entropy loss function is used such that:

[0105] L'BCE(c, W) = 1 + ) + ( lc(j) )10g( lW(i) )]

[0106] The score of the object masks is taken into account in the calculation of the loss function so as to give greater weight to objects having higher confidence. high.

[0107] Thus, each attention map of the master model gives rise to several object masks which are distributed between the attention maps of the student model so that each attention map of the student model specializes towards a single object.

[0108] An advantage of this SUP supervision is the consideration of static objects which are identified in the attention maps of the master model whereas they are absent from the PL pseudo-labels which only target mobile objects.

[0109] The distribution of each attention map of the master model to several attention maps of the student model makes it possible to avoid a semantic bias which would exist in a more naive configuration where a direct association would be made between a map of the master model and a map of the student model.

[0110] In one embodiment of the invention, the attention maps of the student model are supervised using a combination of the two loss functions > VU) and L'bceÇc, W), for example the sum of these two functions possibly weighted by coefficients chosen according to the respective importance to be given to supervision by the master model or to supervision by pseudolabels coming from an external source. The weighting coefficients can be applied dynamically so as to progressively give more weight to the supervision carried out by the master model during training. Such an approach makes it possible to strengthen the robustness of the student model by mitigating the impact of errors resulting from external supervision.

[0111] In an alternative embodiment of the invention, the motion information from an optical flow sequence may be replaced or associated with another source of pseudo-labels. For example, the pseudo-labels may be generated by algorithms for discovering objects in images from characteristics of pre-trained models as presented in reference [6].

[0112] Once the student model is trained, it can be used in inference on new image data to locate objects in the captured scene.

[0113] The trained model is executed on a sequence of images and outputs an object localization mask in the scene for each object. The object localization mask is determined by the attention module via the attention maps. Each attention map corresponds to a distinct object. An object localization mask is obtained, for example, by binarizing the activation values ​​of an attention map. In other words, the values ​​of the attention map above an activation threshold are set to 1 in the object localization mask, while the other values ​​are set to 0.

[0114] The invention has several advantages over prior art techniques.

[0115] The third embodiment of the invention allows better localization of static objects by adding a constraint for localizing the static objects discovered via the SUP supervision module which carries out the supervision of the student attention module from the master attention module. Conversely, the object discovery techniques of the prior art based on “slot attention” approaches do not constrain the model to the discovery of static objects which results in limited localization performance of these static objects.

[0116] The third embodiment of the invention also allows the localization of a greater number of objects, in particular those which are difficult to capture via the methods of the prior art. Indeed, the methods which are based solely on supervision via pseudo-labels obtained by an external source depend on the quality of these pseudo-labels which are most often incomplete. The invention makes it possible to iteratively complete the supervision labels during training thanks to the distillation process.

[0117] The invention also makes it possible to better distinguish moving objects from noise segments in the pseudo-labels thanks to an automatic filtering mechanism which specifically targets the noise and which makes it possible to efficiently eliminate irrelevant segments in the model outputs.

[0118] The methods of the prior art rely exclusively on segments from an optical flow sequence for model supervision, which has limitations, particularly when nearby objects move at the same speed and direction, leading to fusion errors. Conversely, the third embodiment of the invention uses two sources of supervision (the segments of moving objects and the predictions of the master model), allowing the regularization (correction) of errors in the first source thanks to the predictions of the master model which, by learning to generalize, becomes more robust to these errors. This results in better separation of nearby objects.

[0119] The invention may be implemented as a computer program comprising instructions for its execution. The computer program may be recorded on a recording medium readable by a processor.

[0120] Reference to a computer program that, when executed, performs any of the functions described above, is not limited to an application program running on a single host computer. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instruction) that can be used to program one or more processors to implement aspects of the techniques described herein. The computing means or resources may in particular be distributed ("Cloud computing"), possibly using peer-to-peer technologies. The software code may be executed on any suitable processor (e.g., a microprocessor) or processor core or a set of processors, whether provided in a single computing device or distributed among several computing devices (e.g., as may be accessible in the device environment). The executable code of each program enabling the programmable device to implement the processes according to the invention may be stored, for example, in the hard disk or in read-only memory. Generally, the program(s) may be loaded into one of the storage means of the device before being executed.The central unit can control and direct the execution of the instructions or portions of software code of the program(s) according to the invention, instructions which are stored in the hard disk or in the read-only memory or in the other aforementioned storage elements.

[0121] The invention can be implemented on a computing device based, for example, on an embedded processor. The processor can be a generic processor, a specific processor, an application-specific integrated circuit (also known as an ASIC for "Application-Specific Integrated Circuit") or an in situ programmable gate array (also known as an FPGA for "Field-Programmable Gate Array"). The computing device can use one or more dedicated electronic circuits or a general-purpose circuit. The technique of the invention can be carried out on a reprogrammable computing machine (a processor or a microcontroller for example) executing a program comprising a sequence of instructions, or on a dedicated computing machine (for example a set of logic gates such as an FPGA or an ASIC, or any other hardware module). References

[0122] [1] Bao, Zhipeng et al. “Discovering Objects that Can Move.” 2022 IEEE / CVF Conférence on Computer Vision and Pattern Récognition (CVPR) (2022): 11779-11788.

[0123] [2] Bao, Zhipeng, et al. "Object Discovery from Motion-Guided Tokens." Pro- ceedings of the IEEE / CVF Conférence on Computer Vision and Pattern Récognition. 2023.

[0124] [3] Locatello, Francesco, et al. "Object-centric learning with slot attention." Advances in Neural Information Processing Systems 33 (2020): 11525-11538.

[0125] [4] Kara, Sandra, et al. "The Background Also Matters: Background-Aware Motion- Guided Objects Discovery." Proceedings of the IEEE / CVF Winter Conférence on Applications of Computer Vision. 2024.

[0126] [5] Liu, Yen-Cheng, et al. "Unbiased teacher for semi-supervised object détection." arXiv preprint arXiv:2102.09480 (2021).

[0127] [6] Siméoni, Oriane, et al. "Localizing objects with self-supervised transformers and no labels." arXiv preprint arXiv:2109.14279 (2021).

Claims

Claims

1. A computer-implemented method for training an automatic model (MDO.MDOu) for discovering objects in an input image sequence (SI), the model comprising: - an encoder (ENC,ENCel) for encoding each image into a first feature vector, - an attention module (ATT,ATTEL) configured to transform the first feature vector into a plurality of feature vectors called slots (Si,..SK i), the state of a slot being determined from a similarity calculation between the first feature vector and each slot in its current state, each similarity calculation defining an attention map (Wi,..WK-i), - a decoder (DEC) for decoding all the slots so as to reconstruct a sequence of images corresponding to the input image sequence (SI), - training the attention maps (Wi,..WK-i) being supervised (404) by a set of binary masks for discovering moving objects produced by an external source, called pseudo-labels (PL), such that each attention map (Wi „.WK-i) is activated in an area corresponding to a distinct object contained in the pseudo-labels (PL), an additional attention map (Wbg) is activated in an area corresponding to the background of the image, - the pseudo-labels (PL) being filtered (FIL) by means of the steps of: determining (401) an attention map of the foreground of the image (Wfg), calculating (402) a confidence score from the average of the values ​​of the attention map (Wfg) of the foreground of the image at the positions of each moving object present in a pseudo-label, filtering (403) the moving objects of the pseudo-labels for which the confidence score is lower than a predefined threshold,.

2. A method for training an automatic object discovery model according to claim 1 wherein the attention map (Wfg) of the foreground of the image is determined from the attention map (Wbg) of the background image of the model's attention module (ATT,ATTEL).

3. A method for training an automatic object discovery model according to claim 1 wherein said model is a student model (MDOel) trained at least partially via a distillation learning transfer mechanism from a master model (MDOma), the master model (MDOma) comprising an encoder (ENCMA) and an attention module (ATTma), the attention map (Wfg) of the foreground of the image of the student model being determined from the attention map of the background of the image of the attention module of the master model.

4. A method for training an automatic object discovery model according to claim 3 wherein the training of the attention maps of the student model is supervised by the attention maps of the master model such that each attention map is activated in an area corresponding to a distinct object discovered in the attention maps of the master model.

5. A method for training an automatic object discovery model according to claim 4, wherein the training of the attention maps of the student model comprises the steps of: - Binarizing (701) each attention map of the master model, - Determining (702) the set of connected regions in the set of binarized attention maps, each connected region corresponding to a distinct object discovered

6. A method for training an automatic object discovery model according to claim 5 wherein the training of the attention maps of the student model further comprises the steps of: - Calculating (703) a confidence score for each distinct object discovered as being equal to the average value of the activations of said object in each attention map of the master model, - Filtering (704) the objects whose confidence score is lower than a predetermined threshold

7. Method for training an automatic discovery model of objects according to claim 6 in which the supervision (705) of the attention maps of the student model is at least carried out by means of a first cross-entropy loss function applied between the attention maps of the student model and the objects determined from the attention maps of the master model weighted by their confidence score.

8. Method for training an automatic object discovery model according to any one of the preceding claims in which the supervision (404, 705) of the attention maps of the model is at least carried out by means of a second cross-entropy loss function applied between the attention maps of the model and the objects of the pseudo-labels weighted by their confidence score.

9. A method of training an automatic object discovery model according to any preceding claim wherein the pseudo-labels are obtained from the image sequence and an associated optical flow sequence.

10. A computer-implemented method for discovering objects in a sequence of images comprising the steps of: - Receiving a sequence of images, - Executing the automatic object discovery model trained using the method according to any one of the preceding claims for the sequence of images so as to generate at least one location mask of an object in the sequence of images, each location mask being obtained from an attention map.

11. A computer program comprising instructions for executing the method according to any one of claims 1 to 10, when the program is executed by a processor.

12. A processor-readable recording medium having recorded thereon a program comprising instructions for executing the method according to any one of claims 1 to 10, when the program is executed by a processor.

Images