Unsupervised learning method for an optical flux estimation model

Abstract

A computer-implemented method for unsupervised learning of an optical flux model from a sequence of images, the method comprising the joint learning of a first machine learning model configured to estimate the optical flux F1,F2 between two successive images and a second machine learning model configured to estimate a difference in light intensity C1,C2 between the corresponding pixels of the two successive images, the joint learning being performed by optimizing at least a first cost function Lph1 characteristic of an error between the first image I1 of a pair of successive images I1,I2 and a reconstructed version of the first image I1 obtained by: Applying the difference in light intensity C2 estimated between the second image I2 and the first image I1, to the second image I2, so as to obtain a second image compensated for the difference in light intensity C2,Applying the estimated optical flux F1 between the first image I1 and the second image I2 to the second compensated image, so as to obtain the reconstructed version of the first image I1. Figure 4,

Description

Title of the invention: Unsupervised learning method for an optical flow estimation model

[0001] The invention relates to the field of computer vision and in particular to methods for estimating optical flow for a sequence of images. The invention also relates to the field of deep learning, in particular unsupervised learning.

[0002] The invention relates to a novel method for unsupervised learning of an optical flow estimation model.

[0003] In the field of computer vision, a commonly implemented task concerns the estimation of optical flow. Measuring optical flow consists of estimating the relative movement of each pixel in a given scene between several successive instants. In other words, optical flow corresponds to the displacement of each pixel belonging to an object in a scene between two successive images.

[0004] Estimation of the optical flow function is a common function in the field of image processing, computer vision in order to implement motion detection functionalities or segmentations of moving objects relative to the background or other video processing techniques, for example object tracking or anomaly detection.

[0005] Traditional methods for estimating optical flow are based on optimization problems. However, more recent methods are based on deep learning approaches that require training an artificial neural network on a training data set.

[0006] The principle of optical flow estimation methods by machine learning consists of learning a function that minimizes a cost function characteristic of an error between an estimated optical flow and a ground truth. We speak of supervised learning when the cost function takes as an argument the ground truth of the optical flow.

[0007] Supervised learning methods require a large amount of training data to define the ground truth. A disadvantage of these methods is that it is difficult to generate training data accompanied by the ground truth of the optical flow, i.e., data annotated with the actual value of the optical flow. Indeed, annotating video sequences to generate training data associated with the ground truth of the optical flow is a very expensive process.

[0008] There are also unsupervised learning methods that do not require knowledge of the ground truth of the optical flow. These methods present an al-

[0009]

[0010]

[0011]

[0012]

[0013] alternative to solve the problem of annotating training data to provide the ground truth of optical flow or in other words the problem of obtaining the ground truth of optical flow associated with pairs of images of the training data These methods are based on the optimization of a so-called photometric error function which aims to compare an image with its reconstruction from the successive image to which the optical flow function is applied allowing the displacement of the pixels to return to the original image. Reference [1] gives an example of such a method. A disadvantage of this method is that it is not completely robust to changes in light intensity between two successive images. Thus, a pixel belonging to the same object in two successive images may be affected by a change in intensity which may be linked to a shadow, a reflection, or even a change in sensor orientation. These intensity changes can distort the estimation of optical flow and therefore there is a need for a new unsupervised machine learning method that is more robust to intensity changes between two successive images. The invention is based on the joint learning of two learning models, the first being dedicated to learning optical flow while the second is dedicated to learning a map of intensity changes between corresponding pixels of two successive images. The learned intensity changes can thus be taken into account to correct the loss function of the learning model intended for optical flow. The invention relates to a computer-implemented method for unsupervised learning of an optical flow model from a sequence of images, the method comprising the joint learning of a first machine learning model configured to estimate the optical flow Fi,F2 between two successive images and a second machine learning model configured to estimate a difference in light intensity Ci,C2 between the corresponding pixels of the two successive images, the joint learning being carried out by optimizing at least a first cost function Lphi characteristic of an error between the first image h of a pair of successive images Ii,I2 and a reconstructed version jè of the first image L obtained in: - Applying the estimated light intensity difference C2 between the second image I2 and the first image L, to the second image I2 so as to obtain a second image I2 compensated for the light intensity difference C2, - Applying the estimate of the optical flow Fi between the first image L and the second image I2 to the second image J| compensated, so as to obtain the reconstructed version of the first IL image

[0014] According to a particular aspect of the invention, the joint learning is carried out by op timization of a first cost function Lphi characteristic of an error between the first image Ii of a pair of successive images IbI2 and a reconstructed version of the first image Ii and of a second cost function Lph2 characteristic of an error between the second image I2 and a reconstructed version of the second image I2.

[0015] According to a particular aspect of the invention, the second machine learning model receives as input one of the images of the pair of two successive images and an estimated version of said image by applying the estimate of the optical flow to the other image of said pair.

[0016] According to a particular aspect of the invention, the second automatic learning model receives as input an occlusion map ObO2 identifying, for each image of the pair of successive images, the pixels also visible in the other image of said pair and the at least one first cost function is weighted by the occlusion map.

[0017] According to a particular aspect of the invention, the at least one first cost function Lphi is decomposed into a first term Lfphi specifically dedicated to the optimization of the first automatic learning model of the optical flow and a second term Lcphi specifically dedicated to the optimization of the second automatic learning model of the difference in light intensity, the first term Lfphi being characteristic of an error between the first image h of a pair of successive images IbI2 and a combination of the reconstructed version of the first h-image and an estimated version of the first image by applying the estimated optical flow Fl to the second image I2, said combination being obtained by selecting, for each pixel, that of the reconstructed version and the estimated version which is closest to the first image Ib - the second term Lcphi being characteristic of an error between the first image Ii and the reconstructed version of the first image Ib

[0018] According to a particular aspect of the invention, the first machine learning model and / or the second machine learning model consist of at least one convolutional neural network.

[0019] The invention also relates to a computer program comprising ins instructions for executing the method according to the invention, when the program is executed by a processor.

[0020] The invention also relates to a recording medium readable by a processor on which is recorded a program comprising instructions for the execution of the method according to the invention, when the program is executed by a processor.

[0021] The invention also relates to a device for measuring optical flow from a sequence of images comprising an image acquisition sensor and a processing unit configured to implement the optical flow model estimated by means of the steps of the method according to the invention for a sequence of images acquired by said sensor.

[0022] Other characteristics and advantages of the present invention will appear better on reading the description which follows in relation to the following appended drawings.

[0023] [Fig-1] represents a diagram illustrating an unsupervised learning method of an optical flow according to the prior art,

[0024] [Fig.2] represents a block diagram of an unsupervised optical flow learning engine according to an embodiment of the invention,

[0025] [Fig.3] represents a diagram of an example of architecture of an automatic learning engine of intensity differences according to an embodiment of the invention,

[0026] [Fig.4] represents a diagram of the general architecture of the method according to the invention,

[0027] [Fig.5] represents a diagram of a device configured to execute the invention,

[0028] We first present the general principle of estimating an optical flow by an unsupervised learning method according to the prior art. Such a method is for example described in reference [5].

[0029] We denote by T — {Vj. a set of video sequences, each video sequence being an ordered temporal sequence of images denoted by y. — where is an image of dimensions (H,W,C), H being the height of the image, W its width and C the number of channels of the image. For example, for an image in RGB format, C is equal to 3.

[0030] The images of the set T can be rearranged into the union of consecutive images. If we denote (Ii,I2) a pair of consecutive images of this set, the estimation of the optical flow from L to I2 consists of determining the optical flow map in the form of a matrix Fb of dimension (H,W,2) which gives the relative movement in 2D of each pixel of R with respect to the corresponding pixel in I2. This optical flow matrix captures the rigid and non-rigid movements of the objects and sensors.

[0031] This function can be estimated by a machine-learned model, such as a neural network or any other artificial intelligence algorithm: tp (1^, 1^) =

[0032]

[0033]

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040]

[0041]

[0042]

[0043] , with 0 the parameters of the model or network that must be learned. The optical flow Fi can be estimated in the forward temporal direction, i.e. from Ii to I2 or in the reverse direction, i.e. from I2 to Ip. In both cases, it is the same function to be learned from the point of view of the neural network. Considering a set of unannotated images, i.e. without ground truth allowing to know the associated optical flow, we can apply an unsupervised learning method to determine an approximation of the optimal parameters 0* of the network as being those which minimize a cost function or global error function L(0) which can be formulated via the following equation (1): M») =yplL,MW (i) The first term Lp^O} is a photometric error function, The second term L 0) is a so-called smoothing error function, The third term 0) is a so-called self-supervision error function. The terms LpA' Tw V self are the respective weights of the three functions. Each of the error functions can be calculated in the forward or reverse time direction. Subsequently, only the forward time direction is presented for simplicity; the reverse direction is deduced identically. The photometric error function is characteristic of an error between an image and its reconstruction obtained from the estimated optical flow. In other words, by applying the optical flow Fi to the image I2, we obtain a reconstructed image L designated . The photometric error function Lph\ ( 0 ) consists of calculating an error between L and by means of a predefined distance, for example a distance L1 or L2 or a “Census Loss” type distance as introduced in reference [5]. A disadvantage of the photometric error function is that it is not sufficiently robust to all types of intensity changes that may occur between the h-image and the I2-image, resulting in amplitude changes on the pixels that are not due to the optical flow function, i.e., simple pixel displacement, but to other causes that impact the amplitude. More precisely, the photometric error function can be robust to some intensity changes by properly choosing an appropriate distance to compare the two images, but this distance is never insensitive to all light intensity changes. For example, a distance L1 or L2 is sensitive to any light intensity change, while a distance such as the "Census Loss" distance is robust to some locally uniform light intensity changes but not to all types of light intensity changes. The other two error functions L((?), are optional and can be omitted from equation (1).

[0044] The smoothing error function makes it possible to prioritize the fact that pixels belonging to the same object must theoretically present the same local displacement in the optical flow.

[0045] The self-supervision error function addresses the problem of pixels that go outside the image boundaries between two successive images.

[0046] Figure 1 illustrates the principle of the method according to the prior art described above. The photometric error function Lph is calculated from an error between R and In the example of [Fig.l], the images Ii,I2 correspond to two successive views of a car on a road. Between the two shooting times, the rear part of the car is impacted by a change in light intensity due to a shadow. This change impacts the reconstruction error ER produced by the photometric error function Lph by introducing a defect D into the reconstruction.

[0047] The invention aims to improve the learning of the optical flow model as described above by introducing a second model of the intensity changes between R and I2 which is learned jointly with the optical flow. The estimated intensity changes are taken into account in the photometric error function in order to correct them. An advantage of the proposed method is that it makes it possible to prevent the photometric error function from detecting errors related to intensity changes as being due to a poor estimation of the optical flow.

[0048] Figures 2 and 4 represent, on two principle diagrams, the operation of the unsupervised learning method according to one embodiment of the invention.

[0049] [Fig. 2] illustrates the principle of the learning engine or joint learning network RC which consists of a first automatic learning engine of the optical flow RFO of parameters 0 and a second automatic learning engine of the intensity differences RDI of parameters 0C. Each of the two learning engines can be implemented by means of one or more neural networks, for example a convolutional neural network or a “transformer” type network, for example using neural network architectures of the RAFT, GMA or SCV types as described in references [2], [3] or [4]. In particular the architecture of figure 7 of appendix A of document [2] can be used.

[0050] In particular, the second RDI learning engine may be implemented using an encoder and an upsampler as described in the RAFT architecture [2]. The architecture may comprise one or more convolutional neural networks.

[0051] [Fig.3] represents, on a diagram, a non-limiting example of architecture of the second RDI learning engine.

[0052] According to this example, the RDI learning engine comprises an ENC encoder followed by a module for generating a COR correction map and an SUR oversampling module.

[0053] The ENC encoder is composed of several layers of artificial neural networks in series. Each convolution layer is followed by an instance normalization layer. At the input of the encoder, a first convolution layer 501 receives concatenated images at a given resolution HxW with 3 RGB channels for the first image, 3 RGB channels for the second image and, optionally, 1 channel for the occlusion map, i.e. 6 or 7 channels in total. In this example, the first layer 501 uses a convolution kernel of size 7x7 with a shift parameter or "stride" in English equal to 2 in order to increase the size of the receptive field of the neurons of the following layers while reducing the computational cost. The first layer 501 provides 64 output channels corresponding to characteristics or "features" in English. Several residual units 502-507 are then arranged in series.Each residual unit consists of two 3x3 convolution layers with different stride parameters.

[0054] More precisely, the residual units 502, 503, 505, 507 are composed of two convolution layers using a convolution kernel of size 3x3, an offset parameter equal to 1 and as many input and output channels. The number of input and output channels are identified in [Fig.3] for each residual unit. Units 502 and 503 receive 64 input channels and produce 64 output channels. Unit 505 receives 96 input channels and produces 96 output channels. Unit 507 receives 128 input channels and produces 128 output channels. In addition, the input of the first layer is summed with the output of the second layer.

[0055] The residual units 504 and 506 are composed of two convolution layers using a convolution kernel of size 3x3. The first layer uses an offset parameter equal to 2 and returns more channels as output than it receives as input while the second layer has an offset parameter equal to 1 and as many channels as input. The number of input and output channels are identified in [Fig.3] for each residual unit. Unit 504 receives 64 channels as input and produces 96 as output. Unit 506 receives 96 channels as input and produces 128 as output. In addition, the output of the second layer is summed with the input of the first layer to which another convolution layer has been applied using a convolution kernel of size 1x1 and an offset parameter equal to 2.In order to sum data of the same dimension, the number of input and output channels of this layer is the same as that of the residual unit in which it is located.

[0056] The architecture of [Fig.3] makes it possible to reduce the dimensionality of the input data as well as the cost due to the application of the offset parameter equal to 2 for 3 convolution layers, thus reducing the computational cost. As a result, the last convolution layer 508 (which receives 128 input channels, produces 256 output channels and uses a convolution kernel of size 1x1 and an offset parameter of 1) generates data of dimension H / 8xW / 8, i.e. reduced by a factor of 8 in height and width compared to the dimension of the input images.

[0057] To return to an intensity correction map applicable to an image at HxW resolution, an oversampling module SUR is applied to generate oversampling coefficients. It is composed of a first convolution layer with a Relu activation function, 256 input channels, 256 output channels and using a 3x3 dimension convolution kernel with an offset parameter equal to 1, followed by a second convolution layer with a SoftMax activation function, 256 input channels, 8x8x9 output channels and using a 1x1 dimension convolution kernel with an offset parameter equal to 1. The number of output channels corresponds to the 64 weights to be applied to the values ​​of the pixels of the intensity correction map in a neighborhood of size 3x3 pixels in order to oversample each pixel of this map into 8x8 new pixels.

[0058] The COR module allows for the generation of an intensity correction map of dimension H / 8xW / 8x3. It is composed of a first convolution layer with a Relu activation function, 256 input channels, 256 output channels and using a convolution kernel of dimension 3x3 with an offset parameter equal to 1, followed by a second convolution layer with a hyperbolic tangent activation function, 256 input channels, 3 output channels and using a convolution kernel of dimension 1x1 with an offset parameter equal to 1. The hyperbolic tangent function is chosen instead of the ReLU function to be able to generate positive or negative intensity correction values ​​in the entire range of intensity values ​​that the pixels can take.

[0059] The output of the COR module corresponds to the intensity correction values ​​for an RGB image (3 channels) of size H / 8xW / 8. To obtain the correction values ​​for the image at the HxW dimension, each pixel of the map at the output of the COR module must be transformed into 8x8 pixels. To do this, the weighting coefficients obtained at the output of the SUR module are applied to the output of the COR module. More precisely, each value of the final correction map is equal to a weighted sum over a 3x3 neighborhood of the values ​​of the map subsampled by the oversampling coefficients.

[0060] The example architecture given in [Fig.3] is provided for illustrative purposes and is not limiting. For example, if all the shift parameters (stride) are taken equal to 1 In the encoder, the SUR upsampling module is made optional. Generally speaking, any artificial intelligence model, in particular using one or more convolutional neural networks or "transformers" can be used to implement the RFO and RDI learning networks.

[0061] The first RFO learning network receives as input a pair of images (R,I2) composed of two successive images from a sequence of images and learns the function Fi which corresponds to the optical flow making it possible to pass from R to I2.

[0062] The second RDI learning network receives as input the image R and the reconstructed image f by applying the function Fi estimated by the first RFO network to the second image I2 to deform it (via a deformation function DEF) in order to obtain a reconstructed version of the first image. The deformation function DEF consists of moving the pixels of I2 according to the optical flow defined by Fi. Optionally, the second RDI network also receives as input an occultation map Oi which is a binary mask which is equal to 1 for each pixel belonging to an observation point which is visible both in R and in I2 and which is equal to 0 if it is only visible in the image R. The occultation map makes it possible to take into account in the optimization the occultations of one object by another which occur between the two images.The second RDI network learns the intensity correction map Ci of dimensions (H,W,C) which contains the values ​​to be applied to compensate for intensity changes from R to I2 for the pixels visible in both images. This involves estimating the intensity changes between two pixels corresponding to the same points of interest in both images.

[0063] The principle diagram of the joint learning network RC is applicable to estimate Fi and Ci in the direct temporal direction but also to estimate F2 and C2 in the reverse direction, it is then sufficient to reverse the order of the images R,I2 at the input of the two networks.

[0064] A first possibility for calculating the photometric error function consists of estimating an error between R+Ci and f for example by means of equation (2):

[0065] = || 0^^1, + C^ It ! It O, It , (2)

[0066] 0 are the parameters of the first RFO network,

[0067] 0C are the parameters of the second RDI network,

[0068] 0 denotes the term-by-term product operator between two matrices, also called Hadamard product,

[0069] P is an error function, for example a mean square error or more advantageously an error allowing a divergence between two images to be measured. For example, this error function can be defined by the following relation: 100701

[0071] A and B are two images of the same dimensions, p denotes the coordinate of a pixel, N(p) is the set of pixels neighboring pixel p.

[0072] H(z) — the soft Hamming distance

[0073] C(z) - zI

[0074] e' e are parameters, for example equal respectively to a = 0.4, ¢ = 0.1, 6 = 0.81

[0075] || || denotes the L1 norm or squared error

[0076] Using the photometric error function of equation (2), the intensity change correction map Ci is used to minimize this error function Lphl. However, since the images h and are provided as input to the second RDI network, the latter simply needs to estimate the difference between these two images. In the end, this optimization is not satisfactory and can lead to a poor estimation of the optical flow Fb

[0077] To improve this point, it is proposed to modify equation (2) into equation (4): 100781 Il || 71 o. || ,(4)

[0079] is a reconstructed version of the image h which is obtained by the following operation:

[0080] / J =

[0081] w is the deformation function DEF which consists of the following operation:

[0082] w(I2, ^i) (p) ~ Fi(p)) ^P + F^p] is inside the dimensions of I2 and is (0,0,0) otherwise.

[0083] Thus, equation (4) requires estimating both the optical flow and the intensity change map in the direct temporal direction but also in the indirect direction.

[0084] This is illustrated in [Fig.4] which shows the principle of calculating the photometric error function in the forward Lphi and reverse Lph2 time directions.

[0085] In [Fig.4], the joint network RC is represented twice but it corresponds to one and the same network, which is the one described in [Fig.2], but which is trained respectively to estimate Fi,Ci on the one hand and F2,C2 on the other hand.

[0086] In the equation for calculating the photometric error function, I2 is replaced by I2 +C2 so the calculation requires both the estimation of the optical flow in the forward and reverse time directions.

[0087] In one embodiment of the invention, the photometric error function Lph is equal to the sum of the two functions Lph= Lphi+ Lph2 but it can also be equal to one or the other of the two functions Lph= Lphi or Lph= Lph2.

[0088]

[0089]

[0090]

[0091]

[0092]

[0093]

[0094] The global error function of equation (1) is used to estimate the parameters of the two networks RFO, RDI jointly during a back-propagation phase of the error calculated using equation (1) in order to update the weights of the networks. This training process is not described here because it is part of the general knowledge of those skilled in the art. This process consists of calculating the parameters of a neural network using an optimization algorithm comprising a phase inference phase in which the input data is propagated into the network to calculate the error function and then, in a back-propagation phase, the error is back-propagated to calculate the weights of each layer of the neural network at using a gradient descent algorithm. The solution proposed by the invention makes it possible to improve the prediction of optical flow because the estimation is detached from the model calculation graph in the inputs of the second RDI network. Thus, the optimization of the / V / ) model does not impact that of the optical flow model y^.0). More precisely, depends on Fi so by construction, is attached to the graph of calculation of 0). This implies that errors in the photometric cost function related to C2 will be back-propagated into the estimates of y^G) and ​RFO network may be modified. This is not the desired behavior since the second RDI network aims to take into account reconstruction errors which are not taken into account by the estimation of optical flow (i.e. the changes in light intensity). The underlying assumption is that the corrections are estimated from a correctly estimated flow. The errors of the corrector do not should not be back-propagated in the first RFO network (because its predictions are good by assumption). In order to prevent this back-propagation, must be detached of the calculation graph of model 7( / / ). The calculated photometric error is back-propagated from the optical flow Fi and C2 corrections estimates but not from the optical flow F2. It applies for each prediction of the first optical flow calculation network RFO. The intensity changes are instead estimated by the second RDI network from the last optical flow prediction provided by the first RFO network. In an alternative embodiment of the invention, a variant of calculating the photometric error function is proposed which makes it possible to further improve the estimates. In the case of an error in estimating the optical flow F2, reconstructing the image I2 by applying the optical flow F2 to the image L can generate reconstruction errors, even in the non-occluded parts of the image. Thus in such a situation, pixels that do not correspond to the same points of interest can have the same

[0095]

[0096]

[0097]

[0098]

[0099]

[0100]

[0101]

[0102]

[0103]

[0104]

[0105]

[0106] coordinates in I2 and These reconstruction errors can then impact the estimation of the C2 intensity correction map. In order to minimize the impact of such errors in the photometric error function, a modification to the latter is proposed. According to this proposed variant, only the pixels that minimize a reconstruction error p' are retained. More precisely, the photometric error function is separated into two functions respectively specifically dedicated to the optimization of the parameters of the first RFO network and the second RDI network. 0.)=^ (») 4« () = Il 0,0^,0?,-Il ' Il °1 II I Mi is a matrix of the same dimensions as h which is equal to 1 for each pixel which verifies p^f^jjj and otherwise- And M] is the complement of Mb In other words, the first term dedicated to the optimization of the first network RFO calculates an error between L and a combination between and hh ïpM(0< )= Il OPpfôh) Il Z| l °1 II! p' is an error function which is, for example, equal to the Lh norm The invention makes it possible to reduce errors in estimating optical flow compared to prior art methods based on the use of a photometric error function and thus promotes better learning. Indeed, the invention estimates changes in intensity between successive images, which makes it possible to avoid errors in estimating optical flow that would be linked to such changes. The invention can be implemented for any type of neural network architecture capable of estimating a task of interest without the need to modify its architecture but only the calculation of the error function and the introduction of a second model for learning the intensity changes. The invention requires the training of a second RDI neural network but the latter is only used for training the first RFO network. Once the optical flow function is learned, the first RFO network can be used alone, during an inference phase, to estimate an optical flow on a video sequence of interest. The implementation complexity in terms of computational cost therefore remains identical for the inference phase. [Fig.5] represents, in a simplified diagram, an optical flow measurement device DFO comprising an image acquisition sensor CAI, for example a camera which captures a sequence of images I and provides it as input to a unit of UT processing which is configured to execute the method according to any one of the embodiments of the invention in order to estimate an optical flow F.

[0107] The invention may be implemented as a computer program comprising instructions for its execution. The computer program may be recorded on a recording medium readable by a processor.

[0108] Reference to a computer program that, when executed, performs any of the functions described above, is not limited to an application program running on a single host computer. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instruction) that can be used to program one or more processors to implement aspects of the techniques described herein. The computing means or resources may notably be distributed (“Cloud computing”), possibly using peer-to-peer technologies.The software code may be executed on any suitable processor (e.g., a microprocessor) or processor core or set of processors, whether provided in a single computing device or distributed among several computing devices (e.g., as may be accessible in the device environment). The executable code of each program enabling the programmable device to implement the processes according to the invention may be stored, for example, in the hard disk or in read-only memory. Generally, the program(s) may be loaded into one of the storage means of the device before being executed. The central unit may control and direct the execution of the instructions or portions of software code of the program(s) according to the invention, instructions which are stored in the hard disk or in read-only memory or in the other aforementioned storage elements.

[0109] The joint neural network RC can be implemented on a computing device based, for example, on an embedded processor. The processor can be a generic processor, a specific processor, an application-specific integrated circuit (also known as an ASIC for "Application-Specific Integrated Circuit") or an in situ programmable gate array (also known as an FPGA for "Field-Programmable Gate Array"). The computing device can use one or more dedicated electronic circuits or a general-purpose circuit. The technique of the invention can be carried out on a reprogrammable computing machine (a processor or a microcontroller for example) executing a program comprising a sequence of instructions, or on a dedicated computing machine (for example a set of logic gates such as an FPGA or an ASIC, or any other hardware module). References

[0110] [ 1 ] F. Stein, «Efficient computation of optical flow using the census transform.» Joint Pattern Récognition Symposium, 2004.

[0111] [2] Zachary Teed and Jia Deng. Raft: Récurrent all-pairs field transforms for optical flow. In ECCV, 2020

[0112] [3] Shihao Jiang et al, Learning to estimate hidden motions with global motion ag- gregation. In ICCV,2021

[0113] [4] Shihao Jiang et al, Learning optical flow from a few matches. In CVPR, 2021

[0114] [5] Stone, Austin, et al. "SMURF: Self-teaching multi-frame unsupervised RAFT with full-image warping." Proceedings of the IEEE / CVF Conférence on Computer Vision and Pattern Récognition. 2021

Claims

Claims

1. A computer-implemented method for unsupervised learning of an optical flow model from a sequence of images, the method comprising jointly learning a first machine learning model (RFO) configured to estimate the optical flow Fi,F2 between two successive images and a second machine learning model (RDI) configured to estimate a difference in light intensity Ci,C2 between the corresponding pixels of the two successive images, the joint learning being carried out by optimizing at least a first cost function Lphi characteristic of an error between the first image fl of a pair of successive images Ii,I2 and a reconstructed version of the first image fl obtained by: - ​​Applying the difference in light intensity C2 estimated between the second image I2 and the first image Ib to the second image I2 so as to obtain a second image compensated for the difference in light intensity C2,- Applying the estimate of the optical flow Fi between the first image fl and the second image I2 to the second compensated image, so as to obtain the reconstructed version of the first image IL,

2. Method for unsupervised learning of an optical flow model according to claim 1 in which the joint learning is carried out by optimizing a first cost function Lphi characteristic of an error between the first image fl of a pair of successive images IbI2 and a reconstructed version of the first image fl and a second cost function Lph2 characteristic of an error between the second image I2 and a reconstructed version f of the second image I2.

3. Method for unsupervised learning of an optical flow model according to any one of the preceding claims in which the second machine learning model (RDI) receives as input one of the images of the pair of two successive images and an estimated version of said image by applying the estimate of the optical flow to the other image of said pair.

4. Method for unsupervised learning of an optical flow model according to any one of the preceding claims in which the second machine learning model (RDI) receives as input an occlusion map Oi,O2 identifying, for each image of the pair of successive images, the pixels also visible in the other image of said pair and the at least one first cost function is weighted by the occlusion map.

5. Method for unsupervised learning of an optical flow model according to any one of the preceding claims in which the at least one first cost function Lphi is decomposed into a first term Lfphi specifically dedicated to the optimization of the first automatic learning model (RFO) of the optical flow and a second term Lcphi specifically dedicated to the optimization of the second automatic learning model (RDI) of the difference in light intensity, - the first term Lfphi being characteristic of an error between the first image R of a pair of successive images Ii,I2 and a combination of the reconstructed version of the first image R and an estimated version ? of the first image by applying the estimate of the optical flow Fl to the second image I2, - said combination being obtained by selecting, for each pixel,that of the reconstructed version and the estimated version f which is closest to the first image I n - the second term Lcphi being characteristic of an error between the first image h and the reconstructed version jè of the first image Ib,

6. Method for unsupervised learning of an optical flow model according to any one of the preceding claims wherein the first machine learning model (RFO) and / or the second machine learning model (RDI) consist of at least one convolutional neural network.

7. A computer program comprising instructions for executing the method according to any one of claims 1 to 6, when the program is executed by a processor.

8. A processor-readable recording medium having recorded thereon a program comprising instructions for executing the method according to any one of claims 1 to 6, when the program is executed by a processor.

9. Device for measuring optical flow (DFO) from a sequence of images comprising an image acquisition sensor (CAI) and a processing unit (UT) configured to implement the optical flow model estimated by means of the steps of the method according to any one of claims 1 to 6 for a sequence of images acquired by said sensor.

Images