Method and system for training neural network to derive object perspective using unlabeled paired images

Abstract

The present invention relates to a method and system for training a neural network using unlabeled paired images to derive the viewpoint of an object. The method comprises minimizing the following: for each training image in a first set of training images, the difference between the output of the neural network and the viewpoint of the training image; for each pair of training images in a second set of training image pairs, the difference between the second image in each pair of training images in the second set of training image pairs and the output of a decoder neural network when the first image in the pair of training images is input to an encoder neural network, the second image in the pair of training images is input to the neural network to obtain the viewpoint, the obtained encoded image is rotated according to the viewpoint, and the rotated encoded image is decoded.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing using neural networks, such as image processing using neural networks. More specifically, the present disclosure relates to a neural network capable of detecting the perspective / viewpoint of an object visible in an image. Background Art

[0002] It has been proposed to detect three-dimensional objects in images captured by a camera / camcorder by using a neural network implemented on a computer system. It is often desirable to also obtain information related to the 6D pose of objects visible in the image. "6D pose" is an expression well known to those skilled in the art that represents a combination of the three-dimensional position and three-dimensional orientation / position of an object. Obtaining the 6D pose is particularly useful in the field of robotics, where objects are detected and manipulated. It is also useful to determine the orientation of objects in a driving scenario to allow autonomous or partially autonomous driving.

[0003] The viewing angle of an object visible in an image is one such piece of information related to the desired 6D pose. The viewing angle can be defined as the azimuth, elevation, and in-plane rotation of the object relative to the camera used to acquire the image. Neural networks have also been used to automatically determine the viewing angle of objects visible in an image.

[0004] In order to obtain a neural network capable of automatically determining the perspective of an object visible in an image, a training phase of the neural network must be performed. This training phase is usually performed using a set of labeled (labeled) training images. "Labeled" means that for each image in the training set, the perspective of the object of interest is provided; the provided perspectives are called ground truth. Training then consists in inputting images from the training set to the neural network, comparing the output of the neural network with the corresponding ground truth, and adapting the parameters of the neural network based on this comparison (for example using the well-known stochastic gradient method).

[0005] As is well known in the art, a large number of labeled training images are required to obtain a good training of the neural network. Large-scale labeled datasets have become an important driving force for the latest progress in computer vision tasks. However, annotating data is expensive (i.e., time-consuming) and cannot be extended to the growing body of complex visual concepts. In practice, obtaining the ground truth / labeling usually involves using dedicated hardware, a controlled environment, and an operator to manually match the 3D CAS model to the real-world object.

[0006] While it is known from the prior art to use labeled datasets to train neural networks to detect the viewpoint of objects, it is still unclear how to use unlabeled data. However, since unlabeled data is cheap and more readily available, it is desirable to use unlabeled data.

[0007] It has been proposed (e.g., see the paper "Multi-view object class detection with a 3d geometric model", Liebelt, J., Schmid, C., 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1688-1695. IEEE (2010)) to train neural networks by rendering 3D CAD images of objects at different viewpoints (so the ground truth is known for the renderings). Although it is possible to generate a large amount of labeled synthetic data using rendering and simulator tools and use them to learn viewpoint estimators, the difference between synthetic images and real-world images makes their transfer challenging. Therefore, in the paper “Renderfor cnn: Viewpoint estimation in images using cnns trained with rendered 3dmodel view” (Su, H., Qi, CR, Li, Y., Guibas, LJ, Proceedings of the IEEE International Conference on Computer Vision, pp. 2686-2694 (2015)), it has been proposed to overlay rendered images from a large collection of 3D models on top of real images, and when these images are used during training, this approach produces realistic training images and improves the detection of viewpoints. However, this solution requires the existence of a large collection of 3D models and background scenes, which is also a difficulty.

[0008] The use of unlabeled images has also been proposed in various literatures in so-called self-supervised or unsupervised processes. In these processes, information from unlabeled images is used to improve the training of a neural network to detect viewpoint or pose.

[0009] The paper "Unsupervised geometry-aware representation for 3D human pose estimation" (Rhodin, H., Salzmann, M., Fua, P., Proceedings of the European Conference on Computer Vision (ECCV), pp. 750-767 (2018)) discloses an unsupervised approach in which an autoencoder is used to learn to transform an image from a first view to an image from another view in a multi-camera setting. This solution is unsatisfactory because it requires knowledge of the rotation between each pair of cameras.

[0010] More efficient methods are needed to obtain neural networks that can automatically determine the perspective of objects visible on an image. Summary of the Invention

[0011] The present disclosure overcomes one or more deficiencies of the prior art by proposing a method for training a neural network to derive (generate, deliver) a perspective of a given object visible in an image when the image is input to the neural network, the method comprising:

[0012] providing an encoder neural network configured to receive an image as input and produce an encoded image,

[0013] providing a decoder neural network configured to receive an encoded image having the same dimensions as the encoded image derived by the encoder neural network and configured to output a decoded image (i.e., an image),

[0014] providing a first set of training images with, for each image, the viewpoint of an object belonging to a given class visible in the image (i.e., the ground truth),

[0015] A second set of training image pairs is provided, wherein each pair of training images in the second set of training image pairs comprises:

[0016] a first image on which an object belonging to said given class (an image of a class, for example a car, a pedestrian, etc.) is visible;

[0017] - a second image, said object of said first image being visible in said second image at a different viewing angle than in said first image,

[0018] and wherein training the neural network comprises adapting parameters of the neural network, parameters of the encoder neural network, and parameters of the decoder neural network by minimizing the following gaps:

[0019] - for each training image of the first set of training images, the difference between the output of the neural network when the training image is input to the neural network and the perspective of the training image,

[0020] for each pair of training images in the second set of training image pairs, the difference between the second image in each pair of training images in the second set of training image pairs and the output of the decoder neural network when:

[0021] o a first image of the pair of training images is input to the encoder neural network to obtain an encoded image,

[0022] o the second image of the pair of training images is input to the neural network to obtain the perspective,

[0023] o the coded image is rotated with a rotation (degree of rotation, angle of rotation) corresponding to the viewing angle to obtain a rotated coded image,

[0024] o The rotated encoded image is input into the decoder neural network to obtain the output of the decoder neural network.

[0025] The method can be implemented on a computing system to, for example, automatically perform training.

[0026] In this specification, the viewing angle refers to the viewing angle relative to a camera used to acquire an image on which an object is visible.

[0027] Training the neural network can be performed iteratively, for example after each gap calculation or after a given number of gap calculations, using stochastic gradient descent or any other suitable training algorithm or variant of stochastic gradient descent. Stochastic gradient descent can be used to adapt the parameters of the neural network, the encoder neural network, and the decoder neural network in a manner known per se. These parameters are, for example, the weights of these neural networks. Minimizing the gap includes calculating a loss to be minimized and, for example, backpropagating this loss through the decoder neural network, the neural network, and the encoder neural network.

[0028] It should be noted that a person skilled in the art will know how to select the structure of the neural network, the encoder neural network and the decoder neural network. For example, the neural network must be able to receive an image of a given resolution as input with a depth of 3 (if it receives an RGB image), and it must also output at least three numbers for representing the viewing angle (azimuth, elevation, in-plane rotation), and this corresponds to a given number of neurons for the first and last layers of the neural network.

[0029] For another example, the encoder neural network must be able to receive an image of a given resolution as input with a depth of 3 (if it receives an RGB image). It must also be able to output a rotatable encoded image, and this corresponds to a given number of neurons for the first layer of the encoder neural network and the last layer of the encoder neural network. The dimensions of the encoded image can be found in the calibration step. Therefore, the first layer of the decoder neural network has the same number of neurons as the last layer of the encoder neural network because it is able to receive the encoded image, and the last layer of the decoder neural network has the same number of neurons as the first layer of the encoder neural network because it is able to output the image.

[0030] It should be noted that the encoder neural network and the decoder neural network use expressions well known to those skilled in the art to form an autoencoder.

[0031] Those skilled in the art will also know how to determine the difference between two images, such as the difference between the output of the decoder neural network and the second image, or the difference between the output of the neural network and the corresponding view in the first set.

[0032] In the above-described method, the rotation between the two images in a pair does not need to be known; rather, the rotation to be applied is simply derived from the viewpoint of the second image. In fact, the inventors have observed that the encoder neural network can produce an encoded image associated with a common / canonical viewpoint, and applying the rotation derived from the second image is sufficient to obtain a rotated encoded image that will result in a decoded image that is also close to the second image in terms of viewpoint. This behavior is a result of training.

[0033] Furthermore, the above method is able to automatically determine the viewing angle of an object from a given category or of objects from multiple categories.Preferably, the images used in the above method show only one object from the multiple categories per image.

[0034] According to a particular embodiment, the viewing angle of an object visible on an image comprises three values ​​defining a (3D) vector expressed in a reference frame centered relative to the object and oriented towards the image acquisition device used to acquire the image.

[0035] The reference frame can be arranged according to the category of the object. For example, for a car, the reference frame can be centered around the center of the car, with a first axis extending from the front to the back of the car, a second axis extending from side to side, and a third axis being vertical and perpendicular to the other two axes. Different categories can have different reference frames.

[0036] Additionally, the vector corresponds to the three elements defining the viewing angle (ie, azimuth, elevation, and in-plane rotation).

[0037] From these three values, a rotation matrix can be derived in a simple manner, which can then be applied to the coded image.

[0038] Alternative representations of viewing angles, such as quaternions, can also be used.

[0039] According to a particular embodiment, the coded image is a vector having a resolution lower than the resolution of the image.

[0040] Obtaining the vector as the output of the encoder neural network can be accomplished by having a fully connected layer as the last layer of the encoder neural network.

[0041] For example, the resolution of a vector is its depth, and the resolution of an image is its width times its height times 3 (RGB images). It has been observed that lower resolution of vectors provides better encoding of global information from the image.

[0042] According to a particular embodiment, the resolution of the encoded image is a multiple of three.

[0043] For example, the depth of a vector can be expressed as 3 times k, where k is an integer.

[0044] This particular embodiment facilitates multiplying the encoded image by a rotation matrix obtained from the perspective output by the neural network.

[0045] According to a specific embodiment, the training of the neural network is performed using the following loss function:

[0046]

[0047] in:

[0048] L is the loss,

[0049] T is the first set of training images,

[0050] U is the second set of training image pairs,

[0051] I is the first image in the paired training images in the second set of training images or an image in the first training set,

[0052] I′ is the second image in the paired training images,

[0053] f v ,f e and f d are respectively the neural network, the encoder neural network and the decoder neural network,

[0054] θ v ,θ e and θ d They are f v ,f e and f d Parameters,

[0055] v is the viewing angle of image I,

[0056] R(x) is a function that determines the rotation associated with the view angle x, and

[0057] λ is a training hyperparameter.

[0058] It should be noted that if T comprises pairs of images of the same object at different viewpoints, images from T can also be used in the second sum to perform training.

[0059] Alternatively, the training can be performed by processing randomly selected batches of images from T and U. For each batch of images, two sums are calculated before performing a method such as a stochastic gradient method based on the above equation.

[0060] For example, each batch includes 64 individual images.

[0061] According to a specific embodiment, the disparity (ie, ‖x‖) is calculated using perceptual loss.

[0062] The inventors have observed that using a perceptual loss provides high-quality reconstruction (i.e., the operation of the decoder neural network). By high quality, we mean that the image obtained from the decoder neural network is not blurry, which can happen when using other gaps (such as the L1 or L2 norm).

[0063] According to a particular embodiment, the neural network and / or the encoder neural network and / or the decoder neural network is a convolutional neural network.

[0064] The present invention also provides a neural network trained by the method as defined above.

[0065] The neural network may be stored on a recording medium.

[0066] The present invention also provides a system for training a neural network to derive the perspective of a given object visible in an image when the image is input to the neural network, the system comprising:

[0067] an encoder neural network configured to receive an image as input and produce an encoded image,

[0068] a decoder neural network configured to receive an encoded image having the same dimensions as the encoded image derived by the encoder neural network and configured to output a decoded image,

[0069] a first set of training images having for each image a view of an object of a given class visible in the image,

[0070] A second set of training image pairs, wherein each pair of training images in the second set of training image pairs comprises:

[0071] a first image on which an object belonging to said given category is visible;

[0072] - a second image, said object of said first image being visible in said second image at a different viewing angle than in said first image,

[0073] and a training module configured to adapt the parameters of the neural network, the parameters of the encoder neural network, and the parameters of the decoder neural network by minimizing the following gaps:

[0074] - for each training image of the first set of training images, the difference between the output of the neural network when the training image is input to the neural network and the perspective of the training image,

[0075] - for each pair of training images in the second set of training image pairs, the difference between the second image in each pair of training images in the second set of training image pairs and the output of the decoder neural network when:

[0076] o a first image of the pair of training images is input to the encoder neural network to obtain an encoded image,

[0077] o the second image of the pair of training images is input to the neural network to obtain the perspective,

[0078] o the coded image is rotated with a rotation corresponding to the viewing angle to obtain a rotated coded image,

[0079] o The rotated encoded image is input into the decoder neural network to obtain the output of the decoder neural network.

[0080] The system may be configured to perform any embodiment of the method defined above.

[0081] The present invention also provides a system comprising the neural network.

[0082] The invention also provides a vehicle comprising a system as defined above.

[0083] In a specific embodiment, the steps of the method are determined by computer program instructions.

[0084] Therefore, the present invention also relates to a computer program for executing the steps of the method as described above when the program is executed by a computer.

[0085] The program may use any programming language and take the form of source code, object code, or a code intermediate between source code and object code, such as partially compiled form, or any other desired form.

[0086] The invention also relates to a computer-readable information medium comprising instructions of a computer program as described above.

[0087] The information medium may be any entity or device capable of storing a program. For example, the medium may include a storage device such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or a magnetic storage device such as a magnetic disk (floppy disk) or a hard disk.

[0088] Alternatively, the information medium may be an integrated circuit in which the program is embodied, the circuit being adapted to execute the method in question or to be used in its execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] How the present disclosure may be implemented will now be described by way of example with reference to the accompanying drawings, in which:

[0090] - Figure 1 is a diagram of the perspective of an object as viewed by a camera,

[0091] - Figure 2 is a diagram of the structure of the neural network used during training,

[0092] - Figure 3 is a schematic diagram of a system according to an example, and

[0093] - Figure 4 is a vehicle according to an example. DETAILED DESCRIPTION

[0094] An exemplary method for training a neural network to derive the perspective of a given object visible in an image will now be described.

[0095] The viewing angle of an object is defined as the combination of the azimuth angle of the object relative to the camera, the elevation angle of the object, and the in-plane rotation of the object.

[0096] exist Figure 1 In the figure, the object OBJ (here a car) has been shown in the scene observed by the camera CAM (i.e. the object will be visible in the image acquired by the camera CAM). The perspective of the object seen by the camera can be expressed in different ways, for example using an axis-angle representation, a unit quaternion or a rotation matrix. In this specification, the perspective (azimuth, elevation and in-plane rotation) is represented by a vector v of three values, which are the coordinates of this vector starting from the origin of a reference system placed relative to the object OBJ and directed towards the camera CAM. In the figure, this reference system is located at the center OC of the object, and the three coordinates are a 1 、a 2 and a 3 .

[0097] Preferably, the vector v has norm 1 (the three coordinates define a point on a sphere of radius 1), as this facilitates the representation of rotations, as will be described below.

[0098] Furthermore, for all objects of the same class (eg, car), a reference frame is associated with a given orientation of the object.

[0099] The method of the present invention involves training a neural network so that it can output the three values ​​a 1 、a 2 and a 3 .

[0100] As one skilled in the art will appreciate, this training may involve classes of objects. For example, a neural network may be trained to detect the perspective of a car when the car is visible in an image. However, the present invention is not limited to detecting the perspective of cars, but may also involve detecting other objects, preferably objects that can be observed on the road.

[0101] Figure 2 is a diagram of the structure of the neural network used during training and the structure of the neural network to be trained.

[0102] In the figure, reference numeral NN denotes a neural network to be trained (for simplicity, also referred to as a "neural network" in this specification). In the example shown, the neural network NN is a convolutional neural network having multiple layers, which can use 3×3 convolutions. For example, according to standard practice in the field, batch normalization layers and activation functions can also be used. In fact, those skilled in the art will know how to design a neural network suitable for the task of deriving a perspective (a vector of three values) when an image is input to the neural network (a vector matrix with a depth equal to 3 (RGB)).

[0103] In the figure, the input image is designated as I', for reasons that will become clearer below. The viewing angle is designated as v.

[0104] For a given set of m labeled images, their ground-truth view relative to the camera is defined as Among them I i is the RGB image belonging to I, v i =(a 1 ,a 2 ,a 3 )∈V is a three-dimensional vector of the true viewpoints of the objects visible in each image. The neural network NN executes the function f v :I→V, so that f v (I; θ v )=v, where θ v is f v The neural network can be trained in a manner known in the art by minimizing the sum of:

[0105]

[0106] The training will consist of adapting θ, for example by performing stochastic gradient descent v .

[0107] It should be noted that this type of training is often referred to as supervised training.

[0108] In this method, additional images are used to train the neural network. T is a first set of training images, and a second set of training images U is also provided. The images of the second set can be unlabeled, meaning there is no prior knowledge about the viewing angles of the objects visible on the images of this set.

[0109] The second set contains pairs of training images, where each pair contains:

[0110] a first image on which objects belonging to a given category are visible; and

[0111] - a second image on which the object of the first image is visible at a different viewing angle than in the first image.

[0112] Therefore, the second set U is designated as U = {(I i ; I′ i )}, and each pair contains images of the same object (e.g. the same car or airplane) taken from different perspectives.

[0113] In order to train the neural network NN using the second set U, an encoder neural network ENN is provided. The encoder neural network is configured to receive an image (I in the figure) as input and to produce an encoded image as output (EI in the figure).

[0114] For example, the encoder neural network is a convolutional neural network consisting of five blocks, where each block includes two convolutional layers, where the second convolutional layer uses a stride to reduce the spatial dimension. The convolution is 3×3 convolution, and the channel depth starts at 32 and doubles with each block. These five blocks of the encoder neural network are further connected to a fully connected layer.

[0115] Since fully connected layers are used, the output of the encoder neural network is a vector. Preferably, the depth of this vector is lower than the resolution image I (for RGB, the image height multiplied by the image width multiplied by 3). In addition, the resolution of this vector is preferably a multiple of three to facilitate subsequent rotations.

[0116] In the figure, the encoder neural network ENN receives an image I among a pair of images and outputs an encoded image EI. The operation of the encoder neural network ENN is written as fe(I; θ e ), where θ e are the parameters of the encoder neural network ENN that will be adapted during training.

[0117] Furthermore, a decoder neural network DNN is provided, which is configured to receive as input an encoded image having the same dimensions as the encoded image output by the encoder neural network ENN, and the decoder neural network DNN is configured to output an image having the same dimensions as the image input to the encoder neural network ENN.

[0118] In the figure, the decoder neural network DNN receives a rotated encoded image REI (the rotation will be described below) and outputs an image denoted as I′.

[0119] The architecture of the decoder neural network is a mirror image of the architecture of the encoder / decoder neural network.

[0120] It looks like the encoder neural network and the decoder neural network form an autoencoder.

[0121] The operation of the decoder neural network, for example when used in an autoencoder operation, can be written as f d (f e (I; θ e );θ d ), where θ d are the parameters of the decoder neural network ENN that will be adapted during training.

[0122] Although a decoded image can be obtained from an encoded image corresponding to the original image, information about the viewing angle may not be clearly available in the encoded image.In contrast, the present method involves a conditional image generation technique.

[0123] In this method, for a given pair of images (I i ; I′ i ), the perspective of the object visible in the second image I' of the pair will be used to infer the rotation ROT to be applied to the encoded image obtained from the first image I of the pair, and the rotated image will then be input to the decoder neural network. Therefore, the image derived by the decoder neural network should correspond to the second image I', or at least, minimizing the difference between I' and the output of the decoder neural network is the training goal. Therefore, in the figure, the reference numeral I' is also used to denote the output of the decoder neural network.

[0124] If the viewpoint of the image I' is unknown (i.e., I' is an unlabeled image), a neural network NN can be used to determine it. This neural network outputs a viewpoint v, from which a rotation matrix can be derived to perform a rotation operation ROT that will rotate the encoded image EI into a rotated encoded image REI. A multiplication is performed between the rotation matrix and a vector / encoded image EI whose resolution is a multiple of three.

[0125] As an example, the derivation of this rotation matrix from the viewing angle v can be performed using the "look at" transformation known to those skilled in the art. For example, this transformation is used in the OpenGL library as of version 2.1. A description of the operation of this transformation is available at the following URL as of August 2020: https: / / www.khronos.org / registry / OpenGL-Refpages / gl2.1 / xhtml / gluLookAt.xml. In the example described at this URL, the "eye" is equivalent to the viewing angle, the "center" is set to (0,0,0) and the "up" is set to (0,0,1).

[0126] This feature solves the problem of lack of ground truth for I′ and extends the learning of the encoder / decoder neural network to unlabeled images by allowing gradients originating from the decoder to be back-propagated to the neural network NN (in the figure, I′ is therefore used to represent both the input to the neural network NN and the output of the decoder neural network DNN).

[0127] The above-described use of a neural network NN results in a type of training that may be referred to as unsupervised training.

[0128] As you can imagine, using a neural network NN to acquire perspective is only relevant if the neural network NN is well-trained and accurate. In order to synergistically use labeled and unlabeled images during training to better train the neural network NN, it is proposed to combine the loss associated with the unlabeled images U and the loss associated with the labeled images T into a single loss function. Thus, this approach combines supervised and unsupervised training.

[0129] In the present method, training the neural network NN comprises adapting the parameters of the neural network, the parameters of the encoder neural network and the parameters of the decoder neural network (θ respectively) by minimizing the difference between v ,θ e ,θ d ):

[0130] - for each training image in the first set T of training images, the output of the neural network when the training image is input to the neural network is related to the viewing angle of the training image,

[0131] For each pair of training images in the second set U of training image pairs, the second image in each pair of training images in the second set of training image pairs is compared to the output of the decoder neural network when:

[0132] o The first image I of the pair of training images is input to the encoder neural network ENN to obtain an encoded image EI,

[0133] o The second image I′ of the pair of training images is input to the neural network NN to obtain the view angle v,

[0134] o the encoded image EI is rotated by a rotation ROT corresponding to the viewing angle to obtain a rotated encoded image REI,

[0135] o The rotated encoded image REI is input to the decoder neural network to obtain the output of the decoder neural network.

[0136] In other words, the following loss function L is used:

[0137]

[0138] In the above formula, λ is a hyperparameter with a value to be set during the calibration step. This hyperparameter indicates the trade-off between unsupervised training and supervised training.

[0139] Although the above equations involve using the entire set of T and U, training can also be performed iteratively, where each iteration consists of selecting a given number of individual images (e.g., 64) from T and U to use in the two sums mentioned above for computing the loss to be used in backpropagation (e.g., using stochastic gradient descent or other methods).

[0140] Therefore, batch training is performed.

[0141] Figure 3 is configured to execute Figure 2 Schematic diagram of system 100 for describing the method.

[0142] The system includes a processor 101 and a non-volatile memory 102. Therefore, the system 100 has a computer system structure.

[0143] The neural network NN, the encoder neural network ENN, and the decoder neural network DNN are stored in the non-volatile memory 102 .

[0144] Furthermore, the first set T and the second set U are stored in the non-volatile memory 102 .

[0145] The training module TR is also stored in the non-volatile memory 102 and may consist of computer program instructions that, when executed by the processor 101, will perform the training and adapt the weights θ v ,θ e and θ d .

[0146] Figure 4is a schematic diagram of a vehicle 200 , here a car, equipped with a system 201 for determining the perspective of an object visible in an image acquired by a camera 202 of the vehicle.

[0147] System 201 includes a processor 203 and a non-volatile memory 204. Figure 2 After the described training has been performed the neural network NN is stored in the non-volatile memory 204 .

[0148] The above training allows obtaining a neural network that has been observed to perform better in detecting viewpoints than a neural network that was simply trained using a set of labeled training images (supervised training). Notably, it has been observed that using a portion of the labeled dataset for training can yield various increases in accuracy.

Claims

1. A method for training a neural network to derive the perspective of a given object visible in an image when the image is input to the neural network, the method comprising: providing an encoder neural network configured to receive an image as input and produce an encoded image, providing a decoder neural network configured to receive an encoded image having the same dimensions as the encoded image derived by the encoder neural network and configured to output a decoded image, providing a first set of training images having for each image a view of an object belonging to a given class visible in the image, A second set of training image pairs is provided, wherein each pair of training images in the second set of training image pairs comprises: a first image on which an object belonging to said given category is visible; - a second image, said object of said first image being visible in said second image at a different viewing angle than in said first image, and wherein training the neural network comprises adapting parameters of the neural network, parameters of the encoder neural network, and parameters of the decoder neural network by minimizing the difference between: - for each training image in the first set of training images, the output of the neural network when the training image is input to the neural network and the viewing angle of the training image, - for each pair of training images in the second set of training image pairs, the second image in each pair of training images in the second set of training image pairs is equal to the output of the decoder neural network when: o The first image of the pair of training images is input to the encoder neural network to obtain an encoded image, o The second image in the pair of training images is input to the neural network to obtain the perspective, o the coded image is rotated with a rotation corresponding to the viewing angle to obtain a rotated coded image, ○ The rotated encoded image is input into the decoder neural network to obtain the output of the decoder neural network.

2. The method according to claim 1, characterized in that The viewing angle of an object visible on an image comprises three values ​​defining a vector expressed in a reference frame centered with respect to said object and directed towards the image acquisition means used to acquire said image.

3. The method according to claim 1 or 2, characterized in that The coded image is a vector having a resolution lower than the resolution of the image.

4. The method according to claim 1 or 2, characterized in that The dimension of the encoded image is a multiple of three.

5. The method according to claim 1 or 2, characterized in that The training of the neural network is performed using the following loss function: in: L is the loss, T is the first set of training images, U is the second set of training image pairs, I is the first image in the paired training images in the second set of training images or an image in the first set, I′ is the second image in the paired training images, f v ,f e and f d are respectively the neural network, the encoder neural network and the decoder neural network, θ v ,θ e and θ d They are f v ,f e and f d Parameters, v is the viewing angle of image I, R(x) is a function that determines the rotation associated with the view angle x, and λ is a training hyperparameter.

6. The method according to claim 5, characterized in that The gap is calculated using perceptual loss.

7. The method according to claim 1 or 2, characterized in that The neural network and / or the encoder neural network and / or the decoder neural network is a convolutional neural network.

8. A neural network trained by the method according to any one of claims 1 to 7.

9. A system for training a neural network to derive the perspective of a given object visible in an image when the image is input to the neural network, the system comprising: an encoder neural network configured to receive an image as input and produce an encoded image, a decoder neural network configured to receive an encoded image having the same dimensions as the encoded image derived by the encoder neural network and configured to output a decoded image, a first set of training images having for each image a view of an object of a given class visible in the image, A second set of training image pairs, wherein each pair of training images in the second set of training image pairs comprises: a first image on which an object belonging to said given category is visible; - a second image, said object of said first image being visible in said second image at a different viewing angle than in said first image, and a training module configured to adapt parameters of the neural network, parameters of the encoder neural network, and parameters of the decoder neural network by minimizing the difference between: - for each training image in the first set of training images, the output of the neural network when the training image is input to the neural network and the viewing angle of the training image, - for each pair of training images in the second set of training image pairs, the second image in each pair of training images in the second set of training image pairs is equal to the output of the decoder neural network when: o The first image of the pair of training images is input to the encoder neural network to obtain an encoded image, o The second image in the pair of training images is input to the neural network to obtain the perspective, o the coded image is rotated with a rotation corresponding to the viewing angle to obtain a rotated coded image, ○ The rotated encoded image is input into the decoder neural network to obtain the output of the decoder neural network.

10. A system for determining the viewing angle of a given object visible on an image, characterized in that comprising a neural network according to claim 8.

11. A vehicle, characterized in that Comprising a system according to claim 10.

12. A computer-readable recording medium having a computer program recorded thereon, characterized in that: The computer program comprises instructions which, when executed by a processor, perform the steps of the method according to any one of claims 1 to 7.

Images