Receiver for data decompression with auto-encoder enhancement

Abstract

The invention relates to a receiver (120) comprising: - a memory (502) storing a first auto-encoder convolutional neural network designed to obtain enhanced data from compressed data; - a first interface (503) designed to receive compressed data over a communication channel (130); - a processor (501) configured to decompress the data into decompressed data, and to apply the first auto-encoder convolutional neural network to the compressed data to obtain enhanced data; and - a second interface (504) designed to transmit the enhanced data.

Description

Receiver for data decompression with autoencoder enhancement

[0001] The present invention relates to the field of receiving and processing data, in particular image data. The invention applies in particular, but not exclusively, to data exchanged in or by a motor vehicle.

[0002] It is known to compress data, especially image data, in order to limit the amount of data transmitted over a communication channel.

[0003] Indeed, some communication channels are limited in bandwidth. This is particularly the case with the CAN bus, although this type of communication channel is widely used because it is secure and inexpensive. This communication channel is used in particular in motor vehicles, for example to transmit photometric images between a vehicle's central control module and a vehicle's lighting system.

[0004] Image data transmitted, including in motor vehicles, now has high resolutions. At the same level of constraint in terms of throughput on the communication channel, this results in high compression rates, which can induce defects, such as noise, artifacts or low PSNR in the decompressed images on the receiver side. PSNR refers to the peak signal-to-noise ratio, and stands for "Peak Signal to Noise Ratio" in English.

[0005] This is particularly the case with lossy compression algorithms, such as linearization, gradient, JPG, PCA or other algorithms. When these images are photometrics used to control a lighting device for a motor vehicle, defects or artifacts are reflected in the projected beam, especially since the resolution of pixelated beam lighting modules also becomes high. Such defects can lead to safety problems and / or make the light beam non-compliant with regulations.

[0006] There is thus a need to receive and process high resolution image data, via a data rate constrained communication channel, without inducing large defects or artifacts in the images finally processed on the receiver side.

[0007] To this end, a first aspect of the invention relates to a data processing method comprising the following operations: during a preliminary phase, supervised learning of a first auto-encoder convolutional neural network on the basis of a first training data set, the first training data set comprising pairs of images comprising an image of given quality and a compressed image obtained by compression of the image of given quality, in which the supervised learning is capable of minimizing a difference between an enhanced image obtained by processing by the first auto-encoder convolutional neural network of a compressed image, and the image of given quality associated in pairs with the compressed image; storing said first auto-encoder convolutional neural network in a receiver.The method further comprises, during a current phase implemented by the receiver: receiving compressed data from a communication channel; decompressing said compressed data into decompressed data; applying the first auto-encoder convolutional neural network to the compressed data to obtain enhanced data; transmitting said enhanced data.

[0008] Such AI enhancement can remove at least some of the defects in the decompressed data. This makes it possible to provide compression with a high compression ratio, and thus to transmit high-resolution data over data-rate-constrained communication channels.

[0009] According to embodiments, the decompression of said compressed data may be based on a linearization, gradient, JPG or PCA decompression algorithm.

[0010] Thus, it is possible to enhance images compressed by compression algorithms. For this purpose, the first training dataset can advantageously comprise pairs of images, with images compressed according to different compression algorithms. In practice, high levels of enhancement of images compressed by such algorithms can be achieved.

[0011] According to embodiments, the decompression of the compressed data can be implemented by processing by neural layers comprising an output layer and at least one hidden convolutional layer of a second autoencoder type convolutional neural network, said output layer comprising a first number of dimensions and said at least one hidden convolutional layer comprising a second number of dimensions, the second number of dimensions being less than the first number of dimensions.

[0012] Thus, the compression itself can be derived from an autoencoder, called a second autoencoder. This makes it possible to control the compression level by fixing the number of dimensions of the latent vector. Even for high compression levels, enhancement makes it possible to obtain enhanced data of good quality, with few or no defects.

[0013] Additionally, the method may further comprise, during the preliminary phase, unsupervised learning of the second auto-encoder convolutional neural network on a second training data set.

[0014] Thus, the learning step is simplified for the second autoencoder. Unsupervised learning can, for example, consist of optimizing the second autoencoder so as to minimize a gap between input data and output data of the autoencoder, obtained after compression and decompression of the input data.

[0015] According to embodiments, the compressed data and the enhanced data may be representative of automotive vehicle lighting device photometry.

[0016] It is important for regulatory and safety reasons to minimize defects in such photometric data. The use of the enhancement according to the invention is therefore particularly advantageous.

[0017] Additionally, the enhanced data can be transmitted to a control module of a motor vehicle lighting device.

[0018] Thus, the receiver can advantageously be implemented in a lighting device for controlling at least one of the lighting modules of the lighting device.

[0019] Additionally or alternatively, the compressed data may be received from a centralized motor vehicle control module and the communication channel may be a CAN bus.

[0020] Such a communication channel has the advantage of being secure and inexpensive. However, it is constrained in terms of throughput and may require high compression rates, which makes the use of the enhancement according to the invention particularly advantageous.

[0021] A second aspect of the invention relates to a receiver comprising:a memory storing a first auto-encoder convolutional neural network capable of obtaining enhanced data from compressed data;a first interface capable of receiving compressed data over a communication channel;- at least one processor configured to decompress the data into decompressed data, and to apply the first auto-encoder convolutional neural network to the compressed data to obtain enhanced data;a second interface capable of transmitting the enhanced data.

[0022] A third aspect of the invention relates to a system comprising a receiver according to the second aspect of the invention, an encoder capable of receiving input data, compressing the input data into compressed data and transmitting the compressed data to the receiver via a communication channel.

[0023] According to embodiments, the encoder may be integrated into a central control module of a motor vehicle, the receiver may be integrated into a lighting device of the motor vehicle, the communication channel may be a CAN bus and the input data may be representative of lighting photometry.

[0024] Other characteristics and advantages of the invention will appear on examining the detailed description below, and the appended drawings in which:

[0025] illustrates a data transmission system according to embodiments of the invention;

[0026] illustrates the structure of a first auto-encoder convolutional neural network according to embodiments of the invention;

[0027] is a diagram illustrating the steps of a data processing method according to embodiments of the invention;

[0028] illustrates a data compression system by a second auto-encoder convolutional neural network according to an embodiment of the invention;

[0029] illustrates a structure of a receiver according to embodiments of the invention.

[0030] The description focuses on the features that distinguish the methods, system, encoder and decoder from those known in the state of the art.

[0031] Illustrates a system 100 for transmitting data, in particular image data.

[0032] The system 100 comprises an encoder 110 and a receiver 120, connected by a communication channel 130.

[0033] The encoder 110 may be integrated into equipment of a motor vehicle, such as a control module responsible for the lighting of the motor vehicle. Such a control module may be of the PCM type, for “Powertrain Control Module” in English, or ECU type, for “Electronic Control Unit” in English, for example.

[0034] The receiver or decoder 120 can be integrated into equipment of a motor vehicle, such as a lighting device comprising lighting modules capable of performing lighting functions according to data communicated by the control module comprising the encoder 110. Preferably, at least one lighting module of the lighting device is a pixelated module, for example with a matrix of electroluminescent elements such as LEDs, with a matrix of micro-mirrors, of the DMD type for "Digital Micromirror Devices" in English, a monolithic source of electroluminescent elements on the same substrate, or any other technology allowing the production of a pixelated lighting beam. A monolithic source involves a plurality of electroluminescent semiconductor elements with submillimeter dimensions, epitaxially grown directly on a common substrate, the substrate generally being formed of silicon.Unlike conventional LED arrays, in which each elementary light source is an individually produced electronic component mounted on a substrate such as a printed circuit board (PCB), a monolithic source is to be considered as a single electronic component, during the production of which several areas of light-emitting semiconductor junctions are generated on a common substrate, in the form of an array.

[0035] The communication channel 130 can thus be a wired connection such as a CAN bus or an Ethernet connection. In the following, the example of a CAN bus is considered for illustrative purposes. It has the advantage of being a secure and inexpensive connection. However, a CAN bus is constrained in terms of throughput and requires data to be transmitted in the form of integers coded on 8 bits.

[0036] Alternatively, the encoder 110 is integrated into a control module of the motor vehicle and the receiver 120 is integrated into a remote server of the motor vehicle. In this case, the communication channel 130 comprises a wireless communication channel allowing the encoder to access an IP network in which the remote server comprising the decoder 120 is located. Such a wireless communication channel may be a cellular link of the 3G, 4G, 5G type or any subsequent generation.

[0037] No restrictions are attached to the communication channel 130, which can thus be a wired or wireless link. As will be better understood from the following, most communication channels are constrained in the data rate they transmit.

[0038] The encoder 110 comprises a data compression module 111 capable of receiving input data, such as data encoding an image, for example a photometric image, and of obtaining compressed data from the input data, and from a data compression algorithm, such as a linearization, gradient, JPG, PCA or other algorithm. Alternatively, the compressed data corresponds to a latent vector of an auto-encoder convolutional neural network, as will be detailed later with reference to the.

[0039] The decoder 120, or receiver 120, comprises a decompression module 121 corresponding to the decompression module 111, and capable of decompressing the compressed data received via the communication channel 130, in order to obtain decompressed data.

[0040] As indicated previously, when the compression rate is high, for example greater than 80%, defects may appear in the images corresponding to the decompressed data. The invention then provides for adding a module 122, software or physical, for enhancing the compressed data in order to obtain enhanced data, presenting fewer defects than the decompressed data at the output of the decompression module 121.

[0041] For this purpose, the enhancement module 122 may comprise a first auto-encoder convolutional neural network 200 described with reference to the.

[0042] An autoencoder convolutional neural network, also called autoencoder in the following, comprises several layers of neurons, including an input layer 200.1, an output layer 201.1, at least one hidden convolutional layer of convolution, on the input layer 200.1 side, and at least one hidden convolutional layer of deconvolution, on the output layer 201.1 side.

[0043] In the example of the, given for illustrative purposes, the first autoencoder 200 comprises a first hidden convolution layer 200.2 and a second hidden convolution layer 200.3. The first autoencoder 200 comprises, symmetrically, a first hidden deconvolution layer 201.2 and a second hidden deconvolution layer 201.3.

[0044] The first hidden convolution layer 200.2 comprises the same number of dimensions as the first hidden deconvolution layer 201.2. Similarly, the second hidden convolution layer 200.3 comprises the same number of dimensions as the second hidden deconvolution layer 201.3.

[0045] An autoencoder thus comprises a symmetrical set of layers of neurons.

[0046] The hidden layers having the smallest number of dimensions, or central layers, in this case layers 200.3 and 201.3, are able to exchange data called “code” or “latent vector” 210. The latent vector is a compressed version of the data received by the input layer 200.1.

[0047] The auto-encoder 200 according to the invention is capable of enhancing compressed data received as input, into enhanced data having fewer defects and closer to the input images received and compressed by the encoder 110.

[0048] For this purpose, the first auto-encoder 200 is derived from supervised learning based on a first training data set. The first training data set comprises pairs of images, each pair of images comprising: an image of given quality, in particular of optimal quality, i.e. uncompressed. No restriction is attached to the resolution of such an image. The image of given or optimal quality does not have any defect; a compressed image obtained by compressing the image of given quality, by a given compression algorithm, for example among the aforementioned compression algorithms. Such a compressed image may have defects as presented above.

[0049] Preferably, the training dataset comprises pairs of images that vary: by the compression algorithm applied to obtain the compressed image; by the compression rate applied to obtain the compressed image; and / or by the type of defect that the compressed image comprises, among artifacts, a PSNR lower than a given threshold, for example lower than 25, a lack of a part of the image, or by other compression quality indicators, such as a maximum error or a mean square error, or MSE for "Mean Square Error" for example.

[0050] The first training data set comprises more than a hundred pairs of images, preferably several thousand or tens of thousands of pairs of images. Supervised learning then consists of submitting, for each pair of images, the compressed image as input to the first autoencoder 200. The image obtained at the output of the output layer 201.1 is compared to the optimal quality image associated with the compressed image, in order to estimate the difference, for example a quadratic error between the image at the output of the first autoencoder and the optimal quality image. The first autoencoder 200 is then modified according to the determined difference, for example by changing the characteristic values ​​of one or more layers of neurons, in order to reduce the calculated difference.

[0051] In the example considered here, in which the encoder is integrated into a PCM or ECU of a motor vehicle, and the decoder is integrated into a lighting device, the data of the first training data set are pairs of images representative of the light beam to be produced by motor vehicle lighting devices. Such images are also called photometries.

[0052] However, there is no restriction on the data of the first training dataset being any type of image. For example, in the example where the decoder 120 is implemented in a remote server, the data of the training dataset may be images acquired by automotive vehicle cameras.

[0053] After learning on the entire first training data set, the squared error is minimized and the first autoencoder 200 is able to reconstruct images of optimal quality from images comprising one or more defects, following their compression. The autoencoder 200 can thus be implemented in the receiver 120 described previously.

[0054] Advantageously, the first autoencoder 200 may provide for reuse, by deconvolution neuron layers, of characteristics of the convolution neuron layers. Such reuse may be enabled by at least one skip connection, or "skip connection" in English, connecting two non-consecutive convolutional neuron layers of the first autoencoder 200.

[0055] For example, the first autoencoder 200 comprises at least one skip-layer connection 220.1 between a pair of layers having the same number of dimensions, or features, each pair comprising a convolution layer and a deconvolution layer. In the example of the, the skip-layer connections 220.1 may thus comprise a connection between the input layer 200.1 and the output layer 201.1, a connection between the first hidden convolution layer 200.2 and the first hidden deconvolution layer 201.2, and a connection between the second hidden convolution layer 200.3 and the second hidden deconvolution layer 201.3.

[0056] Additionally or alternatively, the first autoencoder 200 comprises at least one skip-layer connection 220.2 between a pair of layers having different numbers of dimensions, or features, each pair comprising a convolution layer and a deconvolution layer. In the example of 1a, the skip-layer connections 220.2 may thus comprise a connection between the first hidden convolution layer 200.2 and the second hidden deconvolution layer 201.3 and a connection between the second hidden convolution layer 200.3 and the first hidden deconvolution layer 201.2.

[0057] Such skip-layer connections advantageously allow complex data processing with deep neural networks.

[0058] The present invention presents a method for processing data according to embodiments of the invention.

[0059] The method comprises a preliminary phase 300, comprising a step 301 of obtaining the first training data set, comprising pairs of images as described previously. No restriction is attached to the manner in which the training data set is obtained. The images of optimal quality can come from real situations or from simulations for example.

[0060] At a step 302 of the preliminary phase 300, the first auto-encoder convolutional neural network 200 is trained by supervised learning on the basis of the image pairs of the first training data set obtained at the previous step 301. The first auto-encoder 200 thus obtained is capable of enhancing the quality of compressed data.

[0061] At a step 303 of the preliminary phase 300, the first auto-encoder convolutional neural network 200 is stored in a receiver, such as the receiver 120 described previously. As previously explained, the receiver 120 can be integrated into a lighting device for a motor vehicle or into a remote server of a motor vehicle.

[0062] The processing method further comprises a current phase 310 comprising a step 311 of reception by the receiver 120 of compressed data, via the communication channel 130 described previously. The received data has in particular been compressed beforehand by the encoder 110 described previously, no restriction being attached to the data compression technique.

[0063] In a step 312, the decompression module 121 decompresses the compressed data received during the previous step 311, as described previously.

[0064] In a step 313, the decompressed data is processed by the first auto-encoder 200 in order to be enhanced. Enhanced data is thus obtained at the end of step 313. In view of the machine learning from which the first auto-encoder 120 is derived, the enhanced data makes it possible to obtain an image of optimal quality, close to the image initially compressed by the encoder 110.

[0065] The enhanced data may be transmitted by the receiver 120 at a step 314. For example, the receiver 120 may transmit the enhanced data to a memory for storage. Advantageously, in the embodiment in which the encoder 110 is integrated into a PCM and the receiver 120 is integrated into a signaling device, the enhanced data may be transmitted to a light source control module for performing photometry corresponding to the output data.

[0066] The invention presents a data compression module 111 and a data decompression module 121 according to an embodiment of the invention.

[0067] As previously mentioned, the compression module 111 and the decompression module 121 may be capable of implementing a compression / decompression algorithm such as linearization, gradient, JPG, PCA or other algorithms.

[0068] According to a variant illustrated in the, the compression / decompression is implemented by means of a second auto-encoder convolutional neural network, or second auto-encoder hereinafter, the compression and decompression modules 111 and 121 each comprising a part of the second auto-encoder. The compression module 111 thus comprises a first part 400 of the second auto-encoder while the decompression module 121 comprises a second part 410 of the second auto-encoder.

[0069] The second autoencoder is capable of compressing input data, including high-quality images, such as photometry images for lighting devices. The second autoencoder may be constructed by unsupervised learning on a second training data set, different from the first training data set.

[0070] The second autoencoder comprises an input layer 401 implemented in the encoder 110 and an output layer 411 implemented in the receiver 120, the input layer and the output layer having the same number of nodes or neurons, therefore the same number of dimensions.

[0071] The autoencoder system further comprises one or more hidden convolutional layers, each hidden convolutional layer having a number of dimensions less than the number of dimensions of the input layer 401 and the output layer 411.

[0072] The hidden convolutional layers with the smallest number of dimensions, or central layers, are able to exchange a "code" or "latent vector" and this latent vector is thus a compressed version of the input data.

[0073] The central layers can thus be shared between the encoder 112 and the decoder 123 so as to exchange compressed data, thus making it possible to reduce the bit rate requirements and the quantity of data exchanged between the encoder and the decoder, while minimizing losses. The first part 400 thus comprises a central encoding layer 402 and the second part 410 comprises a central decoding layer 412. The central encoding layer 402 and the central decoding layer 412 are capable of exchanging a latent code or vector comprising a number of dimensions lower than the input data.

[0074] The second autoencoder is trained by unsupervised learning so as to minimize the squared error between the input data and the output data from the output layer, given a number of dimensions of the central layers, therefore at a given compression level.

[0075] For this purpose, the second training data set may be submitted to the second autoencoder. The training data set may comprise a set of images, such as vehicle lighting photometries in the example considered here. For each image of the second training data set, the autoencoder evaluates the quadratic error between the image submitted to the input layer 401 and the image provided by the output layer 411, and varies the characteristics of its neurons as well as the number of neurons, or even the number of hidden layers, as a function of this quadratic error, while maintaining a constraint to obtain a latent vector having a given number of dimensions. The aim is thus to minimize the quadratic error by learning.

[0076] The latent vector is a compressed version of the input data, with a compression ratio CR according to the following formula: CR = (Nbits * Im_Size – NbitsLV * LVdim) / Nbits * Im_Size ; in which Nbits is the number of bits on which each pixel of the input image is encoded, Im_Size is the size of the image in number of pixels, NbitsLV is the number of bits encoding each dimension of the latent vector, which is fixed and generally equal to 32 bits, and LVdim is the number of dimensions of the latent vector.

[0077] More generally, Nbits*Im_Size represents the size in number of bits of the input data.

[0078] For a given image size, the compression rate CR can thus vary by varying the number of dimensions LVdim of the latent vector.

[0079] The following compression rates can be obtained: CR = 92% for LVdim = 516; CR = 84% for LVdim = 1024; CR = 52% for LVdim = 3072.

[0080] Thus, the lower the number of dimensions of the latent vector, the higher the compression ratio CR. The selection of a compression ratio may depend on quality indicators comparing the output data with the input data. Such indicators may include a peak signal-to-noise ratio, or PSNR, or a mean square error, or MSE, for example.

[0081] For example, a number of dimensions of the latent vector ensuring a PSNR greater than a given threshold, such as 30 for example, could be fixed.

[0082] The first part 400 and the second part 410 of the auto-encoder convolutional neural network are thus obtained, and can be implemented respectively in the encoder 110 and the decoder 120, during the preliminary phase 300 described previously.

[0083] However, when the communication channel 130 is limited in throughput, as is notably the case with a CAN bus, generally used between a PCM and a lighting device, high compression rates, or even reformatting of the data, is necessary in order to allow the transport of the latent vector on the communication channel 130. As a result, the decompressed data from the output layer 411 may have defects, which makes the use of the enhancement module 122 described previously advantageous.

[0084] The PSNR value of the enhanced data can be higher by several points, including 5 points, compared to the decompressed data from the output layer 411 of the second autoencoder. Other indicators such as the maximum error and the root mean square error are also improved.

[0085] In the case where compression / decompression is not performed by the second autoencoder, but by one of the previously discussed compression / decompression algorithms, the PSNR gain enabled by the enhancement module 122 can even reach 10 points. Other indicators such as maximum error and root mean square error are also improved.

[0086] Illustrates the structure of a decoder or receiver 120 according to embodiments of the invention.

[0087] The decoder 120 comprises a processor 501 configured to communicate unidirectionally or bidirectionally, via one or more buses or via a wired connection, with a memory 502 such as a “Random Access Memory” type memory, RAM, or a “Read Only Memory” type memory, ROM, or any other type of memory (Flash, EEPROM, etc.). Alternatively, the memory 502 comprises several memories of the aforementioned types. Preferably, the memory 502 is a non-volatile memory.

[0088] The memory 502 stores, permanently or temporarily, all of the data generated following the implementation of steps 311 to 314 of the data processing method described above. The memory 502 further stores the first auto-encoder convolutional neural network 200 during step 303 described previously.

[0089] Further, memory 502 stores a decompression algorithm or the second part 410 of the second convolutional neural network described with reference to the.

[0090] The processor 501 is capable of executing instructions, stored in the memory 502, for implementing steps 312 and 313 of the method illustrated with reference to the. Alternatively, the processor 501 may be replaced by a microcontroller designed and configured to carry out steps 312 and 313 of the method according to the.

[0091] The decompression module 121 as well as the enhancement module 122 presented previously can thus be produced by the processor 501 or the microcontroller. As a further variant, a processor or a microcontroller is dedicated to the decompression function and another processor or microcontroller is dedicated to the enhancement function.

[0092] The receiver 120 may comprise an input interface 503 capable of receiving compressed data, during the step 311 described previously. No restriction is attached to the first input interface 503, which is functionally connected to the communication channel 130 described previously.

[0093] The decoder 120 may further comprise a second output interface 504 capable of transmitting the output data during the step 314 described previously.

[0094] The present invention is not limited to the embodiments described above as examples; it extends to other variants.

Claims

A data processing method comprising the following operations:during a prior phase (300), supervised learning (302) of a first auto-encoder convolutional neural network on the basis of a first training data set, the first training data set comprising pairs of images comprising an image of given quality and a compressed image obtained by compressing the image of given quality, wherein the supervised learning is capable of minimizing a difference between an enhanced image obtained by processing by the first auto-encoder convolutional neural network, of a compressed image, and the image of given quality associated in pairs with the compressed image;storing (303) said first auto-encoder convolutional neural network in a receiver (120);the method further comprising, during a current phase (310) implemented by said receiver:receiving (311) compressed data from a communication channel (130);decompressing (312) said compressed data into decompressed data;applying (313) the first auto-encoder convolutional neural network to the compressed data to obtain enhanced data;transmitting (314) said enhanced data.; Data processing method according to claim 1, wherein the decompression of said compressed data is based on a linearization, gradient, JPG or PCA decompression algorithm. A data processing method according to claim 1, wherein the decompression of said compressed data is implemented by processing by neural layers comprising an output layer (411) and at least one hidden convolutional layer (412) of a second autoencoder type convolutional neural network, said output layer comprising a first number of dimensions and said at least one hidden convolutional layer comprising a second number of dimensions, the second number of dimensions being less than the first number of dimensions. The method of claim 3, further comprising, during the prior phase (300), unsupervised training of the second auto-encoder convolutional neural network on a second training data set. A method according to any preceding claim, wherein the compressed data and the enhanced data are representative of lighting device photometry. Method according to claim 5, wherein the enhanced data is transmitted to a control module of a lighting device for a motor vehicle. The method of claim 5 or 6, wherein the compressed data is received from an encoder (110) of a centralized control module of a motor vehicle and wherein the communication channel (130) is a CAN bus. A receiver (120) comprising:a memory (502) storing a first auto-encoder convolutional neural network capable of obtaining enhanced data from compressed data;a first interface (503) capable of receiving compressed data over a communication channel (130);a processor (501) configured to decompress the data into decompressed data, and to apply the first auto-encoder convolutional neural network to the compressed data to obtain enhanced data;a second interface (504) capable of transmitting the enhanced data. A system comprising a receiver (120) according to claim 8, an encoder (110) adapted to receive input data, to compress the input data into compressed data and to transmit the compressed data to the receiver via a communication channel (130). System according to claim 9, wherein the encoder (110) is integrated into a central control module of a motor vehicle, wherein the receiver (120) is integrated into a lighting device of the motor vehicle, wherein the communication channel (130) is a CAN bus and wherein the input data is representative of lighting photometry.

Images