Receiver for data decompression with autoencoder enhancement.

Abstract

The present invention provides - a memory (502) storing a first autoencoder convolutional neural network capable of obtaining enhanced data from compressed data; - a first interface (503) capable of receiving compressed data via a communication channel (130); - a processor (501) configured to apply a first autoencoder convolutional neural network to the compressed data to decompress the data into decompressed data and to obtain enhanced data; - a second interface (504) capable of transmitting enhanced data; The present invention relates to a receiver (120) including:

Description

[Technical Field]

[0001] The present invention relates to the field of receiving and processing data, particularly image data, and is particularly, but not exclusively, applicable to data exchanged within or by motor vehicles. [Background technology]

[0002] Data compression, particularly image data compression, is known to limit the amount of data transmitted over a communication channel.

[0003] Some communication channels have limited bandwidth, as is the case with CAN buses in particular, but this type of communication channel is widely used because it is secure and inexpensive, and is particularly used in motor vehicles, for example, to transmit photometric images between the vehicle's central control module and the vehicle's lighting devices.

[0004] Transmitted image data, including automobiles, nowadays have high resolution. The same limitations on communication channel throughput result in high compression ratios, which can cause defects such as noise, artifacts, or low PSNR in the decompressed image at the receiver. PSNR stands for "Peak Signal to Noise Ratio."

[0005] This is especially true for lossy compression algorithms, such as linearization-based, gradient-based, JPG, PCA, or other algorithms. When these images are photometric images that enable the control of automotive vehicle lighting devices, defects or artifacts affect the projected beam, especially as pixelated beam illumination modules become more highly resolved. Such defects can cause safety issues and / or result in a light beam that does not comply with regulations.

[0006] Therefore, there is a need to receive and process high resolution image data over a communication channel having limited throughput without causing substantial defects or artifacts in the final processed image at the receiver. Summary of the Invention

[0007] To this end, a first aspect of the present invention provides a method for detecting a signal comprising the following operations: - in a preliminary stage, supervised learning of a first autoencoder convolutional neural network based on a first set of training data, the first set of training data consisting of image pairs including an image of a given quality and a compressed image obtained by compressing the image of the given quality, wherein the supervised learning can minimize a difference between an enhanced image obtained through processing of the compressed image by the first autoencoder convolutional neural network and an image of the given quality pairwise associated with the compressed image; - storing the first autoencoder convolutional neural network in a receiver. During this stage, which is carried out by the receiver, the method comprises: - receiving compressed data from a communications channel; - decompressing the compressed data into decompressed data; - applying a first auto-encoding convolutional neural network to the compressed data to obtain enhanced data; - transmitting the enhancement data; Further includes:

[0008] Such enhancement through artificial intelligence removes at least some of the imperfections in the decompressed data, thereby providing compression at a high compression ratio so that high resolution data can be transmitted over communication channels with limited throughput.

[0009] Depending on the embodiment, decompression of the compressed data may use linearization-based, gradient-based, JPG, or PCA decompression algorithms.

[0010] In this way, it is possible to enhance the images compressed by the compression algorithm. To this end, the first set of training data may advantageously comprise image pairs, the images being compressed according to different compression algorithms. In practice, a high level of enhancement of the images compressed by such algorithms may be achieved.

[0011] Depending on the embodiment, decompression of the compressed data may be performed through processing by a layer of neurons including an output layer and at least one convolutional hidden layer of a second autoencoder convolutional neural network, the output layer including a first dimensionality and the at least one convolutional hidden layer including a second dimensionality that is less than the first dimensionality.

[0012] The compression itself can result from an autoencoder, hence referred to as the second autoencoder. The compression level can therefore be controlled by setting the dimensionality of the latent vector. Even at high compression levels, the enhancement provides high-quality enhanced data with few artifacts.

[0013] Additionally, in a preliminary step, the method may further include unsupervised training of a second autoencoder convolutional neural network based on a second set of training data.

[0014] The learning step is therefore simplified for the second autoencoder. Unsupervised learning may, for example, involve optimizing the second autoencoder to minimize the difference between the input data and the output data of the autoencoder obtained after compression and decompression of the input data.

[0015] Depending on the embodiment, the compressed data and enhanced data may represent photometry of an automotive vehicle lighting device.

[0016] Minimizing imperfections in such photometric data is important for regulatory and safety reasons, which is why the use of enhancement according to the present invention is particularly advantageous.

[0017] Additionally, the enhancement data may be transmitted to a control module of the motor vehicle lighting device.

[0018] The receiver may therefore advantageously be implemented within a lighting device for controlling at least one of the lighting modules of the lighting device.

[0019] Additionally or alternatively, the compressed data may be received from a centralized control module of the motor vehicle and the communication channel may be a CAN bus.

[0020] This type of communication channel offers the advantage that it is secure and cheap, however it has limited throughput and may require high compression ratios, making the use of the enhancements according to the invention particularly advantageous.

[0021] A second aspect of the present invention is - a memory storing a first autoencoder convolutional neural network capable of obtaining enhanced data from compressed data; - a first interface capable of receiving compressed data via a communications channel; - at least one processor configured to apply a first autoencoder convolutional neural network to the compressed data to decompress the data into decompressed data and to obtain enhanced data; - A second interface through which enhanced data can be transmitted The present invention relates to a receiver including:

[0022] A third aspect of the present invention relates to a system including a receiver according to the second aspect of the present invention, an encoder capable of receiving input data, compressing the input data into compressed data, and transmitting the compressed data to the receiver over a communication channel.

[0023] Depending on the embodiment, the encoder may be integrated into a central control module of the motor vehicle, the receiver may be integrated into a lighting device of the motor vehicle, the communication channel may be a CAN bus, and the input data may represent lighting photometry.

[0024] Other features and advantages of the present invention will become apparent from consideration of the following detailed description and accompanying drawings. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 shows a data transmission system according to an embodiment of the present invention.

[0026] [Figure 2] FIG. 2 shows the structure of a first autoencoder convolutional neural network according to an embodiment of the present invention.

[0027] [Figure 3] FIG. 3 is a diagram illustrating steps of a data processing method according to an embodiment of the present invention.

[0028] [Figure 4] FIG. 4 illustrates a data compression system using a second autoencoder convolutional neural network according to one embodiment of the present invention.

[0029] [Figure 5] FIG. 5 is a diagram illustrating the structure of a receiver according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0030] The present description will focus on features that differentiate the present methods, systems, encoders, and decoders from those known in the prior art.

[0031] FIG. 1 shows a system 100 for transmitting data, particularly image data.

[0032] The system 100 includes an encoder 110 and a receiver 120 connected by a communication channel 130 .

[0033] The encoder 110 may be integrated into a vehicle device, such as a control module responsible for the vehicle's lighting. Such a control module may be, for example, a Powertrain Control Module (PCM) or an Electronic Control Unit (ECU).

[0034] The receiver or decoder 120 may be integrated into an automotive vehicle device, such as a lighting device including an illumination module capable of performing lighting functions based on data communicated by a control module including the encoder 110. At least one illumination module of the lighting device is preferably a pixelated module, which may comprise, for example, a matrix of electroluminescent elements such as LEDs with a matrix of micromirrors such as a DMD (Digital Micromirror Device), a monolithic source of electroluminescent elements on the same substrate, or any other technology that allows for the implementation of a pixelated illumination beam. A monolithic source involves multiple submillimeter-sized electroluminescent semiconductor elements epitaxially grown directly on a common substrate, the substrate typically being made of silicon. Unlike a conventional LED matrix, in which each elementary light source is an individually manufactured electronic component mounted on a substrate such as a printed circuit board (PCB), a monolithic source would be considered a single electronic component, and in its production, several areas of semiconductor electroluminescent elements are produced on a common substrate in the form of a matrix.

[0035] The communication channel 130 may therefore be a wired link such as a CAN bus or an Ethernet link. The example of a CAN bus is discussed below for illustrative purposes. It offers the advantage of being a secure and inexpensive link. However, the CAN bus has limited throughput and requires the transmission of data as 8-bit encoded integers.

[0036] Alternatively, encoder 110 is integrated within a control module of the motor vehicle, and receiver 120 is integrated within a remote server of the motor vehicle. In this case, communication channel 130 includes a wireless communication channel that allows the encoder to access an IP network on which the remote server, including decoder 120, is located. Such a wireless communication channel may be 3G, 4G, 5G, or any subsequent generation of cellular link.

[0037] No restrictions are placed on the communication channel 130, which may be a wired or wireless link. As will be better understood from reading the following description, most communication channels have a limited throughput of the data they transmit.

[0038] The encoder 110 can receive input data, such as data encoding an image, such as a photometric image, and can derive compressed data from the input data and from a data compression algorithm, such as linearization-based, gradient-based, JPG, PCA, or other algorithm. Alternatively, the compressed data corresponds to the latent vectors of an autoencoder convolutional neural network, as described in more detail below with reference to FIG. 4.

[0039] The decoder 120, or receiver 120, includes a decompression module 121 that corresponds to the decompression module 111 and that is capable of decompressing compressed data received over the communication channel 130 to obtain decompressed data.

[0040] As mentioned above, when the compression ratio is high, for example, above 80%, defects appear in the image corresponding to the decompressed data. Therefore, the present invention provides a software or hardware module 122 for enhancing the compressed data to obtain enhanced data that has fewer defects than the decompressed data at the output of the decompression module 121.

[0041] To this end, the enhancement module 122 may include a first autoencoder convolutional neural network 200 as described with reference to FIG.

[0042] The autoencoder convolutional neural network, also referred to hereinafter as an autoencoder, includes multiple layers of neurons, including an input layer 200.1, an output layer 201.1, at least one convolutional hidden layer of convolution on the input layer 200.1, and at least one convolutional hidden layer of deconvolution on the output layer 201.1.

[0043] 2, the first autoencoder 200 includes a first convolutional hidden layer 200.2 and a second convolutional hidden layer 200.3. The first autoencoder 200 also includes a first deconvolutional hidden layer 201.2 and a second deconvolutional hidden layer 201.3 in a symmetric arrangement.

[0044] The first convolutional hidden layer 200.2 has the same dimensionality as the first deconvolutional hidden layer 201.2. Similarly, the second convolutional hidden layer 200.3 has the same dimensionality as the second deconvolutional layer 201.3.

[0045] Thus, an autoencoder contains a symmetric set of layers of neurons.

[0046] The hidden, or central, layer with the least dimensionality, in this case layers 200.3 and 201.3, can exchange data called "codes" or "latent vectors" 210. Latent vectors are compressed versions of the data received by input layer 200.1.

[0047] The autoencoder 200 according to the invention is able to enhance compressed data received at the input into enhanced data that has fewer imperfections and is closer to the input image received and compressed by the encoder 110 .

[0048] To this end, the first autoencoder 200 is the result of supervised learning based on a first set of training data, which includes image pairs, each image pair consisting of: - an image of a given quality, in particular of optimal quality, i.e. uncompressed. No restrictions are imposed on the resolution of this type of image. An image of a given quality or optimal quality has no defects. a compressed image obtained by compressing an image of a given quality by a given decompression algorithm, for example one of the compression algorithms mentioned above, such a compressed image may have defects as explained above.

[0049] The training data set preferably includes image pairs, which vary with respect to: - the compression algorithm applied to obtain the compressed image, - the compression ratio applied to obtain the compressed image, and / or - The type of defects the compressed image has, including artifacts, a PSNR below a given threshold, e.g., less than 25, image defects, or other compression quality indicators such as maximum error or mean squared error (MSE).

[0050] The first set of training data includes more than 100 image pairs, preferably thousands or tens of thousands of image pairs. Supervised learning then consists in presenting a compressed image at the input of the first autoencoder 200 for each image pair. The image obtained at the output of the output layer 201.1 is compared with an optimal quality image associated with the compressed image to estimate, for example, the difference between the image at the output of the first autoencoder and the optimal image quality, e.g., the mean square error. The first autoencoder 200 is then modified according to the determined deviation, for example, by changing the characteristic values ​​of one or more layers of neurons, in order to reduce the calculated deviation.

[0051] In the example considered here, where the encoder is integrated into the PCM or ECU of the motor vehicle and the decoder is integrated into the lighting device, the data from the first set of training data are image pairs representing the light beams to be implemented by the motor vehicle lighting device. Such images are also called photometric images.

[0052] However, no restrictions are placed on the data from the first set of training data, which can be any type of image. In examples where the decoder 120 is implemented on a remote server, the data from the set of training data can be, for example, images acquired by an automotive vehicle camera.

[0053] After training with the entire first set of training data, the mean squared error is minimized, and the first autoencoder 200 can reconstruct an optimal quality image from an image having one or more defects following compression of the image. The autoencoder 200 can thus be implemented in the receiver 120 described above.

[0054] The first autoencoder 200 advantageously provides for the reuse of properties of a convolutional layer of neurons by a deconvolutional layer of neurons, which may be permitted by at least one skip connection connecting two non-consecutive convolutional layers of neurons of the first autoencoder 200.

[0055] The first autoencoder 200 includes at least one skip connection 220.1 between pairs of layers having, for example, the same dimensionality or characteristics, each pair including a convolutional layer and a deconvolutional layer. In the example shown in Figure 2, the skip connections 220.1 may therefore include a connection between the input layer 200.1 and the output layer 201.1, a connection between the first convolutional hidden layer 200.2 and the first deconvolutional hidden layer 201.2, and a connection between the second convolutional hidden layer 200.3 and the second deconvolutional hidden layer 201.3.

[0056] Additionally or alternatively, the first autoencoder 200 includes at least one skip connection 220.2 between pairs of layers with different dimensionality or characteristics, each pair including a convolutional layer and a deconvolutional layer. In the example shown in Figure 2, the skip connection 220.2 may include a connection between the first convolutional hidden layer 200.2 and the second deconvolutional hidden layer 201.3, and a connection between the second convolutional hidden layer 200.3 and the first deconvolutional hidden layer 201.2.

[0057] Such skip connections advantageously enable the implementation of complex data processing operations using deep neural networks.

[0058] FIG. 3 illustrates a data transmission system according to an embodiment of the present invention.

[0059] The method includes a preparatory stage 300 comprising a step 301 of acquiring a first set of training data comprising image pairs as described above. No restrictions are placed on how the training data set is acquired. Optimal quality images may, for example, originate from a real-world situation or from a simulation.

[0060] In step 302 of the preparation stage 300, a first autoencoder convolutional neural network 200 is trained by supervised learning based on a first set of image pairs of training data obtained in the previous step 301. The first autoencoder 200 thus obtained can enhance the quality of the compressed data.

[0061] In step 303 of the preparation stage 300, the first autoencoder convolutional neural network 200 is stored in a receiver, such as the receiver 120 described above. As described above, the receiver 120 may be integrated into a lighting device for the motor vehicle or a remote server of the motor vehicle.

[0062] The processing method further includes a current stage 310 comprising a step of receiving 311 by the receiver 120 compressed data via the communication channel 130 described above. In particular, the received data has been previously compressed by the encoder 110 described above, and no restrictions are imposed on the data compression technique.

[0063] In step 312, the decompression module 121 decompresses the compressed data received in the preceding step 311, as described above.

[0064] In step 313, the decompressed data is processed by the first autoencoder 200 to be enhanced. Enhanced data is thus obtained at the end of step 313. Given the machine learning that the first autoencoder 120 produces, the enhanced data allows for optimal image quality, close to the image originally compressed in the encoder 110.

[0065] The enhanced data may be transmitted by receiver 120 in step 314. Receiver 120 may transmit the enhanced data for storage, for example, to a memory. In embodiments where encoder 110 is integrated within a PCM and receiver 120 is integrated within a signaling device, the enhanced data may advantageously be transmitted to a light source control module for performing photometry corresponding to the output data.

[0066] FIG. 4 illustrates a data compression module 111 and a data decompression module 121 according to one embodiment of the present invention.

[0067] As mentioned above, the compression module 111 and decompression module 121 may be capable of implementing compression / decompression algorithms such as linearization-based, gradient-based, JPG, PCA, or other algorithms.

[0068] 4, the compression / decompression is performed using a second autoencoder convolutional neural network, also referred to as the second autoencoder in the following, and the compression and decompression modules 111 and 121 each comprise a part of the second autoencoder. The compression module 111 thus comprises a first part 400 of the second autoencoder, while the decompression module 121 comprises a second part 410 of the second autoencoder.

[0069] The second autoencoder can compress input data, particularly optimal quality images, such as photometric images for a lighting device. The second autoencoder can be constructed through unsupervised learning based on a second set of training data that is different from the first set of training data.

[0070] The second autoencoder consists of an input layer 401 implemented in the encoder 110 and an output layer 411 implemented in the receiver 120, where the input and output layers have the same number of nodes or neurons and therefore the same dimensionality.

[0071] The autoencoder system further includes one or more convolutional hidden layers, each of which has fewer dimensions than the dimensionality of the input layer 401 and the output layer 411.

[0072] The convolutional hidden layer, or central layer, with the minimum dimensionality can exchange a "code" or "latent vector," which is therefore a compressed version of the input data.

[0073] The central layer may therefore be shared between the encoder 112 and the decoder 123 to exchange compressed data, thus reducing throughput requirements and the amount of data exchanged between the encoder and decoder while minimizing loss. The first portion 400 therefore includes a central encoding layer 402, and the second portion 410 includes a central decoding layer 412. The central encoding layer 402 and the central decoding layer 412 may exchange codes or latent vectors that contain fewer dimensions than the input data.

[0074] The second autoencoder is trained through unsupervised learning in such a way as to minimize the mean squared error between the input data and the output data resulting from the output layer for a given dimensionality of the central layer, i.e., a given compression level.

[0075] For this purpose, a second training data set can be presented to the second autoencoder. The training data set can include a set of images, such as photometric lighting images for a vehicle in the example considered here. For each image in the second set of training data, the autoencoder evaluates the mean squared error between the image presented to the input layer 401 and the image provided by the output layer 411, and varies the characteristics of its neurons as well as the number of neurons, or even the number of hidden layers, based on this mean squared error, while maintaining constraints to obtain a latent vector with a given dimensionality. Thus, the aim is to minimize the mean squared error throughout the learning process.

[0076] The latent vector is a compressed version of the input data at a compression ratio CR according to the following formula: - CR=(Nbits * Im_Size - NbitsLV * LVdim) / Nbits * Im_Size where Nbits is the number of bits with which each pixel of the input image is coded, Im_Size is the size of the image in pixels, NbitsLV is the number of bits to code each dimension of the latent vector, which is specified and typically equal to 32 bits, and LVdim is the number of dimensions of the latent vector.

[0077] More generally, Nbits * im_Size represents the size of the input data in units of bits.

[0078] For a given image size, the compression ratio CR can therefore be varied by varying the dimensionality LVdim of the latent vector.

[0079] In particular, the following compression ratios can be obtained: - LVdim=516: CR=92% - LVdim=1024: CR=84% - LVdim=3072: CR=52%

[0080] Therefore, the lower the dimensionality of the latent vector, the higher the compression ratio CR. The selection of the compression ratio may depend on a quality indicator that compares the output data to the input data. Such indicators may include, for example, the peak signal-to-noise ratio (PSNR) or the mean square error (MSE).

[0081] For example, a dimensionality of the latent vector that provides a PSNR greater than a given threshold, such as 30, may be defined.

[0082] The first part 400 and the second part 410 of the autoencoder convolutional neural network are thus obtained and can be implemented in the encoder 110 and the decoder 120, respectively, during the preparation stage 300 described above.

[0083] However, when the communication channel 130 has limited throughput, particularly the CAN bus commonly used between PCMs and lighting devices as in this case, a high compression ratio or even data reformatting is required to enable the transfer of the latent vectors over the communication channel 130. As a result, the decompressed data from the output layer 411 may have imperfections, hence making it advantageous to use the enhancement module 122 described above.

[0084] The PSNR value of the enhanced data may be several points, especially more than 5 points, compared to the decompressed data resulting from the output layer 411 of the second autoencoder. Other indicators such as maximum error and mean squared error are also improved.

[0085] If the compression / decompression is not performed by the second autoencoder, but by one of the compression / decompression algorithms discussed above, the PSNR gain allowed by the enhancement module 122 may even reach 10 points. Other indicators, such as maximum error and mean square error, are also improved.

[0086] FIG. 5 shows the structure of a decoder or receiver 120 according to an embodiment of the invention.

[0087] The decoder 120 includes a processor 501 configured to communicate uni- or bi-directionally via one or more buses or via a wired connection with a memory 502, such as a random access memory (RAM) or a read-only memory (ROM) or any other type of memory (flash, EEPROM, etc.). In a variant, the memory 502 includes several memories of the types mentioned above. The memory 502 is preferably a non-volatile memory.

[0088] The memory 502 permanently or temporarily stores all of the data generated by performing steps 311-314 of the data processing method described above. The memory 502 further stores the first autoencoder convolutional neural network 200 in step 303 described above.

[0089] The memory 502 further stores a decompression algorithm or a second portion 410 of a second convolutional neural network as described with reference to FIG.

[0090] The processor 501 is capable of executing instructions stored in the memory 502 to perform steps 312 and 313 of the method illustrated with reference to Figure 3. Alternatively, the processor 501 may be replaced by a microcontroller designed and configured to perform steps 312 and 313 of the method according to Figure 3.

[0091] The decompression module 121 and enhancement module 122 shown above may therefore be implemented by a processor 501 or a microcontroller. As a further alternative, one processor or one microcontroller is dedicated to the decompression function and another processor or microcontroller is dedicated to the enhancement function.

[0092] The receiver 120 may include an input interface 503 capable of receiving the compressed data in step 311 described above. No restriction is placed on the first input interface 503 being operatively coupled to the communication channel 130 described above.

[0093] The decoder 120 may further include a second interface, namely, an output interface 504, to which the output data may be transmitted in step 314 described above.

[0094] The invention is not limited to the embodiments described above by way of example, but extends to other alternatives.

Claims

1. The following works: - in a preparatory stage (300), supervised learning (302) of a first autoencoder convolutional neural network based on a first set of training data, said first set of training data consisting of image pairs including an image of a given quality and a compressed image obtained by compressing said image of the given quality, said supervised learning being able to minimize the difference between an enhanced image obtained through processing of the compressed image by said first autoencoder convolutional neural network and said image of the given quality pairwise associated with said compressed image; - storing (303) said first autoencoder convolutional neural network in a receiver (120), During the current step (310) performed by the receiver, the method comprises: receiving (311) compressed data from a communication channel (130); - decompressing (312) said compressed data into decompressed data; applying the first auto-encoding convolutional neural network to the compressed data to obtain enhanced data (313); transmitting the enhanced data (314); The data processing method further comprises:

2. 2. The data processing method of claim 1, wherein decompressing the compressed data uses a linearization-based, gradient-based, JPG, or PCA decompression algorithm.

3. 2. The data processing method of claim 1, wherein decompression of the compressed data is performed through processing by layers of neurons including an output layer (411) and at least one convolutional hidden layer (412) of a second autoencoder convolutional neural network, the output layer including a first number of dimensions and the at least one convolutional layer including a second number of dimensions, the second number of dimensions being less than the first number of dimensions.

4. 4. The method of claim 3, wherein the preparing step (300) further comprises unsupervised training of the second autoencoder convolutional neural network based on a second set of training data.

5. The method of any one of claims 1 to 4, wherein the compressed data and the enhanced data represent photometry of a lighting device.

6. The method of claim 5 , wherein the enhancement data is transmitted to a control module of a lighting device for a motor vehicle.

7. 7. The method of claim 5 or 6, wherein the compressed data is received from an encoder (110) of a central control module of a motor vehicle, and the communication channel (130) is a CAN bus.

8. A receiver (120), a memory (502) storing a first autoencoder convolutional neural network capable of obtaining enhanced data from compressed data; a first interface (503) capable of receiving compressed data via a communication channel (130); a processor (501) configured to apply said first autoencoder convolutional neural network to said compressed data so as to decompress said data into decompressed data and to obtain enhanced data; a second interface (504) capable of transmitting said enhanced data; a receiver (120) including:

9. 10. A system including the receiver (120) of claim 8, an encoder (110) capable of receiving input data, compressing the input data into compressed data, and transmitting the compressed data to the receiver over a communication channel (130).

10. 10. The system of claim 9, wherein the encoder (110) is integrated into a central control module of a motor vehicle, the receiver (120) is integrated into a lighting device of the motor vehicle, the communication channel (130) is a CAN bus, and the input data represents lighting photometry.

Images