Method and computing system for generating a compressed version of original image data

Abstract

A method for generating a compressed version (CI) of original image data (OI) comprises the steps of feeding the original image data (OI) into an autoencoder (1) comprising an encoder (11) and a decoder (13),and, in response, receiving reconstructed image data (RI) from the autoencoder (1), and iterating the previous step at least once. After the one or more iteration steps the compressed version (AEI) of the original image data (OI) is provided, wherein the compressed version (CI) is represented by encoded image data (EI) received from the encoder (1) in the last iteration step and by parameters defining at least the decoder (13) after the last iteration step.

Description

[0001] Method and computing system for generating a compressed version of original image data

[0002] Technical Field

[0003] The invention refers to a method and a computing system for generating a compressed version of original image data.

[0004] Background Art

[0005] Image compression contributes to reducing the amount of space or bandwidth needed to store or transmit image data. Image compression can be useful in a variety of daily situations, such as, but not limited to, saving images in a smartphone or laptop, or when downloading, uploading or transmitting an image file to / from / via the internet. Compression can save time and resources.

[0006] Disclosure of the Invention

[0007] A method is provided for generating a compressed version of original image data, the method comprising the steps of: feeding the original image data into an autoencoder and, in response, receiving reconstructed image data from the autoencoder; iterating the previous step at least once. After the one or more iteration / s the compressed version of the original image data is provided, wherein the compressed version is represented by encoded image data received from an encoder of the autoencoder in the last iteration step and by parameters defining at least a decoder of the autoencoder after the last iteration step. Preferably, in each iteration, the autoencoder encodes the original image data into encoded image data, and reconstructs the original image data from the encoded image data.

[0008] The autoencoder comprises the encoder and the decoder and is a machine learning model, preferably an artificial neural network (ANN), preferably a convolutional neural network, preferably comprising a deep learning model.

[0009] The generic functionality of the autoencoder is to improve the encoding of original image data into a latent space representation, and decoding the latent space representation into reconstructed image data. For encoding, the autoencoder preferably comprises the encoder which is a means for producing output data, i.e. encoded data, from input data thereby reducing dimensionality of the input data. Accordingly, the output data represents the input data in a more compact and more efficient way. The input data preferably is a vector of given dimensionality, and the output data is a more compact representation of the input vector of lower dimensionality. Accordingly, the encoder encodes and thereby compresses the input vector.

[0010] The input data or input vector presently is image data, also referred to original image data. The original image data may represent a digital image, e.g., produced by a camera, e.g. stored in an image file in a suitable file format such as png, bmp, tif, etc., which original image data is desired to be compressed. Preferably, the original image data represents data in an image space. The original image data may also be part of video data.

[0011] The output data of the encoder is a representation of the input data in latent space. The output data is also referred to as latent space representation of the input data, i.e., a latent space representation of the original image data, i.e. encoded image data. In contrast to the image space, the latent space representation of the original image data is hidden in the latent space, and requires reconstruction prior to representing a visible image again.

[0012] Preferably, the encoder detects features in the original image data. By means of extracting features from and / or classifying the original image data, the original image data can be represented by a collection of features encoded as latent space representation or encoded image data of lower dimensionality than the original image data. Features may include one or more of structural features such as edges, angles, curves, shapes, etc. color features, patterns, and / or other features in the original image suited for classifying an image. The latent space representation preferably is a representation of the input data absent noise or redundant information, solely comprising relevant information, which effects its lower dimensionality compared to the original image data.

[0013] Preferably, the encoder is implemented by a convolutional neural network comprising convolutional layers, preferably between 2 and 4 convolutional layers, preferably, two, or three, or four convolutional layers. Such convolutional network of the encoder is suited to extract features from an image. Each convolutional layer receives the output of the previous convolutional layers as input, wherein the first convolutional layer receives the original image data, or pre-processed original image data as input. In each convolutional layer one or more filters are applied, which filters are also referred to as image kernel or kernel matrix, used to detect features in the original image data. Preferably, the weights of such filters are iteratively determined, e.g. by optimizing loss functions. Such weights preferably represent parameters defining the encoder. Other elements, next to the weights, such as bias, channel weights, etc. may also be iteratively adapted, and may also represent parameters defining the encoder. Other parameters of the convolutional neural network model, or the machine learning model in general, may be predefined, such as the number of convolutional layers, the number of filters or kernels, a stride size, a padding parameter, pooling, etc., and may also contribute to the parameters defining the encoder . The design of such predefined parameters which - in contrast to the weights - do not change during operating the encoder, may have an impact on the overall storage size of the encoder, the operating time during training, and the quality of the result of the training. Preferably, the stride is set to two (pixels), while padding is applied, preferably by one pixel around the original image data. Preferably, the one or more convolutional layers encode original image data of k x 1 pixels and a channel of size m into an encoded image of n x o pixels and a channel of size p, wherein n and o are preferably at least ten times smaller than k and 1, respectively, and wherein p exceeds m. E.g., original image data of pixel size 256 x 256 and channel of size 3 (e.g. representing RGB colours) is encoded into encoded image data of pixel size 8 x 8, and channel of size 8.

[0014] Preferably, the decoder is also implemented by a convolutional neural network comprising convolutional layers, preferably between 2 and 4 convolutional layers, preferably, two, or three, or four convolutional layers. Such convolutional network of the decoder is suited to reconstruct the original image data to its best from the encoded image data. Each convolutional layer receives the output of the previous convolutional layers as input, now bottom up, wherein the first convolutional layer receives the encoded image data as input. In each convolutional layer one or more filters are applied, which filters are also referred to as image kernel or kernel matrix, used to decode the encoded image data. Preferably, the weights of such filters are iteratively determined, e.g. by optimizing loss functions. Such weights preferably represent parameters defining the decoder. Other elements, next to the weights, such as bias, channel weights, etc. may also be iteratively adapted, and may also represent parameters defining the decoder. Other parameters of the convolutional neural network model, or the machine learning model in general representing the decoder may be predefined, such as the number of convolutional layers, the number of filters or kernels, a stride size, a padding parameter, pooling, etc., and may also contribute to the parameters defining the decoder. The design of such predefined parameters which - in contrast to the weights - do not change during operating the decoder, may have an impact on the overall storage size of the decoder, the operating time during training, and the quality of the result of the training. Preferably, the stride is set to two (pixels), while padding is applied, preferably by one pixel around the original image data. Preferably, the one or more convolutional layers decode encoded image data of pixel size 8 x 8, and channel of size 8 into reconstructed image data of pixel size 256 x 256 and channel of size 3 (e.g. representing RGB colours). Generally, encoded image data of n x o pixels and a channel of size p maybe decoded into reconstructed image data of k x 1 pixels and a channel of size m, wherein n and o are preferably at least ten times smaller than k and 1, respectively, and wherein p exceeds m.

[0015] Traditionally, an autoencoder - before being operatively used - is fed with multiple different images to enable the encoder to improve its capability to detect features in images. Such process is referred to as training. Hence, when the trained autoencoder receives a new image, it is capable of detecting the features in the new image. The process of learning and detecting features in the input data including comparing the detected features with features of images of the previous training sets is hidden. The encoder is responsible for learning and detecting features in the input data and for simplifying the corresponding data representation in the latent space.

[0016] The autoencoder preferably also comprises the decoder that reconstructs the latent space representation into reconstructed data. Presently, given that the input data for the autoencoder is original image data in the image space, the reconstructed data is reconstructed image data, again in the image space. The reconstructed image data resembles the original image data, however, is not identical in view of the transformation of the original image data into the latent space representation, i.e. the encoded image data, and the transformation back from the latent space representation into the reconstructed image data.

[0017] Preferably, during training the autoencoder with images, weights of the preferred convolutional neural network models of the encoder and the decoder are adapted. The training preferably involves an evaluation of the output data i.e. the reconstructed image data versus the input data, i.e. the original image data. Such evaluation preferably is based on a loss function determined for the reconstructed image data versus the original image data. Subject to the loss, the weights of the autoencoder model are adjusted in order to reduce the loss function for future original image data of the same kind, e.g. having similar features. Accordingly, the autoencoder learns efficient encoding and decoding during training, i.e. while being fed with training image data. Traditionally, the training image data comprise a multitude of different original image data. By varying the training input data as much as possible, the autoencoder learns to be prepared to new input data and provides, in view of the training, the best possible result in encoding and decoding.

[0018] In contrast, in the presently provided method and computing system, the autoencoder is not fed with loads of different images, i.e. different original image data, but with the same original image data, i.e. with the same image, again and again, at minimum twice, however, preferably more than 50 times, preferably more than 100 times, preferably more than 200 times, preferably more than 400 times, preferably between 400 and 600 times. In other words, the autoencoder is misused by not being trained with very different input data but is trained with the same original input data again and again, preferably in two or more iterations, also referred to as iteration loops. In case the feeding step of the encoder with the original image data, the encoding into the latent space representation, the decoding of the reconstructed image data from the latent space representation, the determination of the loss between the reconstructed image data and the original image data, and the subsequent adaptation of the weights of the encoder and decoder are considered as a training step, the autoencoder is no longer trained by high volumes of different original images, but by the same image again and again in order to make the autoencoder adapt to and learn this very specific original image, and improve its en- and decoding in each training step.

[0019] The autoencoder that is provided at the beginning is preferably an untrained autoencoder, or a slightly trained autoencoder. Accordingly, such autoencoder is then trained on this single original image data, resulting in weights being improved in each training step, and preferably resulting in stepwise improved reconstructed images, wherein an improved reconstructed image implies a lower loss than in the previous step / s. Hence, by the present method and computing system, the autoencoder is trained to optimize the encoding and decoding for a very specific original image data, i.e. the autoencoder provided at the beginning is not trained and hence is not specifically designed for encoding and decoding arbitrary original images, but is trained to encode and decode a specific original image.

[0020] Once the training of the autoencoder with the very same original image data is stopped, i.e. iteratively feeding the autoencoder by the same original image data is terminated, the most recent encoded image data contributes to the compressed version of the original image data, as well as parameters defining at least the decoder of the autoencoder in this very state, i.e. after or during the very last iteration. Accordingly, the encoded original image data in combination with the parameters defining the decoder model during or after this last iteration step represents a compressed version of the original image data, that preferably is stored. In an alternative approach, the parameters of the autoencoder encoding both the encoder and the decoder is stored in combination with the encoded image data.

[0021] The encoded image data on its own is not considered as sufficient stand-alone data representing the original image data in a compressed version given a decoding algorithm is required to be known for decoding the encoded image data. The autoencoder in this very state the encoded image data is taken from, however, includes such decoding model. Accordingly, in addition to the encoded image data, the model of the autoencoder, and at least the model of the autoencoder's decoder allowing for a reconstruction of the original image data from the encoded image data - in a state at which the iterative training is stopped - contributes to the compressed version of the original image data. The decoder model preferably is described or defined by its parameters. Parameters defining the decoder of the autoencoder may be sufficient on its own. In an alternative, the parameters define both the decoder and the encoder. In the first embodiment, encoder and decoder are preferably separate neural network models and parameters may be assigned individually to encoder and decoder. In the latter embodiment, the autoencoder may be represented by a common neural network, in which parameters define both the encoder and the decoder. In a different embodiment, the parameters of the encoder at the same time define the decoder as inverse. At least, the parameters that allow decoding of the encoded image data contribute to the compressed version.

[0022] Accordingly, what is provided as compressed version of the original image data preferably includes at least the weights at the time the iterations are stopped, and, preferably, the setup of the decoder model, e.g. being defined by the number of e.g. convolutional layers, the number of filters per layer, the sizes of the filters, the number of channels. Accordingly, the trained decoder model, at the time of terminating the training, in combination with the encoded image data, represents a compressed version of the original image data, i.e. requires less storage space than the original image data. Hence, the method relies on an autoencoder, preferably a form of an artificial neural network, used for the purpose of image compression. The autoencoder preferably comprises two core components, i.e. an encoder, which reduces original image data supplied to or fed into the encoder to a lower-dimensional encoded representation, preferably by extracting essential features, and a decoder, which reconstructs an approximation of the original image from this encoded representation. Together, encoder and decoder achieve size reduction while maintaining the image's vital information.

[0023] In one embodiment, the number of iterations, i.e. the number of training steps, is a predefined, fixed number, e.g. determined based on trials. The number of iterations impacts the duration of the compression method. In case it is very high, the compression takes a longer time, however, the resulting autoencoder provides a better quality representation of the original image data, possibly also at the cost of storage space the resulting autoencoder requires. On the other hand, less iterations improve the compression time at the cost of the quality of the representation of the original image data.

[0024] In a different embodiment, the number of iterations is variable and varies from image to be compressed to image to be compressed. Preferably, the number of iterations made dependent on the calculated loss after an iteration. Accordingly, after an iteration (or at the end of an iteration, in case loss is determined as part of the training), a loss, preferably a mean square error, is calculated. Subject to this loss determined after the most recent feeding of the encoder with the original image data, another iteration is initiated, or the iterating process is stopped. Preferably, a threshold is defined which distinguishes the determined loss into a loss sufficiently low not to continue training the autoencoder, or a loss not yet being sufficiently low, such that the autoencoder is fed once more with the original image data. It is desired that such additional feeding of the autoencoder improves the loss, i.e. lowers the loss in the next iteration.

[0025] In case the criterion for stopping the training of the autoencoder is fulfilled, which criterion e.g. is the meeting or undercutting of a threshold by the loss, the autoencoder in this very state, i.e. at least the decoder defined by its weights at this point in time of stopping the training, in combination with the encoded image data, represents the compressed version of the original image data.

[0026] The loss determined for evaluating whether to stop or continue the training, may be the same loss used in the autoencoder for improving its parameters, i.e. the weights of its model, such that the loss is processed only once for two purposes, i.e. for learning and as a stop criterion. In a different variant, the loss functions may be different ones for the learning on the one hand, and for defining the exit criterion from the iteration loop on the other hand.

[0027] Processing the loss function may be assigned to the autoencoder and considered as task within the autoencoder. In a different embodiment, determining the loss function, at least for defining the exit from the loop / the iterations, is performed outside the model of the autoencoder.

[0028] Preferably, the machine learning model of the decoder or the autoencoder includes at least the weights as determined in the most previous iteration step at the time of stopping the iteration loop. At least these weights, in combination with the encoded image data, represent the compressed version of the original image data, which preferably is stored in a storage and / or is transmitted over a transmission network.

[0029] In a preferred embodiment, the method is executed computer-implemented. This implies, that sequentially feeding the original image data into the autoencoder for a number of times is performed by a software routine executed by a processing unit. Preferably, the autoencoder is provided in an untrained state. The autoencoder preferably is a software module, and preferably is stored in a storage. The processing unit has access to the autoencoder and is capable of feeding the original image data into the autoencoder, which original data may be stored in the same or a different storage.

[0030] Summarizing, the method preferably makes use of an input image that is to be compressed. Such input image is referred to as original input data, and preferably encompasses a single, digital image in one variant. In a different variant, the original image data includes a series of input images, and, may as such also represent a video. The original image data serves as a starting point for the compression process, and preferably is fed into an encoder which encoder is a component of a preferably not yet trained autoencoder. The encoder preferably processes the input image through its multiple layers. Each layer preferably transforms the input image, reduces its dimensionality and extracts key features. The encoder's task is to learn the most important aspects of the image, effectively capturing its essence in a smaller, more compact form.

[0031] This process continues until the image is transformed into a lower-dimensional representation, known as the latent space representation. The latent space representation is also referred to as encoded image data. This representation is a compact encoding of the image's key features, distilled by the encoder. It serves as a bottleneck in the network, ensuring that the compression is efficient by retaining only the most critical information required to reconstruct the image.

[0032] Following the compression, preferably a decoder component of the autoencoder takes over. It receives the latent space representation and begins the process of reconstructing the image. Starting from this compressed form, the decoder works to gradually increase the representation's dimensionality, layer by layer, aiming to recreate the original image. The decoder preferably mirrors the encoder's structure but operates in reverse, i.e. transforms the compressed encoding back into a reconstructed image, aka reconstructed image data.

[0033] Finally, the decoder provides an image at its output, which is a reconstruction of the original image based on the compressed data. This reconstructed image approximates the original input, reconstructed from the latent space representation. The fidelity of the reconstructed image to the original image depends on the autoencoder's ability to capture and preserve the most significant features during compression.

[0034] Conventional autoencoders are large neural network models trained by millions of different original images, in order to improve the model's ability to compress any new original image data irrespective of content. In such conventional approach, the autoencoder trained in such way is required to decode any arbitrary original image data at its best. Accordingly, in the conventional approach, the encoded image data is stored in combination with the large autoencoder model for decoding, given that encoded image data absent the decoding means is of no value. In contrast, in the present approach, at least the decoder model preferably is stored in combination with the encoded image data. The decoder model is a small model, given that was trained with the same original image data again and again and is preferably designed upfront to small scale. Hence, the capability of such autoencoder is not focused on en- and decoding any arbitrary original image data, but is for en- and decoding a single image, i.e. the single original image data only. Accordingly, a smaller autoencoder is required, such that its parameters require less storage capacity than an autoencoder model that is trained to encode arbitrary original image data. Accordingly, the compressed version of the original image data presently includes the encoded image data and the parameters defining the decoder trained by the very same original image data, at a time when the iterative training of the autoencoder by the original image data is stopped.

[0035] Hence, each original image to be compressed is trained specifically on the previously untrained autoencoder to optimize the compression and reconstruction accuracy. The training involves inputting the original image multiple times into the autoencoder, thereby adjusting weights of the encoder and decoder to minimize the difference between the input and output images, thereby ensuring efficient compression and high-quality reconstruction.

[0036] Hence, in case two original images are desired to be stored in a compressed version, presently the two respective encoded images are stored, each in combination with the assigned decoder each trained by its assigned original image, the autoencoders being represented by corresponding parameters. Accordingly, in this approach, each original image data requires, in its compressed version, the storage of at least the corresponding decoder specifically trained by this very original image data. Or, in different words, each autoencoder and corresponding decoder is different per original image. At least the latter, i.e. the trained decode, is made available, i.e. stored, in combination with the corresponding encoded image data, in order make the encoded image data reconstructable.

[0037] According to another aspect of the present invention, a computer program is provided. The computer program comprises computer program code elements configured to carry out the method according to any one of the above embodiments, when executed on a processing unit.

[0038] According to a further aspect of the present invention, a computing system is provided for generating a compressed version of original image data. The computing system comprises an autoencoder, and a processing unit configured to conduct a method according to any of the preceding embodiments.

[0039] In a preferred embodiment, the autoencoder is represented by a machine learning model, preferably by an artificial neural network model, preferably by a convolutional neural network model, preferably comprising an encoder and a decoder, each comprising convolutional layers and weights assigned to filters of each layer. The convolutional neural network model preferably comprises between 2 and 4 convolutional layers, preferably 2 or 3 convolutional layers.

[0040] In another preferred embodiment, the computing system comprises a storage, wherein the processing unit is configured to store the compressed version of the original image data in a state after the one or more iteration steps. And / or the compressed version of the original image data are transmitted over a transmission network.

[0041] According to a further aspect of the present invention, a compressed version of original image data is generated by a computing system or by a method according to any of the preceding embodiments.

[0042] According to a further aspect of the present invention, a usage of an autoencoder is provided that is trained multiple times with the same original image data. After training, the model of at least a decoder of the autoencoder, preferably represented by its parameters, in combination with the encoded image data is used as a compressed version of the original image data. Other advantageous embodiments are listed in the dependent claims as well as in the description below.

[0043] Brief Description of the Drawings

[0044] The invention will be better understood and objects other than those set forth above will become apparent from the following detailed description thereof. Such description makes reference to the annexed drawings, wherein:

[0045] Figure i shows a schematic diagram of an autoencoder as used in a method and a computing system for generating a compressed version of original image data, according to embodiments of the present invention;

[0046] Figure 2 shows a block diagram of a computing system for generating a compressed version of original image data, according to an embodiment of the present invention; and

[0047] Figure 3 illustrates a flow chart illustrating a training method according to and embodiment of the present invention, as well as a method according to an embodiment of the present invention.

[0048] Modes for Carrying Out the Invention

[0049] Figure 1 illustrates a schematic diagram of an autoencoder as used in a method and a computing system for generating a compressed version of original image data, according to embodiments of the present invention.

[0050] The autoencoder 1 is represented by software model and comprises an encoder 11 and a decoder 13. Original image data 01 is input into the encoder 13 of the autoencoder 1. The original image data 01 preferably is a digital image. The original image data 01 maybe pre-processed prior to being fed into the encoder 11, or alternatively, may be pre-processed within the encoder 11, in order to fit the input requirements of the encoder 11. The encoder n preferably is implemented as a machine learning model, preferably as an artificial neural network also referred to ANN, and preferably a convolutional neural network also referred to as CNN, which comprises two convolutional layers indicated by in, a number of filters per layer, weights defining the filters, and / or channels for additional attributes of an image, such as colour coding.

[0051] In case of original image data 01 being input to the encoder n, the encoder n transforms the original image data 01, by means of the original image data 01 passing through the layers ill, into a virtual space, also referred to as latent space 12. In the latent space 12, the original image data 01 is represented by a latent space representation LSR thereof, i.e. encoded image data El, which encoded image data El is of lower dimension than the original image data 01.

[0052] A decoder 13 of the autoencoder 1 can be understood as inverse of the encoder 11, at least in its functionality. The decoder 13 receives the encoded image data El as provided by the encoder 11, and reconstructs the original image data RI from the encoded image data EL However, given that the encoding process reduces the dimensionality of the original image data 01, i.e. compresses the original image data 01, the reconstruction of the latent space representation LSR is reconstructed image data RI, that resembles the original image data 01, but is not identical to the original image 01.

[0053] The decoder 13 preferably is implemented as a machine learning model, preferably as an artificial neural network also referred to ANN, preferable as a convolutional neural network also referred to as CNN, which comprises - presently two - convolutional layers indicated by 131, filters per layer, weights defining the filters, and / or channels.

[0054] The decoder 13 transforms the latent space representation LSR, by means of passing the latent space representation LSR through the layers 131, back into the image space again, and adds dimension compared to the latent space representation LSR. Accordingly, the reconstructed image data RI no longer is a (significantly) compressed version of the original data 01, but is close to the original image data 01, however, not identical to the original image data 01 given that it was reconstructed from a compressed version of the original image data 01, i.e. the latent space representation LSR. The reconstructed image data RI is a temporary representation at the output of the decoder 13 during one iteration step of training the autoencoder 1.

[0055] Figure 2 illustrates a block diagram of a computing system for generating a compressed version of original image data, according to an embodiment of the present invention.

[0056] The computing system comprises a processing unit 2 and a storage 3. The processing unit 2 maybe the processing unit of a computing entity, such as a personal computer. The processing unit 2 may be a remote processing unit in the cloud or may be a distributed processing unit. The storage 3 may be a permanent storage such as an HDD or a flash memory, and preferably is a storage 3 controlled by the processing unit 2 for fetching data from and depositing data on. For this purpose, the storage 3 may not necessarily be located at the location of the processing unit 2, but may e.g. be in the cloud, while the processing unit 2 is local, or vice versa.

[0057] The computing system further comprises an autoencoder 1. The autoencoder 1 of Figure 2 can be embodied by the autoencoder 1 illustrated in Figure 1. Same reference numerals are used. The autoencoder 1 is a software model for learning ways to encode and decode input data. Accordingly, in an initial state, the untrained or semi-trained model of the autoencoder 1, also referred to as untrained autoencoder model UAE, preferably is stored in the storage 3. It may be sent to the storage 3 via a transmission network 4. The untrained autoencoder model UAE is at least untrained in the sense, that is not trained with the specific, individual original image data that is desired to be compressed.

[0058] The original image data 01, e.g. representing a single digital image, preferably is also stored in the storage 3, as is indicated in Figure 2. The original image data 01 may e.g. represent a picture taken by a digital camera or from a database, and may also ma supplied to the storage 3 via the transmission network 4.

[0059] The processing unit 2 is configured to, by means of a computer program, run a process for compressing the original image data 01, thereby using the autoencoder i. Accordingly, the processing unit 2 fetches the original image data 01 from the storage 3 or elsewhere, and feeds the autoencoder 1 with the original image data 01. Specifically, and as explained in connection with Figure 1, the original image data 01 is supplied to the encoder 11 of the autoencoder 1. The encoder 11 encodes the original image data 01 into a latent space representation LSR in the latent space 12. For this purpose, it is preferred that the encoder 11 is embodied as convolutional neural network, hence as machine learning entity, which preferably is trained in encoding original images 01 into a latent representation of lower dimensionality in the latent space 12, i.e. encoded image data El.

[0060] The latent space representation LSR is input to a decoder 13 of the autoencoder 1. Accordingly, the decoder 13 reconstructs reconstructed image data RI from the latent space representation LSR of the original image data 01. The reconstructed image data RI maybe stored in the storage 3 if desired.

[0061] In a next step, the processing unit 2 is configured to process a loss of the reconstructed image data RI versus the original image data 01. For example, a mean square error is processed for the reconstructed image data RI over the original image data 01. The processing unit 2 then is configured to compare the determined loss with a threshold, and evaluates is the determined loss exceeds the threshold, e.g. threshold T. If so, the processing unit is configured to, again, feed the very same original image data 01 into the autoencoder 1, specifically into the encoder 11 of the autoencoder 1.

[0062] Given that the autoencoder 1 is represented by a machine learning model, it aims at improving the encoding of the original image data into the latent space representation, and decoding the latent space representation into reconstructed image data. Accordingly, the autoencoder learns efficient encoding and decoding, typically, i.e. in its conventional usage byway of being trained with loads of different training data, in case of images, different original image data. Given that the autoencoder 1 is machine learning model, it determines the quality of its output versus its input, in order to learn. Accordingly, in traditional autoencoders, the reconstructed image is assessed with respect to the original image data, also by way of a loss function, for example. In response to the loss determined, the parameters of the autoencoder, such as weights of the autoencoder model are adjusted. Traditionally, an autoencoder is trained by loads of different training data in order to improve the encoding and decoding irrespective what input data will be fed into the autoencoder. By varying the training input data as much as possible, the autoencoder learns to be prepared to never before seen input data and provides, in view of the training, a best possible result in encoding and decoding.

[0063] Accordingly, traditionally, the parameters of the autoencoder are adapted subject to the iterative feeding of the encoder with loads of different training data. Instead, the present autoencoder is trained multiple times, i.e. at least twice but preferably more often, with the same, i.e. identical training data. Accordingly, the autoencoder is trained to optimize the encoding and decoding of a very specific original image data, i.e. is optimized not for encoding and decoding arbitrary input data or arbitrary original images, but is trained to encode and decode a specific original image to the best. While the working principle of the autoencoder remains the same, by every iteration of the original image data being fed into the autoencoder, a loss function is generated at the decoder output for determining the loss between the reconstructed image data and the original image data. This loss is also used in the present autoencoder to improve the autoencoder's machine learning model, i.e. to adapt the parameters of the autoencoder to improve the encoding and decoding with every iteration. However, in addition, the determined loss is used to decide when to stop training the autoencoder with the same training data i.e. the original image data. In one variant, the loss function between the reconstructed image data and the original image data is the same for learning, i.e. for adapting the parameters of the autoencoder model, and for deciding when to stop iterating the feeding of the autoencoder with the same original image data. In a different variant, the loss functions may be different ones for training purposes on the one hand, and for defining the exit criterion from the iteration loop. In one embodiment, determining the loss function may be assigned to the autoencoder and be represented by a task within the autoencoder. In a different embodiment, determining the loss function, at least for defining the loop exit is performed by the processing unit within a routine that feds the autoencoder.

[0064] Finally, the processing unit stops the training process in case the determined loss is equal or below a given threshold. This is taken as an indicator that the encoded image data El is of a sufficient quality to contribute to a compressed version CI of the original image data 01. In addition to the encoded image data El, parameters defining the then trained decoder, or the then trained complete autoencoder, referred to as TAE, contribute to the compressed version CI of the original image data 01. The compressed version CI preferably is stored in the storage 3, or is transmitted via the transmission network 4.

[0065] Figure 3 illustrates a flow chart illustrating a method according for generating a compressed version of original image data according to an embodiment of the present invention.

[0066] In step Si, an autoencoder is provided. In step S2, original image data is provided, e.g. representing a digital original image. In step S2 the original image data maybe pre-processed, if needed (optional step). In step S3, the original image data is fed into the autoencoder, is encoded into encoded image data, and reconstructed image data reconstructed from the encoded image data and is provided by the autoencoder in step S4. In step S5, a loss between the reconstructed image data and the original image data is processed. In step S6 the loss is evaluated. Specifically, it is evaluated if the processed loss is above a threshold. If not (N), it is continued with step S3, i.e. the same original image data again is fed into the autoencoder. By this means, the autoencoder is trained once more to improve the encoding of the original image data and the subsequent decoding. During such training step, it can be expected, that weights of the machine learning model of the autoencoder, and specifically weights of nodes assigned to layers of an artificial neural network representing such machine learning model are adapted in order to provide a more efficient compression of the original image data into a latent space representation, preferably leading to a latent space representation of lower dimension that after the first training of the encoder with the same original im- age data, and to improve the decoding / reconstructing of a reconstructed image from the latent space representation which, in the second iteration loop of training the autoencoder with the same original image data comes closer to the original image data than after the first iteration loop, i.e. the first training step. Accordingly, it can be expected, that the reconstructed image data after the second training step shows a lower loss than the reconstructed image after the first iteration step / training step.

[0067] Accordingly, after each iteration loop / training step, the reconstructed image data is expected to improve in terms of loss versus the original image data, owed to the training of the autoencoder machine learning model.

[0068] Preferably, it is expected, that at minimum 50 iterations, preferably more than 100, preferably more than 200 iterations, preferably more than 400 iterations, preferably between 400 and 600 iterations are required to get to a sufficiently reconstructed image data, wherein sufficient implies that the loss is less than the threshold. However, the lower the threshold is defined, the more iterations are required to reach / fall under the loss threshold. On the other hand, the lower the threshold is set for the loss, the more complex the autoencoder may get in terms of nodes, channels, layers, if those parameters are not restricted. In turn, the autoencoder itself requires more storage capacity, the better the reconstructed image data becomes. However, since finally, the trained autoencoder, at least its decoder, in combination with the original image data encoded by this trained autoencoder, serves as a compressed representation of the original image data, aiming at the lowest loss possible has to be weighed against the storage space the autoencoder requires.

[0069] In case, the threshold is achieved by the loss, or undercut (Y), the autoencoder, at least its decoder, with the then valid weights, nodes, layers, collectively referred to as parameters, is provided in combination with the encoded image data as a representation of a compressed version of the specific original image data.

[0070] In step S8, the autoencoder model is stored, or transmitted to a destination, e.g. a user requesting for the compression of the original image data, and in step S9 the process for compressing original image data is terminated.

Claims

Claims1. Method for generating a compressed version (CI) of original image data (01), comprising the steps of: feeding the original image data (01) into an autoencoder (i) comprising an encoder (n) and a decoder (13), and, in response, receiving reconstructed image data (RI) from the autoencoder (1), iterating the previous step at least once, after the one or more iteration steps providing the compressed version (CI) of the original image data (01), wherein the compressed version (CI) is represented by encoded image data (El) received from the encoder (11) in the last iteration step and by parameters defining at least the decoder (13) after the last iteration step.

2. Method according to claim 1, wherein the number of iterations is more than 50, preferably more than 100, preferably more than 200, preferably more than 400, preferably between 400 and 600.

3. Method according to claim 1 or claim 2, storing the compressed version (CI) of the original image data (01), or transmitting the compressed version (CI) of the original image data (01) over a transmission network (4).

4. Method according to any of the preceding claims, comprising after each feeding step determining a loss between the reconstructed data (RI) and the original image data (01), wherein the number of iterations is dependent on the determined loss, preferably wherein the loss applied is a mean square error.

5. Method according to claim 4, wherein the launching of another feeding step is determined dependent on the most recent determined loss.

6. Method according to claim 4 or claim 5, wherein the launching of another feeding step is determined based on the most recent determined loss exceeding a threshold.

7. Method according to any of the preceding claims 4 to 6, comprising launching another feeding step if the most recent determined loss exceeds the threshold, and else stopping the iteration loop and providing the compressed version (CI) of the original image data (01).

8. Method according to any of the preceding claims, wherein in each feeding step the encoder (11) encodes the original image data (01) into the encoded image data (El) representing a lower dimensional representation (LSR) of the original image data (01) and the decoder (13) decodes the encoded image data into the reconstructed image data (RI), the reconstructed image data (RI) representing an approximation of the original image data (01).

9. Method according to any of the preceding claims, wherein in each feeding step, the encoder (11) is fed by the same original image data (01), preferably wherein the parameters defining the autoencoder (1) vary between the iterations, preferably wherein the encoded image data (El) varies between the iterations, and preferably wherein the reconstructed image data (RI) varies between the iterations.

10. Method according to any of the preceding claims, wherein the autoencoder (i) is represented by a machine learning model, preferably by an artificial neural network model, preferably by a convolutional neural network model comprising convolutional layers (111,131) and weights assigned to each convolutional layer (111,131), wherein the parameters defining the autoencoder (1) at least include parameters, preferably weights, enabling to reconstruct the original image data (01) from the encoded image data (El), preferably wherein the parameters comprise first parameters defining the encoder (11) and second parameters defining the decoder (13).

11. Method according to any of the preceding claims, comprising computer implemented executing the steps of the method.

12. Method according to any of the preceding claims, comprising providing the autoencoder (1) to be iteratively fed by the original image data (01), which autoencoder (1) is represented by a machine learning model, preferably by an artificial neural network model, preferably by a convolutional neural network model comprising convolutional layers (111,131) and weights assigned to each convolutional layer (111,131), and which convolutional neural network model comprises between 2 and 4 convolutional layers (111,131), preferably 2 or 3 convolutional layers (111,131).

13. Computer program comprising computer program code elements configured to carry out the method according to any one of the preceding claims 1 to 11 when executed on a processing unit (2).

14. Computing system for generating a compressed version (C I) of original image data (01), comprising- an autoencoder (1),a processing unit (2) configured to conduct a method according to any of the preceding claims 1 to 11.

15. Computing system according to claim 14, wherein the autoencoder (1) is represented by a machine learning model, preferably by an artificial neural network model, preferably by a convolutional neural network model, comprising an encoder (11) and a decoder (13), each comprising layers (111,131) and weights assigned to nodes of each layer (111,131), and wherein the convolutional neural network model comprises between 2 and 4 convolutional layers (111,131), preferably 2 or 3 convolutional layers (111,131).

16. Computing system according to claim 14 or claim 15, comprising- a storage (3), wherein the processing unit (2) is configured to store the compressed version (CI) of the original image data (01) and / or- a transmission network (4), wherein the processing unit(2) is configured to transmit the compressed version (CI) of the original image data (01) over the transmission network (4).

17. A compressed version (CI) of original image data (01) generated by a computing system or by a method according to any of the preceding claims.

18. Usage of an autoencoder (1) comprising an encoder (11) and a decoder (13) trained in multiple iterations with the same original image data (01) for generating a compressed version (CI) of the original image data (01), the compressed version (CI) being represented by encoded image data (El) received from the encoder (11) in the last iteration step and by parameters defining at least the decoder (13) after the last iteration step.

Images