Method and apparatus for steganographic processing and compression of image data

Abstract

The invention relates to a method for steganographic processing and compression of image data with negligible information loss, wherein said image data comprises noise and information. The method comprises the steps of: - acquiring image data (210) to be processed and / or compressed for storage and / or transmission, - preparing (235) the acquired image data (210) for compression, said preparing step (235) comprising the steps of: ° determining an input noise model (215) and corresponding parameters adapted to reflect the noise produced by a source image sensor (SA) used to capture said image data, ° removing noise (305) from the acquired image data (210) by using said input noise model (215), with negligible information loss, to produce denoised image data (325), - storing and / or transmitting said image data, the step of preparing (235) the acquired image data (210) further comprising the steps of: ° determining a target noise model (220) and corresponding parameters adapted to reflect the noise produced by a real or ideal target image sensor (SB), ° generating (320) pseudo-noise according to said target noise model (220), and ° adding (330) the generated pseudo-noise to said denoised image data (325) in a steganographic manner, so as to produce pseudo-noise image data (335) which is statistically equivalent to image data acquired with said target image sensor (SB), and said storing and / or transmitting step comprises: ° storing and / or transmitting said pseudo-noise image data (335) to allow reproduction and / or subtraction of said pseudo-noise based on said pseudo-noise image data (335) and the parameters (220) of said target noise model.

Description

TECHNICAL FIELD

[0001] The present invention relates to a method for steganographic processing and compression of image data with negligible loss of information and a corresponding device, wherein the image data comprises noise and information. The method comprises the steps of acquiring image data and preparing the acquired image data, wherein the preparing step comprises the steps of determining an input noise model and corresponding parameters adapted to reflect the noise created by a source image sensor used to capture the image data, removing the noise from the acquired image data with negligible loss of information, e.g. to produce noise-reduced image data, the method further comprising the step of storing and / or transmitting the image data. BACKGROUND

[0002] The number of digital images and videos produced every year has been accelerating. For decades, compression has played a crucial role in enabling efficient storage and distribution of these images. Generally, this technology has been split into two areas: lossless compression at the cost of a lower compression ratio (typically 2:1) that preserves a complete, bit-for-bit identical copy of the original, and lossy image compression at a high compression ratio (typically of the order of 10:1) that produces an image that is visually similar but not identical to the original. However, lossy compression can generate "artifacts", i.e. features of the image that are not present in the original (e.g. blocking artifacts, posterization, aliasing, blurring...), and introduce bias. Modern algorithms, such as JPEG2000 or H265, try to make these artifacts almost invisible to the human eye and can be expressed as "visually lossless". However, such algorithms are mainly designed for the distribution of data to end users and post-processed images compressed using such algorithms can expose artifacts that were not visible to the human eye before, e.g. observed when "lifting the shadows" on a JPEG image. Moreover, re-compressing an image that has already been compressed by such an algorithm can cause an interaction between the two algorithms and produce visible artifacts. Figure One

[0003] In professional and technical applications, where images are considered as valuable data, users have always been cautious about using lossy compression, as this can lead to unforeseeable consequences or introduce bias in the data, leading to metrologically inaccurate. With the increasing use of machine learning methods, it becomes less important to ensure that the compression loss is not visible to the human visual system and the image quality becomes more critical. In particular in critical applications, such as medical diagnosis, self-driving cars or infrastructure monitoring, it is crucial to preserve the image quality in an objective way.

[0004] ​To meet this need, the Applicant, in previous work, developed a compression technique in which the information loss can be limited according to the information content, which is an objective measure independent of the human visual system, as disclosed for example in European patent application EP 3 185 555. The technique proposed therein achieves compression ratios of 5:1 to 10:1, while providing some stringent guarantees of artifact-free from a metrology point of view. Although this compression technique has proven satisfactory in many cases, some inconveniences have been found during long-term use. In particular, the results produced by certain specific analysis techniques are statistically significantly different from the results obtained by analyzing uncompressed data. Moreover, with this compression technique, it is not possible to anticipate in advance which analyses the compressed data are suitable for, and which analyses they are not suitable for. For example, the compressed data can be suitable for all the current analysis techniques, but can become insufficient in the future once the techniques become more advanced. For the user, this means that, when using the existing compression technique, there is no clear, simple guidance of the disadvantages brought by the compression and of its applicability to a specific data set. SUMMARY

[0005] OBJECT OF THE INVENTION

[0006] It is an object of the present invention to overcome the above difficulties and to provide lossless or at least partially lossless image compression, allowing to provide the user with a strong guarantee that the processed, and optionally compressed and decompressed, image data gives reliable results, i.e. is statistically equivalent to the image data obtained using the original, uncompressed sensor data, when processed or analyzed by any algorithm (current or future), while allowing relatively high compression ratios, typically up to 10:1.

[0007] It is a further object of the present invention to ensure that the processed, and possibly compressed and decompressed, image can only be difficultly distinguished from the unprocessed original image produced by a specific, real or ideal sensor model, without meeting certain conditions.

[0008] It is a further object of the present invention to allow embedding several types of useful information into the processed image, while guaranteeing that the processed image can still be used in any processing application that can use the original image, and gives results that are statistically difficult to distinguish or close to indistinguishable.

[0009] It is a further object of the present invention to allow the users to benefit from this new compression technique by allowing them to use the technique without major modifications to their workflow or to their existing systems, for example by allowing them to continue using "container" files or options thereof, such as tiff, dng, png, dicom, jp2, etc.

[0010] The solution according to the present invention

[0011] To this end, the application proposes a method for steganographic processing and compression of image data with negligible loss of information, characterized by the features listed in claim 1 and which allows to achieve the above-mentioned objects. In particular, the method according to the application is distinguished by the fact that the step of preparing the acquired image data further comprises the steps of determining a target noise model and corresponding parameters adapted to reflect the noise produced by a real or ideal target image sensor, generating pseudo-noise according to said target noise model, adding the generated pseudo-noise to said reduced noise image data in a steganographic manner, for example to produce pseudo-noise image data statistically equivalent to image data acquired with said target image sensor, and by the fact that the storing and / or transmitting step comprises storing and / or transmitting said pseudo-noise image data, for example to allow to reproduce and / or subtract the pseudo-noise based on said pseudo-noise image data and the parameters of the target noise model.

[0012] According to different embodiments of the method according to the application, said target noise model and its parameters are either constant and generally available, for example on an internet site or on a specific document used in the given application of the method or by other means, and / or stored and / or transmitted together with the pseudo-noise image data.

[0013] In preferred embodiments of the method according to the application, the step of preparing the acquired image data further comprises providing at least one steganographic key and storing and / or transmitting the parameters of the target noise model used to generate the pseudo-noise to produce said pseudo-noise image data by using said steganographic key. Preferably, these parameters are embedded into said pseudo-noise image data. In even more preferred embodiments, said steganographic key comprises at least a seed for a pseudo-random number generator.

[0014] In other preferred embodiments of the method according to the application, the step of embedding the parameters of the noise model used to generate the pseudo-noise into said pseudo-noise image data comprises encrypting said parameters.

[0015] In some embodiments of the method according to the application, the pseudo-noise is generated by means of a pseudo-random number generator and associated seed value(s). For those embodiments, said seed value(s) can be used as one of the steganographic keys.

[0016] In preferred embodiments of the method according to the application, said noise model adapted to reflect the noise created by said source image sensor used to capture said image data and / or to simulate the noise of a desired target image sensor can be implemented by a Poisson-Gaussian model or by a simplified model.

[0017] In an embodiment of the method according to the application, the removal of noise from the acquired image data can be achieved by quantization or de-noising techniques.

[0018] In a more preferred embodiment of the method according to the application, the preparation step further comprises the step of embedding other data, such as metadata of the image, calibration data of the sensor and / or data useful for the security, insurance and validity of the compressed and / or enhanced image data, into said pseudo-noise image data. The pseudo-noise image data comprising the parameters of said noise model and optionally said other data can be stored and / or transmitted as is or embedded in a container file of any known file format, such as a tiff, jp2, dng or png format container file.

[0019] In another preferred embodiment of the method according to the application, the method further comprises the step of compressing said pseudo-noise image data, said pseudo-noise image data resulting from the preparation step and optionally comprising the parameters of said target noise model, either as is or as part of a container file.

[0020] The present application also proposes a method for decompressing image data without loss of information of image data compressed by using the above-mentioned method.

[0021] The present application also relates to devices equipped with computer program means stored in a computer readable medium adapted to implement these methods, in particular microprocessors, field programmable gate arrays, image sensors, microscopes, satellites, mobile phones, tablets and / or personal computers adapted to implement these methods.

[0022] Other features and advantages of the present application are described in the dependent claims and in the following disclosure of the application, with reference to the attached drawings. BRIEF DESCRIPTION OF DRAWINGS

[0023] The attached drawings exemplify the principles of the application and several embodiments thereof.

[0024] Figure 1 The underlying steganographic model of the present application is schematically illustrated.

[0025] Figure 2a , 2b and 2c schematically illustrate the main steps of the method according to the present application.

[0026] Figure 3 The noise substitution step of the preparation step of the method according to the present application is schematically illustrated.

[0027] Figure 4 The compression step of the method according to the present application is schematically illustrated. DETAILED DESCRIPTION

[0028] In the following, the present application will be described in detail with reference to the above figures.

[0029] Generally, the present application uses techniques derived from cryptography and steganography to ensure that compressed and decompressed images cannot be distinguished from original "raw" images originating from a specific (real or ideal) sensor model, especially only with difficulty, unless the user has a specific encryption key (also referred to herein as steganographic key). In this case, any post-processing algorithm that does not have access to the key must produce the same result as the raw sensor data, statistically speaking. In the following, the terms "image", "video" and "data" are used synonymously, except when explicitly indicated otherwise.

[0030] Figure 1A bottom layer steganographic model is shown: a user 101 of the method according to the invention, arbitrarily called Alice, is tasked to acquire an image of a given object 100. She can achieve this by selecting either an image sensor SA 105 (hereinafter also specifically referred to as source sensor SA) or an image sensor SB 115 (hereinafter also specifically referred to as target sensor SB), yielding an unprocessed image set A 120 or B 140, respectively. Another user 102, arbitrarily called Bob, will process the images in various ways. Among other things, he can perform some analysis 145 of the object, try to determine which sensor acquired the image by studying noise 150, or apply lossless compression 155. If Alice hands him the image set A 120, Bob will find the result A 165 when analyzing 145 the object, and the sensor noise determination 150 will yield the sensor SA as output 170. When Bob is handed the image set B 140, he will correspondingly receive the result B 190 from the object analysis 145, and will output the sensor SB as the sensor noise determination 150. When Bob applies lossless compression to the image set B 140, he will obtain a compression ratio of about 2:1 180. Note that the image sets A and B are assumed to show the same object 100 under the same conditions. However, since the respective noise in both image sets will affect the result of the object analysis 145, the results A 165 and B 190 can differ statistically significantly. However, Alice can choose to apply a steganographic processing 125 to the image set A 120 (optionally including compression and decompression) to obtain a processed image set D 130. In doing so, she can specify that the processed image set D 130 mimics the noise model of the sensor SB, which is the target sensor 110. Alice can then freely choose 135 which of the two image sets B 140 or D 130 she will send to Bob. Without a corresponding steganographic key 160, Bob will not be able to distinguish between the image sets B and D, in particular, he will not be able to detect that one of the image sets B and D, respectively, has been steganographically processed / compressed, especially which one of the image sets B or D, regardless of what processing he applies, if he is not previously informed. More precisely, when performing lossless compression 155, sensor noise determination 150, and object analysis 145 on either of the two image sets, he will obtain results 180, 185, 190 corresponding to the image set B 140 within a statistical fluctuation range. This provides a strong guarantee of quality and indistinguishability of (processed) compressed data with respect to (unprocessed) unprocessed data. Only in case Bob possesses the correct steganographic key 160 that was used for the steganographic processing of the image set A to produce the image set D 120, he will be able to distinguish the image set D 130 from the image set B 140.For example, when applying lossless compression 155, he can use the key 160 to perform efficient compression with a compression ratio 175 of about 10:1, whereas without the key 160 he would still only obtain a low compression ratio 180 of about 2:1.

[0031] It is noted here that the source sensor SA 105 that generates the image set A 120 is not necessarily the same as the target sensor SB 115 that is used as a model to take the image set B 140. This is because in practical applications it is advantageous to provide output data that has already been corrected for various imperfections of the physical sensor (e.g. pixel non-uniformity, fixed pattern noise, etc.), so that the sensor SB 140 can be an "ideal" sensor. The important fact is that the image set D 130 is consistent with coming from a (real or ideal) sensor with well-defined and specified properties. This allows to ensure that any post-processing that will be performed on the image set D 130 will give reliable results as if the image set D 130 came from the target sensor SB 115.

[0032] Another advantage of using an ideal model for the target sensor SB is that multiple real sensors SA-1, SA-2,..., SA-N (each with different properties) can be mapped to a unique ideal sensor SB. This is particularly important for machine learning approaches, where for example a neural network trained on images from sensor SA-1 will not necessarily give reliable inference results when used with images from another sensor SA-2. In many practical situations it is necessary to train a neural network using historical data acquired through sensor SA-1 and eventually replace sensor SA-1 with a more recent model SA-2 because sensor SA-1 can stop being used, can become less competitive, or because the end user prefers not to "lock-in" to a single manufacturer. Mapping data from different sensors SA-1...SA-N so that it is consistent with data from the ideal sensor SB will therefore be helpful in these situations.

[0033] In one embodiment of the invention, the target sensor SB is characterized by a set of properties, including its quantum efficiency, read noise, black level and gain. In another embodiment, the target sensor SB is assumed to form an ideal device that outputs the exact number of photons absorbed by each pixel. It is also noted that, as with all physical systems and their inherent imperfections, due to practical limitations on collected statistical data, and finite imperfections or instabilities in the physical source sensor SA, the mapping of data from the source sensor SA to the ideal model target sensor SB can be performed in an approximate manner. This does not depart from the invention, and in practice these imperfections are acceptable to the user and can be precisely specified and limited.

[0034] In general, data from the target sensor SB 115 will not be suitable for lossless compression 155 at a ratio 180 generally greater than about 2: 1. This means that, without the steganographic key 160, Bob will not be able to losslessly compress at a higher ratio. On the other hand, with the key 160 in hand, Bob can perform lossless compression 155 of the data at a much higher ratio 175, generally up to about 10: 1. This allows the present invention to simultaneously provide the user with data that he can use as if he would use the unprocessed data, but on which he can apply lossless compression at a factor generally up to 10: 1.

[0035] The present invention described here has many other advantages, which stem from its close integration with steganography. In particular, the set of processed and possibly compressed and subsequently decompressed images D, which is generally 10 times larger than the corresponding data compressed by the method according to the present invention, is advantageously used to embed useful information, while guaranteeing that the set of data D can still be used in any processing application that can use the unprocessed set of images B, and from which statistically indistinguishable results are derived. In a preferred embodiment, this allows embedding other information as well as pseudo-noise parameters in the final processed images, for example:

[0036] - image metadata, such as the time at which this image metadata was taken, the system used to take this image metadata, the camera system used to take this image metadata,

[0037] - data that assist in compression, such as the properties of the device that took the image, the image size and binary layout (e.g. strips, tiles, etc.),

[0038] - data that enhance the security, integrity and validity of the image, such as a hash of the unprocessed image data or similar that can prove the authenticity, ownership or copyright information of the image data, a digital signature from the device that originally acquired the data, watermark data that can be recovered even after processing, and error correction data that can be used to recover parts of the image and metadata if lost or corrupted.

[0039] Furthermore, to enable users to fully benefit from this novel compression technique, it is important that they can use the technique without major changes to their workflow or their existing systems, for example, this means continuing to use the container files of their choice, such as known file formats like tiff, dng, png, dicom, jp2, etc.

[0040] Figures 2a to 2c The main steps of a preferred embodiment of the method according to the present invention, which implements the above features, are schematically illustrated. In Figure 2aThe preparation step 235 shown prepares the raw image data 210 captured by the source sensor SA 205 so that during the image acquisition unpredictable noise is generated in the source sensor SA 205, which, despite its unpredictable nature, is replaced 240 by pseudo-random predictable noise that complies with a target noise model 220 from a real or ideal target sensor SB, which can but does not necessarily differ from the source sensor SA 205. Encryption data 222 can be used to generate the pseudo-random noise and other sub-steps of the preparation step 235. Optionally, this preparation step 235 also includes sub-steps for correcting 245 for sensor defects, embedding 250 metadata 225 or embedding other data 230 as described above, and / or performing watermarking 255 so that the data prepared in this way can be easily identified at a later stage, even if only a part of it is available. This preparation operation results in "prepared raw" data, also denoted as "praw" data 260 in the following description and in the attached figures. The praw data 260 is suitable for any application that uses raw data, such as described above. This praw data 260 can be used on its own, as it contains the necessary metadata that make it more practical than standard raw data. Alternatively and optionally, the praw data 260 can be embedded in a container file 270 in any known file format (e.g. tiff, jp2, dng, png, etc.) as is usually done for raw data. The container file with praw data is compatible with any application that is already able to use this file type, but has the advantage of being highly compressible. In the container file 270, non-image data 275 can also be stored, which can contain any metadata 225 or other data 230. This can be used to store a copy of the data that has been directly embedded in the praw data 260.

[0041] In a preferred embodiment, after the preparation step and before the compression step shown in Figure 2b The compression step uses the steganographic / encryption data 222 to predict the pseudo noise and effectively and losslessly compress 155 the praw data 260. The steganographic / encryption data 222 can be coded into the compressor itself, in the praw file and / or given by the user. The steganographic / encryption data 222 can also be a predetermined constant or communicated in any other way. If such data is embedded during the preparation step, the compression step is also able to extract the metadata 225 and other data 230 and optionally use this data to verify the quality, authenticity and integrity of the file and / or apply error correction if needed. The compression step then creates a compressed file 280 that contains data that allows to restore the original container file 270 including the praw data 260 in a lossless manner.

[0042] Finally, after the compression step and before the decompression step shown inFigure 2c In the decompression step shown in the middle, the compressed file 280 is decompressed 295, e.g. to again produce the praw data 260, in particular the container file 270 containing the praw data 260 and optionally some other non-image data 275, as described above for the preparation step.

[0043] The container file 270 and the compressed file 280 can have a complex structure containing many frames or images. The structure of these files and the location of the binary praw data 260 can be unknown to the compressor used in the compression step. In this case, the watermark 255 embedded in the praw data 260 allows the compressor to identify and locate this data, extract any embedded data, and use this data to effectively perform the compression. The compressor and decompressor can be explicitly implemented and operated by a user, or can operate in the background, so that the files are transparently compressed and decompressed, e.g. when transferred over a network, when stored on a disk, when transmitted over a bus. Verification, integrity check, authenticity, error correction and / or data / metadata extraction features can operate independently of the compression and / or decompression.

[0044] To describe the method according to the invention in more detail, it is in addition noted that digital images captured by image sensors typically contain a large amount of entropy, which is largely due to sensor noise and also due to the inherent noise of the light impinging on the sensor. The presence of this noise is an important aspect of making an image have its natural statistical properties. As described above, removing this noise can lead to processing algorithms producing different results for images that still contain the noise. However, the presence of noise strongly limits the data size reduction that can be achieved using lossless compression. The present invention allows for processing digital images in such a way that a high lossless compression ratio can be achieved, while at the same time maintaining a realistic noise profile that neither visually nor statistically, e.g. via a histogram of the pixel values of the image, reveals that the image has been processed. The fact that the image is highly compressible is hidden in the image processed by the method according to the invention in a steganographic way that is not detectable and can only be revealed with the help of a steganographic / cryptographic key or function.

[0045] Thus, the method according to the present invention typically comprises the following steps for preparing a given digital image and a corresponding noise model of a digital camera or image sensor with specific settings.(1) In a suitable mathematical sense, removing most of the noise in the image captured by the source image sensor, e.g. by quantization or denoising. This can include correction of sensor-specific noise, e.g. fixed pattern noise or light response non-uniformity. The result of this step is denoted as denoised image data.(2) Generating pseudo-noise by suitably applying a pseudo-random number generator with well-defined initial conditions (e.g. integer seed) and then adding the generated pseudo-noise to the denoised image data in such a way that the resulting pseudo-noise image closely mimics a desired target noise model. Said desired target noise model can be adapted to mimic the source sensor that acquired the original image data, or it can be adapted to a different real or ideal target sensor. In the ideal functioning of the method according to the present invention, it is not possible to say that the (most of the) noise of the final processed image was generated using pseudo-random numbers.(3) The pseudo-noise image data, optionally including the noise model with the corresponding parameters (including the seed), and, if present and desired, the metadata and other data are stored to disk and / or transmitted over a bus or network. In a preferred embodiment, and depending on the requirements of the case, this can happen in different ways, which can be partly combined with each other: a) different data segments are stored / transmitted as separate files, separate parts of a container file or separate data packets; b) metadata and other data are embedded into the pseudo-noise image as described in further detail below, while the pseudo-noise parameters are stored / transmitted separately; c) the pseudo-noise parameters are embedded into a subset of the pixels of the pseudo-noise image using some steganographic method, and option b) is used for embedding metadata and / or other data. Furthermore, the storage and / or transmission of the pseudo-noise image data can optionally include one or more of the following steps:

[0046] - any or all data stored / transmitted with the pseudo-noise image, whether embedded into said pseudo-noise image or not, can be encrypted;

[0047] - any or all data stored / transmitted with the pseudo-noise image, whether embedded into said pseudo-noise image or not, can be encoded to assist error correction;

[0048] - any or all data stored / transmitted with the pseudo-noise image, whether embedded into said pseudo-noise image or not, as well as the pseudo-noise image itself can be complemented with a watermark or digital signature to assist authentication or identification of the data;

[0049] - said noise model and the corresponding parameters (including the seed) can be constant and fixed, e.g. in the case when hard-coded into hardware or software adapted to implement the method of the present invention, or disclosed on a website, in a manual or similar.

[0050] The simplest embodiment of the method according to the present application replaces most of the unpredictable noise in an image with predictable pseudo-noise. After acquiring image data captured by a source image sensor, this comprises the steps of determining a source and a target noise model, generating pseudo-noise, removing most of the noise from said image data, and adding the pseudo-noise to the noise-reduced image data, e.g. to produce pseudo-noise image data which is statistically equivalent to image data acquired with said target image sensor, and storing and / or transmitting said pseudo-noise image data. Because the pseudo-noise image data is statistically equivalent to image data acquired using said target image sensor, we refer to the above process as steganography to highlight the fact that an image prepared in this way is not easily distinguishable from a natural image produced from a sensor to which the target noise model applies. In this simplest embodiment, the natural noise can be replaced with the generated pseudo-noise in a steganographic manner without using a steganographic key, and neither the parameters of said target noise model nor any metadata or other data of the image need to be stored and / or transmitted, e.g. by being stored and / or transmitted together with said pseudo-noise image data, in particular by being embedded into said pseudo-noise image data. In fact, in this case, the target noise model and its parameters can be chosen to be constant and thus can become available / accessible through other means such as the ones mentioned above. As will become clear below, more complex and detailed embodiments of the method according to the present application are preferred and allow, depending on the application of the method, to add the parameters of said target noise model and / or metadata and / or other data of the image to said pseudo-noise image data, in particular by embedding this additional information into said pseudo-noise image data, optionally by using steganographic key(s), in different ways and at the discretion of the individual. Thus, in any of these cases, it is guaranteed that the target noise model and the corresponding parameters (used to generate the pseudo-noise which is added to the noise-reduced image data to produce the pseudo-noise image data) are available, i.e. presented as somehow accessible, during the compression / decompression steps of the method according to the present application, in particular during any other desired steps assumed to be applied to the pseudo-noise image data, in order to allow the reproduction / subtraction of the pseudo-noise based on said pseudo-noise image data and the parameters of the target noise model.

[0051] In this context, the steganographic key(s) can be secret or public, depending on the application, and the steganographic key(s), in particular, serve the purpose of adding the generated pseudo-noise to the noise-reduced image data in a steganographic manner, typically when generating the pseudo-noise image data.

[0052] - the fact that the file has been processed is in fact undetectable by using the stego key(s), i.e. the original image data has been replaced by pseudo-noise image data according to a given target noise model.

[0053] - the fact that the pseudo-noise image can be effectively and losslessly compressed as a result of the processing is also in fact undetectable. More precisely, lossless compression would be as inefficient as with the unprocessed image data, and effective compression is only "unlocked" and available by using the stego key(s).

[0054] - the fact that the pseudo-noise image data beyond the actual image data can also contain additional data is also in fact undetectable. The additional data can only be accessed with the help of the stego key(s).

[0055] At a later stage, the pseudo-noise image can be effectively compressed without loss by using the pseudo-noise parameters to generate an identical copy of the pseudo-noise, and then subtracting the identical copy of the pseudo-noise from the image, so that the noise-reduced image is retrieved that is exactly the same as in step (1) above. Depending on the possibility to use step (2) above, it can be necessary to extract the pseudo-noise parameters from the image itself using the stego key, or to read the parameters from a separate file or data packet, or to make them otherwise available in the case where the noise model and the corresponding parameters, including the seed, are constant and fixed. In addition, metadata and other data can be extracted from the image if previously embedded in the image. Next, the denoised image can be effectively compressed (typically by a factor of 5 to 10) using existing or custom lossless image codecs, e.g. as defined in the JPEG2000 standard, due to the previous denoising. The parameters of the pseudo-noise must be preserved and can be stored, for example, together with the compressed image data.

[0056] The corresponding decompression can be performed by first decompressing the image data using the same lossless codec, and then re-applying steps (2) and (3) above with the same parameters of the pseudo-noise. In the case of ideal functioning of the method according to the present application, the result will be identical in bits to the final processed image obtained after step (3) above.

[0057] In view of the principles set out above, different embodiments of the method according to the present application can be implemented in general by implementing these steps at different stages of the image pipeline and / or by dividing these steps between different parts of hardware and software. For example, one implementation can be integrated directly into an image sensor, then steps (1) to (3) are performed before the image data is transferred to a storage area of the sensor or to an output bus, while the subsequent lossless compression and decompression functions are provided by a software module for a computer, tablet, mobile phone or similar.

[0058] Thus, the method according to the present invention allows, thanks to said preparation step, to retrieve from the noisy version of the same image a denoised version of the image that is suitable for efficient lossless compression. Preferentially, this is achieved by replacing the natural noise, whose origin is usually a combination of the quantum noise of the light captured by the image sensor and the electronic noise of the image sensor itself, by artificial pseudo-noise, generated with a pseudo-random number generator. While the natural noise is unpredictable, the pseudo-noise is deterministic and can be reproduced identically if the exact parameters and algorithm generating the noise are known. However, a fundamental part of the invention consists in that the pseudo-noise is made to mimic its characteristics by using a noise model, so as to be as similar as possible to the natural noise of a given desired real or ideal target image sensor, as will be described in more details hereafter. Accordingly, the invention also provides a method for lossless compression and decompression of images with pseudo-noise, said compression and decompression steps being different compared to the corresponding steps in the prior art, as will also be described in more details hereafter.

[0059] Further details will be given hereafter regarding some sub-steps of the image preparation 235, compression 155 and decompression 295.

[0060] Image preparation

[0061] The first step is said image preparation step 235, in which the raw unprocessed image data 210 is processed in such a way that it is replaced by a pseudo-noise version of the same image. Preferentially, the pseudo-noise parameters are stored / sent with the pseudo-noise image, or are directly embedded into the image using steganographic methods (that is to say ideally undetectable). Without the corresponding steganographic key, it is not possible to say that the image contains pseudo-noise instead of natural noise, and that the image can be used for the same purposes and applications as the raw image data 210. As mentioned above and schematically shown in Figure 2a The metadata 225 and other related data 230 can also optionally be embedded into the pseudo-noise image, resulting in a praw data 260 that is self-contained and can represent additional value compared to the raw image data, as schematically shown in

[0062] Figure 2a The image preparation step 235 is schematically shown. The description in this and the following sections will focus on so-called unprocessed image data, that is, mostly unprocessed image data directly from a CMOS or CCD image sensor or equivalent. The skilled person will understand that the invention can be equivalently applied to other kinds of image data, by adapting some of the steps involved, in particular by using an appropriate noise model.

[0063] The first step within the image preparation step 235 is to acquire the raw image data 210 and perform noise substitution 240, in which most of the random noise in the image characterized by the input noise model 215 is replaced by pseudo noise characterized by the target noise model 220. The noise substitution process is illustrated in Figure 3 Figure 3. It first applies a denoising or quantization step 305 by means of an appropriate input noise model 215 and associated parameters to generate a denoised image 325. Another component can be a pseudo random number generator 320, the use of which will become clear in the following sections. The Poisson-Gaussian model generally describes the noise in raw image data captured by CCD or CMOS sensors very well, for which the estimated standard deviation σ i of a pixel i with value x i is given by

[0064]

[0065] where the noise parameters of the model are a, which is related to the signal amplification inside the image sensor, the black level x0of the sensor, and b, which is a parameter related to the readout noise of the sensor. Although this noise model is preferred for CCD and CMOS raw image data, the following description will use a simplified model by assuming that the noise has a standard deviation σ0that is independent of the pixel value. For certain types of image sensors, this is a perfectly valid assumption. In this case, the number of noise bits per pixel for image data with integer values can be calculated as

[0066]

[0067] Depending on the acquisition settings, this number can be as high as 6 to 8 bits. For image data encoded as 16 bits, this severely limits the compression ratio that can be achieved using lossless compression. This situation can be improved by applying one of the many denoising techniques known in the art. Typically, these techniques are themselves computationally intensive and require extra care in certain subsequent steps of the image preparation 235 and decompression 295. Therefore, instead of applying such known denoising techniques, the image preparation step 235 of the method according to the present invention preferably comprises a step of quantization by scaling the pixel values by a fraction q of the user-specified noise standard deviation σ0to obtain new pixel values. Many known quantization defects can be avoided by using subtractive dithering. For this purpose, the above-mentioned pseudo random number generator 320 with a seed S 315, where S is typically an integer, can be used to generate a pseudo random number R i from a uniform distribution over the interval from 0 to 1 for each pixel i i and generate a denoised image 325

[0068]

[0069] The denoised pixel values y i The quantized image consisting of denoised pixel values y i has fewer distinct values than the original image data consisting of original pixel values x i and can thus be losslessly compressed more efficiently. If necessary, the denoised pixel values y i can be scaled back to the original range by multiplication with σ0 / q. As mentioned, a similar result can be achieved by applying known denoising techniques, but at the cost of computational effort and higher care only during some subsequent steps of the method.

[0070] In the process of generating the denoised image 325 by quantization or by denoising technique 305, most of the natural noise is removed and a little bit of artificial, deterministic noise can be added by introducing R i .

[0071] In a second step within the image preparation step 235, the image is re-noised 330 according to the target noise model 220. For example, the original noise level can be substantially restored by a transformation

[0072]

[0073] For computing the restored pixel values z i , the input noise model 215 and its parameters are used as described above, where the former specifies the functional form of the transformation to be used and the latter specifies the precise values of the parameters σ0and q. It is to be noted that instead of using the original input noise model, a different target noise model 220 can be mimicked by using a corresponding function and parameters for computing the restored pixel values z i . If used as described above, the pseudo-random number generator 320 is reused for creating pseudo noise of essentially the form -R i · σ0 / q, which is then added to the image data to form the pseudo-noise image 335 consisting of the restored pixel values z i . For both computing the denoised pixel values y i and the restored pixel values z i , the value R i is essentially the same.

[0074] The third step within the image preparation step 235 of the method according to the present application is the storage of the pseudo-noise image data and optionally the pseudo-noise parameters and other data, including metadata, respectively, especially in packets for transmission. If performed correctly, this step allows a) to use the pseudo-noise image data as soon as unprocessed image data of the respective type and format is used. Additionally, the steganographic key allows b) to retrieve the denoised image data 325 from the pseudo-noise image 335 at a later stage, so that this data can be stored and transmitted efficiently by means of lossless compression, if desired. And c) to retrieve the metadata 225 and / or other data 230 that can be used to enable, assist or improve image processing or analysis.

[0075] In some embodiments of the method according to the present application, the pseudo-noise image, the noise parameters and other data are stored together in a suitable data structure, directory or other container, for example as provided by the Tagged Image File Format (TIFF / EP, ISO 12234-2). The seed value 315 of the pseudo-random number generator 320 can then be used as a steganographic key, since it can be combined with the noise parameters to retrieve the denoised image 325 as described below. Depending on the specific requirements of the given application of such an embodiment, the seed can be secret or publicly available.

[0076] However, it is preferred to make the pseudo-noise image 335 as self-contained as possible, that is, to embed as much data as possible in the pseudo-noise image data itself. In Figure 3 In one embodiment of the method according to the present application, schematically shown in Fig. 3, the embedding of the image metadata 225 and other data 230 during the re-noising step 330 can include encryption and / or error correction means by first encoding the data into a suitable bit string 340. This embodiment then preferably makes use of two different seed values 315 S1 and S2, of which S2 can be derived from S1. By using S1 to generate random numbers when the bit is 0 and S2 to generate random numbers in other cases, the bit string 340 can be embedded into a sequence of pixels, each pixel having one bit of the string. Occasionally, S1 and S2 can lead to the same pseudo-noise pixel value z i resulting in so-called collisions. When the bit string is extracted from the pseudo-noise image 335 at a later stage, the collisions can be handled by suitable error correction methods known in the art. Assuming that the pseudo-noise cannot be detected when using a single seed value 315 for the pseudo-random number generator 320, it should be equally impossible to detect when using two different seed values, so the data is safely hidden in the pseudo-noise image 335.

[0077] In another embodiment of the method according to the present application, the data embedding is extended to include the pseudo-noise parameters themselves. For example, by using an encryption step with the steganographic key K to generate a ciphered version C of the parameters P = {σ0, q, S, T}P Here, S denotes one or more seed values of the random number generator 320, and T is a concatenation of certain pixels at fixed, predetermined locations in the pseudo-noise image 335. T is optional and can be used as an encryption salt, which guarantees C P From image to image, even if σ0, q, and S are the same. Note that P does not necessarily contain information about the type of noise model used in these steps, but can easily be extended to contain this information. The next step is the actual embedding process. In a simple embodiment, the cryptographic conversion of C P is converted into a bit string of length k, and the least significant bits of the first k pixels not used in the generation of T are overwritten with the corresponding values in this string. Thus, the values of these pixels change from z i to z i '. This way of embedding C P has limited security, and more complex embodiments can use other data embedding methods. As mentioned above, once C P has been embedded, the metadata 225 and other data 230 can be embedded into the rest of the image using two different seed values.

[0078] More complex embodiments can also add redundancy and means for error correction for all data embedding, so that the embedded data can be retrieved when the image data undergoes cropping or other modifications. Independently of the specific method of data embedding, it must be noted that the recovered pixel values z i ' satisfy the relation y i (z′ i ) = y i (x i ), that is, the processed image with pseudo-noise parameters, which is the result of the embedding step within the image preparation step 235 according to the method of the invention, can be used as well as the original image data 210 to obtain denoised image data 325, assuming all other parameters are known and unmodified.

[0079] Compression

[0080] The second step of the method according to the present application is a compression step 155, which is generally optional and can be performed at a later stage. It basically consists in retrieving the denoised image 325 from the pseudo-noise image 335, by means of the pseudo-noise parameters and seed value 315 for the pseudo-random number generator 320. Then, the denoised image 325 is compressed using a lossless compression algorithm. Preferably, the pseudo-noise parameters and possibly additional data are combined with the compressed image data 285 to form a compressed image with pseudo-noise parameters, for example by storing all contents in a single container file 280 or directory, to facilitate the subsequent decompression step 295. Optionally, the compressed image with pseudo-noise parameters can be created so that it can be opened or directly viewed by the imaging application, either by using a container file (especially of a certain format) which allows direct data access to the denoised image data, or by adding a certain preview image to the container.

[0081] Reference will now be made to Figure 4 The optional compression step of the image data resulting from the above image preparation process 235 will now be described in detail. For simplicity, this description will use the same simple noise model as in the previous section, i.e. a constant, signal-independent noise represented by a standard deviation σ0. However, this description is similarly applicable to any other target noise model 220. The compression step takes as a starting point the container file 270 or equivalent data structure previously generated by the image preparation step 235. The first step is the retrieval 405 of the pseudo-noise parameters and seed value 315. For embodiments where these parameters are embedded in the pseudo-noise image, the extraction proceeds as follows. First, a bit string of length k is generated by concatenating the least significant bits of the first k pixels generated by T. This bit string is equal to the password of the parameter C P . C P is then decrypted with the key K (which can be represented by the seed value 315 as described above) to retrieve the parameters P = {σ0, q, S, T} which provide the required information for the target noise model 220 and the pseudo-random number generator 320. In a second step, these parameters are then used to retrieve the metadata 225 and other data 230 by bit string decoding 410 and to regenerate the pseudo-noise according to the target noise model 220, which is basically given by -R i · σ0 / q. Then, the noise is subtracted 415 from the pseudo-noise image 335 in the praw data 260 of the container file 270 to retrieve the denoised image 325. For the simple noise model used here, these steps combine to compute the denoised image 325 by computing

[0082]

[0083] It can be noted here that this is identical to the formula used during the image preparation step 235 for computing the denoised image 325, except that here the pixel values z iprocessed image data, not the raw pixel values x i raw unprocessed image data 210. In a third step, lossless compression 420 is applied to the denoised image data 325. The compression algorithm can be chosen from a variety of existing or custom image data algorithms, such as lossless JPEG or PNG or general compression (e.g. ZIP), depending on the requirements of the context in which the method according to the invention is employed. Finally, the pseudo-noise parameters, metadata 225 and other data 230 are optionally encrypted, and packaged 425 together with the compressed image data into a compressed file 280 or equivalent data structure. Depending on the application, this concatenation can be performed by simply appending the binary data to the same file, by using several files in a container like an archive or by using a tool of a special file format (e.g. TIFF).

[0084] decompression

[0085] In view of the above description of the example embodiments for the preparation 235 and compression 155 steps, the corresponding decompression steps of the method according to the invention can be briefly described and only schematically illustrated in Figure 2c . Starting from the compressed file 280 or equivalent data structure, the first step of such a decompression step is the separation of the parameters P. If the parameters are stored in encrypted form with the password of the parameters C P , the separation optionally includes decryption. Then, the denoised image data 325 is recovered using lossless decompression, corresponding to the exact same pixel values y i as obtained during the image preparation step 235. Correspondingly, the decompression proceeds by reproducing the same steps as during the preparation step, i.e. adding pseudo-noise to the denoised image data 325 in the re-noising step 330, and storing or embedding the pseudo-noise parameters, metadata and / or other data, that is, generating the praw data 260 and putting everything into the container file 270. The output of the decompression process is a bit-for-bit copy of the praw data 260 and / or the container file 270 produced by the image preparation step 235.

[0086] In view of the above description of several embodiments of the method according to the present application, it is clear to the person skilled in the art that the present application has several important advantages. First, the present application allows for processing digital image data in such a way that a high lossless compression ratio, typically up to about 10:1, can be achieved while maintaining a true noise profile that reveals neither visually nor statistically that the image has been processed. Second, the present application ensures that the image data compressed and decompressed from the original "unprocessed" image data cannot be distinguished from actual unprocessed image data of a real or ideal sensor using its noise model without knowing the steganographic method or the key used to process the image data. Third, the present application advantageously also allows for embedding several types of useful information into the processed image data while still guaranteeing that the processed image data can be used in any application where unprocessed image data can be used and gives results that are statistically indistinguishable therefrom. Finally, the present application fully benefits the user by allowing the user to use the technology without having to make significant changes to their workflow or their existing systems, for example by allowing the user to continue using their container files of choice (e.g. files in known file formats such as tiff, dng, png, dicom, jp2, etc.)

Claims

1. A method for steganographic processing and compression of image data with negligible loss of information, wherein said image data comprises noise and information, said method comprising the steps of: - acquiring image data (210) to be processed and / or compressed for storage and / or transmission, - preparing (235) the acquired image data (210) for compression, said preparing step (235) comprising the steps of: ° determining an input noise model (215) and corresponding parameters adapted to reflect noise produced by a source image sensor (SA) used to capture said image data, ° canceling noise (305) from the acquired image data (210) with negligible loss of information by using said input noise model (215) to produce denoised image data (325), - storing and / or transmitting said image data, characterized in that the step of preparing (235) the acquired image data (210) comprises the steps of: ° determining a target noise model (220) and corresponding parameters adapted to reflect noise produced by a real or ideal target image sensor (SB), ° generating (320) pseudo noise according to said target noise model (220), and ° adding (330) the generated pseudo noise to said denoised image data (325) in a steganographic way so as to produce pseudo noise image data (335) which is statistically equivalent to image data acquired with said target image sensor (SB), and in that the step of storing and / or transmitting comprises: ° storing and / or transmitting said pseudo noise image data (335) to allow reproduction and / or subtraction of said pseudo noise based on said pseudo noise image data (335) and parameters of said target noise model (220).

2. The method according to the preceding claim, characterized in that, Said target noise model (220) and its parameters are either constant and available, and / or are stored and / or transmitted with said pseudo noise image data (335).

3. The method according to one of the preceding claims, characterized in that, The step of preparing (235) the acquired image data comprises the steps of: - providing a steganographic key (K), - storing and / or transmitting, by using said steganographic key (K), parameters of said target noise model (220) used to generate said pseudo noise to produce said pseudo noise image data (335).

4. The method according to claim 3, further comprising embedding (250) into said pseudo noise image data (335) parameters of said target noise model (220) used to generate said pseudo noise.

5. The method according to claim 3, wherein said steganographic key (K) comprises at least a seed (315) for a pseudo random number generator (320).

6. The method of claim 4, wherein, The step of embedding (250) into said pseudo noise image data (335) parameters of said target noise model (220) used to generate said pseudo noise comprises encrypting said parameters based on said steganographic key (K).

7. The method of claim 1, wherein, Said preparing step further comprises the step of: - correcting the acquired image data for sensor defects, - embedding metadata (225) of the image, data enhancing security, integrity and validity of the image data, and calibration data of the sensor and / or data useful for compression into the pseudo-noise image data (335), and wherein the step of embedding comprises a step of encoding the data being embedded and switching between two or more seed values (315) for a random number generator (320) used when adding (330) pseudo-noise to the denoised image data (325); and / or - embedding error correction data into the pseudo-noise image data.

8. The method of claim 7, wherein data useful for compression comprises properties of the sensor used to capture the acquired image data, image size and / or binary layout.

9. The method of claim 7, wherein data enhancing security, integrity and validity of the image data comprises a hash of the acquired raw image data, ownership and / or copyright information, and / or watermark data.

10. The method of claim 1, wherein, The pseudo-noise image data (335) is stored and / or transmitted as is or is embedded into a file or container file (270) of any format, so that the pseudo-noise image data (335) is compatible with any application adapted to be used with the format of the file or container file (270).

11. The method of claim 1, wherein, The input noise model (215) adapted to reflect the noise generated by the source image sensor (SA) and / or the target noise model (220) adapted to reflect the noise generated by the target image sensor (SB) are implemented by a Poisson-Gaussian model with a standard deviation of the Poisson-Gaussian model for the value of a pixel i is given by where the noise parameters are an internal signal amplification parameter of the sensor a black level parameter of the sensor and a read-out noise parameter of the sensor Alternatively, the input noise model (215) adapted to reflect the noise generated by the source image sensor (SA) and / or the target noise model (220) adapted to reflect the noise generated by the target image sensor (SB) are implemented by a simplified model with a standard deviation independent of the pixel value of the sensor.

12. The method of claim 1, wherein, The method further comprises the steps of: - compressing (155) the pseudo-noise image data (335) resulting from the preparation step (235), as is or embedded into a container file (270).

13. The method of claim 12, wherein, The step of compressing (155) comprises the steps of: - acquiring the pseudo-noise image data (335) resulting from the preparation step (235), - in case the parameters of the target noise model (220) are stored with the denoised image data (325) and are based on a steganographic key (K), extracting (405) the parameters of the noise model (220) for generating the pseudo-noise, - generating pseudo-noise according to the extracted parameters of the target noise model (220), - subtracting (415) the generated pseudo-noise from the pseudo-noise image data (335) resulting from the preparation step (235) to produce denoised image data (325), - losslessly compressing (420) the denoised image data (325) to produce compressed denoised image data (325), - storing and / or transmitting the compressed denoised image data.

14. The method of claim 13, further comprising: - packing (425) the parameters of the target noise model (220) with the compressed denoised image data; - storing and / or transmitting the compressed denoised image data and the parameters of the target noise model (220) used to generate the pseudo-noise.

15. The method of claim 14, further comprising: - further packing (425) the compressed denoised image data with metadata (225) and one or more types of data from: calibration data of the sensor, data useful for compression and data enhancing security, integrity and validity of the image data; - storing and / or transmitting the compressed denoised image data, the parameters of the target noise model (220) used to generate the pseudo-noise and metadata (225) and the one or more types of data.

16. A method for decompressing image data without loss of information, characterized by, The method comprises the steps of: - obtaining the parameters of a target noise model (220) used to generate a pseudo-noise and obtaining compressed image data resulting from a compression step (155) by the method according to claim 12, and - decompressing (295) the compressed image data resulting from the compression step (155).

17. The method of claim 16, wherein, The method comprises the steps of: - separating the compressed denoised image data and the parameters of a target noise model (220) used to generate the pseudo-noise in case the parameters of the target noise model (220) are embedded in the denoised image data (325), - losslessly decompressing the compressed denoised image data to produce decompressed denoised image data (325), - extracting the parameters of the target noise model (220) used to generate the pseudo-noise in case the parameters of the target noise model (220) are stored with the denoised image data (325) and are based on a steganographic key (K), - generating a pseudo-noise according to the extracted parameters of the noise model (220), - adding (330) the generated pseudo-noise to the decompressed denoised image data (325) to produce pseudo-noise image data (335), - storing and / or transmitting the pseudo-noise image data (335).

18. The method according to claim 17, further comprising: - embedding (250) the parameters of the noise model (220) into the pseudo-noise image data (335); - storing and / or transmitting the embedded pseudo-noise image data comprising the parameters of the noise model (260).

19. The method according to claim 18, further comprising: - further embedding (250) metadata (225) and / or one or more types of data from: calibration data of the sensor, data useful for compression and data enhancing security, integrity and validity of the image data into the pseudo-noise image data (335); - storing and / or transmitting the embedded pseudo-noise image data comprising the parameters of the noise model (260) and metadata (225) and / or the one or more types of data.

20. The method of claim 16, wherein, The steganographic key (K) used in the compression (155) and decompression steps (295), respectively in the compressor and in the decompressor, is encoded in the pseudo-noise image data (335), is available at predefined information sources including web sites and internal documents, or is publicly available, and / or is given by the user of the method.

21. The method of claim 16, wherein, The generation of pseudo-noise and / or the embedding (250) according to the parameters of the noise model (220) is implemented by using a pseudo-random number generator (320).

22. The method according to claim 21, wherein the embedding (250) comprises encryption, and the encryption is implemented by using a pseudo-random number generator (320).

23. The method of claim 16, wherein, The steps of the method for steganographic processing and compression of image data and / or of the method for decompression of image data are performed as divided between different hardware and / or software parts.

24. Computer program means stored in a computer readable medium, adapted to implement the method according to one of the preceding claims.

25. An apparatus equipped with the computer program device according to claim 24, characterized in that, The device is selected from the group comprising: microprocessor, field programmable gate array, image sensor, mobile phone, digital photo device, digital video camera, scanning device, tablet, personal computer, server, microscope, satellite.

26. The device according to claim 25, wherein the mobile phone comprises a smartphone equipped with a digital camera.

Images