Method and device for steganographic processing and compression of image data - Patents.com

Abstract

To provide a method and device for steganographic processing and compression of image data with negligible information loss, wherein the image data comprises noise and information.SOLUTION: In steganographic processing, a step of pre-processing for compression of image data 210 includes: removing, with negligible information loss, noise (305) from acquired image data (210) by use of an input noise model (215) such as to produce noise-reduced image data (325); generating (320) pseudo-noise according to a target noise model (220); and adding the generated pseudo-noise to the noise-reduced image data (325) such as to produce pseudo-noisy image data (335). A step of storing and / or transmitting comprises storing and / or transmitting the pseudo-noisy image data (335), such as to allow for reproduction and / or subtraction of the pseudo-noise.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a method and corresponding device for steganographic processing and compression of image data with negligible information loss, the image data including noise and information. The method includes the steps of acquiring image data and pre-processing the acquired image data, the pre-processing step including the steps of determining an input noise model and corresponding parameters adapted to reflect noise created by a source image sensor used to capture the image data, and removing noise from the acquired image data with negligible information loss to create noise-reduced image data, the method further including the steps for storing and / or transmitting the image data. [Background technology]

[0002] The number of digital images and videos produced each year continues to grow at an accelerating rate. For decades, compression has played an essential role in enabling the efficient storage and dissemination of these images. Usually, the technique is divided into two areas: lossless compression, which preserves a perfect, bit-for-bit identical copy of the original at the expense of a low compression ratio (usually 2:1), and lossy image compression, which produces an image visually similar to the original but not identical at a high compression ratio (usually around 10:1). Lossy compression may, however, produce "artefacts", i.e. image features that were not part of the original (e.g. block artefacts, posterization ringing, aliasing, blurring, ...) and introduce biases. Modern algorithms, such as JPEG2000 or H265, among others, attempt to make these artefacts almost invisible to the human eye and may be described as "visually lossless". However, such algorithms are inherently crafted to disseminate data to end users, and post-processing of images compressed with such algorithms can reveal artifacts previously invisible to the human eye, as observed, for example, when "lifting the shadows" in a JPEG image. Furthermore, recompressing an image already compressed with such an algorithm can result in interactions between the two algorithms and the creation of visible artifacts.

[0003] In professional and technical applications where images are considered valuable data, users have been cautious about using lossy compression because it introduces biases into the data that can have unforeseen consequences or make the data metrologically inaccurate. With the increasing use of machine learning techniques, it has become less relevant to ensure that compression losses are not visible by the human visual system and image quality has become more critical. Especially in critical applications such as medical diagnosis, autonomous vehicles or infrastructure monitoring, it is of utmost importance to objectively preserve image quality.

[0004] To satisfy this need, in previous work, the applicant developed a compression technique, as disclosed in EP 3185555, in which the information loss can be limited in terms of information content, an objective measure not related to the human visual system. The technique presented there achieves compression ratios ranging from 5:1 to 10:1, while at the same time providing some strict guarantees for the absence of artefacts from a metrological standpoint. Although the compression technique has been found to be satisfactory in many situations, some disadvantages have been identified during extended use. In particular, some specific analysis techniques have produced results that are statistically significantly different from those obtained by analyzing uncompressed data. Also, with this compression technique, it is not possible to infer in advance which analyses the compressed data is suitable for and which are not. For example, the compressed data is suitable for all current analysis techniques, but may become inadequate in the future as the techniques become more advanced. For users, this means that with existing compression techniques there are situations where there are no clear, easy-to-understand guidelines of the drawbacks introduced by compression and the suitability of applying compression to a particular data set. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] European Patent Application Publication No. 3185555 Summary of the Invention [Problem to be solved by the invention]

[0006] The object of the present invention is to overcome the above-mentioned problems and to provide a lossless, or at least partially lossless, image compression that allows the user to have strong assurance that the processed, optionally compressed and decompressed image data, when processed or analyzed by any current or future algorithm, will give reliable results, i.e., statistically equivalent to those obtained using raw uncompressed sensor data, while at the same time allowing relatively high compression rates, typically up to about 10:1.

[0007] It is also an object of the present invention to ensure that a processed, possibly compressed and decompressed image can only be distinguished with difficulty from an unprocessed raw image originating from a particular real or ideal sensor model if certain conditions are not met.

[0008] Another object of the present invention is to enable embedding of some type of useful information into the processed image, ensuring that the processed image can still be used in any processing application in which the raw image can be used, and providing results that are statistically indistinguishable or close to indistinguishable.

[0009] It is a further object of the present invention to allow users to fully benefit from this new compression technology by enabling them to use this technology without significant modifications to their workflow or to their existing systems, for example by allowing users to continue using their "container" files of choice, such as tiff, dng, png, dicom, jp2, etc. [Means for solving the problem]

[0010] To this effect, the present invention proposes a method for steganographic processing and compression of image data with negligible information loss, which method is characterized by the features recited in claim 1 and allows achieving the above identified objective. In particular, the method according to the invention differentiates itself by the fact that the step of pre-processing the acquired image data further comprises the steps of determining a target noise model and corresponding parameters adapted to reflect the noise created by a real or ideal target image sensor, generating a pseudo-noise according to said target noise model, and adding the generated pseudo-noise in a steganographic manner to said noise-reduced image data to create a statistically equivalent pseudo-noise image data compared to the image data acquired with said target image sensor, and by the fact that the step of storing and / or transmitting comprises storing and / or transmitting said pseudo-noise image data in such a way as to enable the reproduction and / or subtraction of the pseudo-noise based on said pseudo-noise image data and the parameters of the target noise model.

[0011] According to different embodiments of the method according to the invention, the target noise model and its parameters are constant and generally available, for example on an internet site or in a particular document used in a given application of the method or by other means, and / or stored and / or transmitted together with the pseudo-noise image data.

[0012] In a preferred embodiment of the method according to the invention, the step of pre-processing the acquired image data further comprises providing at least one steganographic key and storing and / or transmitting parameters of the target noise model used to generate pseudo-noise for creating the pseudo-noise image data by using said steganographic key. Preferably, these parameters are embedded in said pseudo-noise image data. In an even more preferred embodiment, said steganographic key consists of at least one seed for a pseudo-random number generator.

[0013] In another preferred embodiment of the method according to the invention, the step of embedding parameters of the noise model used to generate the pseudo-noise in the pseudo-noise image data comprises encrypting said parameters.

[0014] In some embodiments of the method according to the invention, the pseudo-noise is generated with the aid of a pseudo-random number generator and an associated seed value, for which said seed value may serve as one of the steganographic keys.

[0015] In a preferred embodiment of the method according to the invention, the noise model adapted to reflect the noise created by the source image sensor used to capture the image data and / or to emulate the noise of a desired target image sensor may be realised by a Poisson-Gaussian model or by a simplified model.

[0016] In embodiments of the method according to the present invention, removing noise from the acquired image data may be achieved by quantization or denoising techniques.

[0017] In an even more preferred embodiment of the method according to the invention, the pre-processing step also comprises embedding other data into said pseudo-noise image data, such as image metadata, sensor calibration data and / or data useful for compression and / or for enhancing security, safety and validation of image data. The pseudo-noise image data including the parameters of the noise model and optionally said other data may be stored and / or transmitted as is or may also be embedded into a container file of any known file format, for example a container file in tiff, jp2, dng or png format.

[0018] In a further preferred embodiment of the method according to the invention, the method further comprises the step of compressing said pseudo-noise image data created by the pre-processing step and optionally including parameters of said target noise model, either as is or as part of a container file.

[0019] The invention further proposes a method for decompression of image data without information loss of the image data compressed by using the above method.

[0020] The invention further relates to devices with computer program means stored on a computer readable medium adapted to implement these methods, in particular microprocessors, field programmable gate arrays, image sensors, microscopes, satellites, mobile phones, tablets and / or personal computers adapted to implement these methods.

[0021] Other features and advantages of the invention are mentioned in the dependent claims and in the description that follows, which discloses the invention in more detail with reference to the drawings.

[0022] The accompanying drawings illustrate, by way of example and diagrammatically, the principles and some embodiments of the present invention. [Brief description of the drawings]

[0023] [Figure 1] FIG. 1 shows a schematic diagram of the steganographic model on which the present invention is based. [Figure 2a] FIG. 2 shows a schematic diagram of the main steps of the method according to the invention; [Figure 2b] FIG. 2 shows a schematic diagram of the main steps of the method according to the invention; [Figure 2c] FIG. 2 shows a schematic diagram of the main steps of the method according to the invention; [Diagram 3] FIG. 2 illustrates diagrammatically the noise substitution step of the pre-processing step of the method according to the invention; [Figure 4] FIG. 2 shows a schematic representation of the compression step of the method according to the invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] In the following, the invention will be described in detail with reference to the above-mentioned drawings.

[0025] In general, the present invention uses techniques derived from cryptography and steganography to ensure that a compressed and decompressed image is indistinguishable, or can only be distinguished with difficulty, from the original "raw" image generated from a particular (real or ideal) sensor model, unless the user has a particular encryption key (also referred to herein as a steganographic key). In this case, any post-processing algorithm without access to the key should produce the same results, in a statistical sense, as it would have produced on the raw sensor data. In the following, the terms "image", "video" and "data" are used synonymously, unless otherwise specified.

[0026] Fig. 1 shows the underlying steganographic model. A user 101 of the method according to the invention, optionally called Alice, is tasked with acquiring an image of a given object 100. She acquires the image by choosing either an image sensor SA 105, also called source sensor SA hereafter, or an image sensor SB 115, also called target sensor SB hereafter, resulting in a raw image set A 120 or B 140, respectively. Another user 102, optionally called Bob, processes the image in various ways. He may, among other things, perform some analysis 145 of the object, try to determine which sensor acquired the image by looking at the noise 150, or apply lossless compression 155. If Alice gives him the image set A 120, Bob finds the result A 165 when analyzing the object in 145, and the sensor noise judgment 150 produces the sensor SA as output 170. Bob is given the image set B140 and in response receives the results B190 from the target analysis 145 and the sensor SB as the output 185 of the sensor noise determination 150. When Bob applies lossless compression to the image set B140, he obtains a compression ratio 180 of about 2:1. Note that the image sets A and B are assumed to show the same target 100 under identical conditions. Nevertheless, the results A165 and B190 may differ at a statistically significant level, since the respective noises in the two image sets affect the output results of the target analysis 145. However, Alice may choose to apply a steganographic process 125, optionally including compression and decompression, to the image set A120 to obtain a processed image set D130. In doing so, she may specify that the processed image set D130 mimics the noise model of the sensor SB as the target sensor 110. Alice may then freely choose 135 which of the two image sets B 140 or D 130 to send to Bob.Without the corresponding steganographic key 160, Bob cannot distinguish between image sets B and D, and in particular cannot detect that one of image sets B and D has gone through steganographic processing / compression, or which of image sets B or D has gone through steganographic processing / compression, unless it is known in advance what processing he applies. More precisely, when he performs lossless compression 155, sensor noise determination 150 and object analysis 145 on either of the two image sets, he obtains results 180, 185, 190 that correspond, within statistical variation, to image set B 140. This gives strong guarantees on the quality and indistinguishability of the (processed) compressed data relative to the (unprocessed) raw data. Bob can distinguish image set D 130 from image set B 140 only in the situation where he possesses the correct steganographic key 160 that was previously used for the steganographic processing of image set A to yield image set D 120. He can, for example, use the key 160 to achieve efficient compression at a ratio 175 of about 10:1 when applying lossless compression 155; without knowing the key 160, he still obtains only a low compression ratio 180 of about 2:1.

[0027] It should be noted here that the source sensor SA105 that gave rise to the image set A120 is not necessarily identical to the target sensor SB115 used as a model to obtain the image set B140. This is because in practical applications it is preferable to provide output data in which various imperfections of the physical sensor, e.g. pixel inhomogeneity, fixed pattern noise, etc., have been corrected, so that the sensor SB115 can be an "ideal" sensor. What is important is to be consistent with the image set D130 originating from a (real or ideal) sensor with well-defined and specified properties. This ensures that any post-processing that occurs on the image set D130 will provide reliable results as if the image set D130 had originated from the target sensor SB115.

[0028] Using an ideal model for the target sensor SB has the additional advantage that several real sensors SA-1, SA-2, ..., SA-N, each with different properties, can be mapped to a unique ideal sensor SB. This is particularly important for machine learning methods, where a neural network trained on images originating from, for example, sensor SA-1, does not necessarily give reliable inference results when used with images from another sensor SA-2. In many practical situations, after using historical data acquired with sensor SA-1 to train a neural network, it is eventually necessary to replace sensor SA-1 with a newer model SA-2, since sensor SA-1 may be discontinued, may become uncompetitive, or end users may not be "attached" to one manufacturer. Therefore, having data from different sensors SA-1, ..., SA-N mapped in a way that is consistent with originating from an ideal sensor SB is useful in these situations.

[0029] In one embodiment of the present invention, the target sensor SB is characterized by a set of properties including its quantum efficiency, read noise, black level and gain. In another embodiment, it is assumed that the target sensor SB forms an ideal device that outputs the exact number of photons absorbed by each pixel. It is also noted here that, as with all physical systems and their inherent imperfections, the mapping of data from the source sensor SA to the ideal model target sensor SB may be done approximately, due to practical limitations on the collected statistics and due to finite imperfections or instabilities in the physical source sensor SA. This does not detract from the invention, and in fact these imperfections may be strictly specified and limited, acceptable to the user.

[0030] In general, data from the target sensor SB 115 is not amenable to lossless compression 155 at ratios 180 that are typically greater than about 2:1. This means that without the steganographic key 160, Bob is unable to losslessly compress at higher ratios. On the other hand, with the key 160 in hand, Bob can perform lossless compression 155 of the data at much higher ratios 175, typically up to about 10:1. This allows the invention to simultaneously provide the user with data that can be used by the user in exactly the same way as the raw data, but to which lossless compression can typically be applied at up to a factor of 10:1.

[0031] The invention described herein has several other advantages that derive from its tight integration with steganography: in particular, the processed and possibly compressed and then decompressed image set D, which is typically ten times larger than the corresponding data compressed by the method according to the invention, is advantageously used to embed useful information, ensuring that the image set D can still be used in any processing application in which the raw image set B can be used, giving results that are statistically indistinguishable from it. In a preferred embodiment, this allows several other pieces of information to be embedded in the final processed image together with the pseudo-noise parameters, for example: Image metadata such as the time the image was captured, the system used to capture the image, and the camera system the image was taken with Data that aids in compression, e.g. properties of the device that took the image, image size and binary layout (e.g. strips, blocks, etc.) Data that enhances the security, integrity and validation of images, such as a hash or similar of the raw image data that may prove the authenticity of the image data, ownership or copyright information, a digital signature from the device that originally captured the data, watermarking data that may be recovered even after processing, and error correction data that may be used to recover any portion of the image and metadata if it is lost or corrupted.

[0032] Furthermore, in order for users to be able to fully benefit from this new compression technology, it is important that they can use it without significant modifications to their workflow or to their existing systems, which implies continuing to use container files of their choice, e.g., known file formats such as tiff, dng, png, dicom, jp2, etc.

[0033] 2a-c show in a schematic way the main steps of a preferred embodiment of the method according to the invention, enabling the above mentioned features. In a pre-processing step 235 shown in Fig. 2a, the raw image data 210 captured by the source sensor SA 205 is pre-processed so that, despite the unpredictable noise occurring at the source sensor SA 205 during the acquisition of the image being unpredictable according to a given input noise model 215, said noise is replaced 240 by a pseudo-random predictable noise according to a target noise model 220 from a real or ideal target sensor SB, which may but not necessarily differ from the source sensor SA 205. Encryption data 222 can be used in the generation of the pseudo-random noise and in the other sub-steps of the pre-processing step 235. Optionally, this preprocessing step 235 also includes substeps 245 for correcting the data for sensor imperfections, substeps 250 for embedding metadata 225 or other data 230 as described above, and / or performing watermarking 255 so that such preprocessed data can be easily identified in subsequent stages even if only a portion of it is available. This preprocessing operation produces "preprocessed raw" data, also referred to as "praw" data 260 in the following description and in the figures. Said praw data 260 is suitable to be used in any application that uses raw data, as described above. This praw data 260, as opposed to standard raw data, can be used alone, since it contains the necessary metadata that makes the data more practically useful. Alternatively and optionally, the praw data 260 may be embedded in a container file 270 of any known file format, such as tiff, jp2, dng, png, etc., as is usually done with raw data. A container file with praw data works with any application that can already be associated with this file type, but has the advantage of being highly compressible. A container file 270 can also store non-image data 275, which may typically include any metadata 225 or other data 230.This can be used to store a copy of data that is already embedded directly in the praw data 260.

[0034] In a preferred embodiment, the compression step following the pre-processing step and illustrated in FIG. 2b uses the steganographic / cryptographic data 222 to predict pseudo-noise and efficiently losslessly compress the praw data 260 at 155. The steganographic / cryptographic data 222 may be encoded in the praw file at the compressor itself and / or provided by the user. The steganographic / cryptographic data 222 may also be a pre-agreed constant or transferred in any other manner. The compression step may also extract metadata 225 and other data 230 if such data was embedded during the pre-processing step, and optionally utilize this data to verify the quality, authenticity and integrity of the file and / or apply error correction if required. The compression step then creates a compressed file 280 containing data that allows the original container file 270 containing the praw data 260 to be losslessly restored.

[0035] Finally, in a decompression step following the compression step and shown in FIG. 2c, the compressed file 280 is decompressed at 295, such as to re-produce the praw data 260, or a container file 270 containing the praw data 260 and optionally some other non-image data 275 as mentioned above for the preprocessing step.

[0036] The container file 270 and the compressed file 280 may have complex structures containing many frames or images. The structure of these files and the location of the binary praw data 260 may not be known in advance to the compressor used in the compression step. In this case, the watermark embedded in the praw data 260 at 255 allows the compressor to identify and locate this data, extract any embedded data, and use the data to perform the compression efficiently. The compressor and decompressor may be explicitly implemented and operated by the user, or may run in the background, such that the files are transparently compressed and decompressed, for example, when passing through a network, when stored on disk, when transferred over a bus, etc. Verification, integrity check, authenticity, error correction, and / or data / metadata extraction features may be operated independently of the compression and / or decompression.

[0037] To describe the method according to the invention in more detail, it is further noted that digital images captured by an image sensor generally contain a significant amount of entropy, mostly due to sensor noise and also to the inherent noise of the light that strikes the sensor. The presence of this noise is an important aspect of what gives an image its inherent statistical properties. As mentioned above, removing this noise may result in processing algorithms that produce different results with respect to images that still contain said noise. However, the presence of noise strongly limits the data size reduction that can be achieved using lossless compression. The invention makes it possible to process digital images such that high lossless compression ratios can be achieved while maintaining a realistic noise profile that does not reveal, visually or statistically, that the image has been processed, for example by a histogram of the image's pixel values. The fact that the image is highly compressible can be hidden in an undetectable steganographic way in images processed by the method according to the invention and can only be revealed with the aid of a steganographic / encryption key or function.

[0038] Thus, the method according to the invention generally comprises the following steps for pre-processing of a given digital image and a corresponding noise model of a digital camera or image sensor with specific settings: (1) Remove most of the noise, in the appropriate mathematical sense, from the image captured by the source image sensor, for example by quantization or denoising. This may include correction of sensor-specific noise such as fixed pattern noise or photoresponse non-uniformity. The result of this step is noted as noise-reduced image data; (2) Generate pseudo-noise by suitable application of a pseudo-random number generator with well-defined initial conditions, for example an integer seed, and add the generated pseudo-noise to the noise-reduced image data such that the resulting pseudo-noise image closely mimics a desired target noise model. The desired target noise model may be suitable to mimic the source sensor that acquired the original image data or may be adapted to different real or ideal target sensors. In the case of ideal functioning of the method according to the invention, it is not possible to know that (most of) the noise of the final processed image was generated using pseudo-random numbers. (3) The pseudo-noise image data, optionally including the noise model with corresponding parameters (including the seed), and metadata and other data, if present and desired, are stored on disk and / or transmitted over a bus or network. In a preferred embodiment and depending on the requirements of the situation, this can happen in different forms that can be partially combined with each other: a) different data are stored / transmitted as separate files, separate parts of a container file or separate data packages, b) metadata and other data are embedded in the pseudo-noise image and the pseudo-noise parameters are stored / transmitted separately, as described in more detail below, c) the pseudo-noise parameters are embedded in a subset of the pixels of the pseudo-noise image using some steganographic method, option b) is used to embed the metadata and / or other data. Furthermore, the storage and / or transmission of the pseudo-noise image data may optionally include one or more of the following steps: Any or all of the data stored / transmitted together with the pseudo-noise image may be encrypted, whether or not embedded in said pseudo-noise image. Any or all of the data stored / transmitted along with the pseudo-noise image, whether or not embedded in said pseudo-noise image, may be coded to facilitate error correction. Any or all of the data stored / transmitted together with the pseudo-noise image as well as the pseudo-noise image itself may be supplemented with a watermark or digital signature to facilitate authentication or identification of the data. The noise model and corresponding parameters (including the seed) may be constant and fixed, for example as when hard-coded into hardware or software adapted to implement the method of the invention, or published on a website, in a specification, or similar.

[0039] In the simplest embodiment of the method according to the invention, most of the unpredictable noise in an image is replaced by a predictable pseudo-noise, which includes the steps of determining a source and a target noise model after acquiring image data captured by a source image sensor, generating a pseudo-noise, removing most of the noise from said image data, adding the pseudo-noise to the noise-reduced image data to create a pseudo-noise image data that is statistically equivalent to the image data acquired with said target image sensor, and storing and / or transmitting said pseudo-noise image data. Since the pseudo-noise image data is statistically equivalent to the image data acquired with said target image sensor, the process described above is called steganographic to emphasize that the image thus preprocessed is not easily distinguishable from the natural image generated by the sensor to which the target noise model is applied. In this simplest embodiment, the replacement of the natural noise with the generated pseudo-noise in a steganographic way may occur without using a steganographic key, and neither the target noise model nor any metadata or other data of the image need to be stored and / or transmitted, for example together with the pseudo-noise image data, in particular by embedding in the pseudo-noise image data. Indeed, in this case the target noise model and its parameters are chosen to be constant and can therefore be made available / accessible by other means as described above. As will become clear below, more sophisticated and precise embodiments of the method according to the invention are preferred, making it possible to add, in different ways and preferences, the parameters of the target noise model and / or metadata and / or other data of the image to the pseudo-noise image data, in particular by embedding this additional information in the pseudo-noise image data, optionally using a steganographic key.Therefore, in any of these cases, it is ensured that the target noise model and corresponding parameters used to generate the pseudo-noise that is added to the noise-reduced image data to create the pseudo-noise image data are available, i.e. made accessible in some way during the compression / decompression step or during any other desired step of the method according to the invention that is envisaged to be applied to the pseudo-noise image data, so as to enable the reconstruction and / or subtraction of the pseudo-noise based on said pseudo-noise image data and the parameters of the target noise model.

[0040] In this context, the steganographic key can be secret or public depending on the application, and the steganographic key, or generally, when creating pseudo-noise image data, adds the generated pseudo-noise in a steganographic manner to the noise-reduced image data, serving the following purposes: The fact that the file has been processed, i.e. the original image data has been replaced by pseudo-noise image data according to a given target noise model, is practically undetectable except with the aid of said steganographic key. It is again practically undetectable that the pseudo-noise image can be efficiently losslessly compressed as a result of the processing. More precisely, lossless compression is as inefficient as it is for the unprocessed image data, and efficient compression can only be "unlocked" and made available by using said steganographic key. The fact that in addition to the actual image data the pseudo-noise image data may also contain additional data, which is also practically undetectable, can only be accessed with the aid of said steganographic key.

[0041] In a subsequent step, the pseudo-noise image can be efficiently compressed without loss by using the pseudo-noise parameters to generate an identical copy of the pseudo-noise, which is then subtracted from the image so that an identical noise-reduced image of step (1) above is retrieved. Depending on which possibility is used for step (2) above, it may be necessary to extract the pseudo-noise parameters from the image itself using a steganographic key, or to read said parameters from a separate file or data package, or otherwise make available a file or data package in case the noise model and the corresponding parameters (including the seed) are constant and fixed. Furthermore, metadata and other data can be extracted from the image if they were previously embedded therein. The noise-reduced image can then be efficiently compressed (typically by a factor of 5-10) using a custom lossless image codec that owes its existence or to previous noise reduction, for example as defined in the JPEG 2000 standard. The pseudo-noise parameters must be kept and can for example be stored together with the compressed image data.

[0042] The corresponding decompression can be performed by first decompressing the image data using the same lossless codec and then reapplying steps (2) and (3) above with the same parameters of the pseudo-noise. In case of ideal functioning of the method according to the invention, the result will be bit-for-bit identical to the final processed image obtained after step (3) above.

[0043] In view of the principles discussed above, it is generally possible to realize different embodiments of the method according to the invention by implementing these steps in different stages of an image pipeline and / or by splitting these steps between different hardware and software. For example, an implementation may be directly integrated into an image sensor, which then performs steps (1)-(3) before the image data is transferred to a storage area of ​​the sensor or to an output bus, with subsequent lossless compression and decompression functionality being provided by a software module for a computer, tablet, mobile phone or similar.

[0044] Thus, the method according to the invention, thanks to said pre-processing step, allows a noise-reduced version of an image to be suitably retrieved for efficient lossless compression from a noisy version of the same image. Preferably, this is achieved by replacing the natural noise, whose source is usually a combination of the quantum noise of the light captured by the image sensor and the electronic noise of the image sensor itself, with an artificial pseudo-noise generated with the aid of a pseudo-random number generator. While the natural noise is unpredictable, the pseudo-noise is deterministic and can be reproduced identically if the exact parameters and algorithms for the generation of the noise are known. However, an essential part of the invention is that the pseudo-noise is as similar as possible to the natural noise of a given desired real or ideal target image sensor by mimicking its characteristics through the use of a noise model, as will be explained in more detail below. Accordingly, the invention further provides a method for lossless compression and decompression of images, although the pseudo-noise, said compression and decompression steps are different compared to the corresponding steps in the prior art, also as will be explained in more detail below.

[0045] Further details about some of the sub-steps of image pre-processing 235, compression 155, and decompression 295 are provided below.

[0046] Image preprocessing The first step is said image pre-processing step 235, during which the original raw image data 210 is processed so as to be replaced by a pseudo-noise version of the same image. The pseudo-noise parameters are preferably stored / transmitted together with the pseudo-noise image or embedded directly in the image using a steganographic, i.e. ideally undetectable, method. Without the corresponding steganographic key, it should be impossible to know that the image contains pseudo-noise and not natural noise, and the image can be used for the same purposes and applications as the original image data 210. As already mentioned above and shown diagrammatically in Fig. 2a, metadata 225 and other relevant data 230 may optionally be embedded in the pseudo-noise image, resulting in praw data 260 that is self-contained and may represent additional value compared to the original image data. For example, it is possible to embed other parameters important for optimal processing of the image data or to embed data that attests to the authenticity of the image.

[0047] 2a shows a schematic representation of the image pre-processing step 235. The description in this section and in the following sections concentrates on so-called raw image data, i.e. mostly unprocessed image data coming directly from a CMOS or CCD image sensor or equivalent device. Those skilled in the art will understand that the invention can be equivalently applied to other kinds of image data by adapting some of the steps involved, in particular by using appropriate noise models.

[0048] The first step within the image pre-processing step 235 is to take the raw image data 210 and perform a noise substitution 240, where most of the random noise in the image, characterized by the input noise model 215, is replaced with pseudo-noise, characterized by the target noise model 220. The noise substitution procedure is illustrated in FIG. 3. It starts with applying a denoising or quantisation step 305 with the help of a suitable input noise model 215 and associated parameters to generate a noise reduced image 325. A further component may be a pseudo-random number generator 320, the use of which will become clear in the following section. The noise in raw image data captured by a CCD or CMOS sensor is often well described by a Poisson-Gaussian model, for which the value x i The estimated standard deviation σ of pixel i with i teeth,

number

number

number

[0049] Noise reduction pixel value y i The quantized image consists of the original pixel values ​​x i Subsequent lossless compression of the quantized image is more efficient since it has fewer distinct values ​​than the original image data consisting of noise-reduced pixel values ​​y i may be scaled back to the original range by multiplication with σ / q. As mentioned, similar results can be achieved by applying known denoising techniques, but only at the expense of computational cost and greater care in some of the subsequent steps of the method.

[0050] During the generation of the noise reduced image 325, either by quantization or by noise reduction techniques 305, most of the natural noise is removed and a small amount of artificial deterministic noise is removed, R i may have been added due to the introduction of

[0051] In a second step within the image pre-processing step 235, the image is re-noised at 330 according to the target noise model 220. For example, the original noise level is essentially

number

[0052] The third step within the image pre-processing step 235 of the method according to the invention is to store or package for transmission the pseudo-noise image data and optionally other data including pseudo-noise parameters and metadata. If properly implemented, this step a) allows the pseudo-noise image data to be used even if raw image data of the corresponding type and format has been used. Furthermore, the steganographic key allows b) in a subsequent stage to retrieve the noise-reduced image data 325 from the pseudo-noise image 335 so that it can be efficiently stored and transmitted with the aid of lossless compression if so desired, and c) to retrieve metadata 225 and / or other data 230 that can be used to enable, facilitate or enhance image processing or analysis.

[0053] In some embodiments of the method according to the invention, the pseudo-noise image, noise parameters and other data are stored together in a suitable data structure, directory or other container, as for example provided by the Tagged Image File Format (TIFF / EP, ISO 12234-2). The seed value 315 for the pseudo-random number generator 320 may then act as a steganographic key, as it can be combined with the noise parameters to derive the noise-reduced image 325, as described below. Depending on the specific requirements of a given application of such an embodiment, the seed can be secret or publicly available.

[0054] However, it is preferred to make the pseudo-noise image 335 as self-contained as possible, i.e. to embed as much data as possible in the pseudo-noise image data itself. In one embodiment of the method according to the invention, shown diagrammatically in Fig. 3, the image metadata 225 and other data 230 are embedded during the renoising step 330 by first encoding the data as a suitable bit string 340, which encoding may include means for encryption and / or error correction. The embodiment then preferably makes use of two separate seed values ​​315S1 and S2, where S2 may be derived from S1. The bit string 340 may be embedded in the sequence of pixels by using S1 to generate a random number whenever the bit is 0 and S2 otherwise, one bit of the string per pixel. Sometimes S1 and S2 may be embedded in the same pseudo-noise pixel value z i , resulting in the occurrence of so-called collisions. Collisions can be handled by suitable error correction methods known in the art when the bit strings are extracted from the pseudo-noise image 335 in a subsequent stage. Given that the pseudo-noise is undetectable when a single seed value 315 for the pseudo-random number generator 320 is used, the use of two separate seed values ​​should be equally impossible to detect, and thus the data is safely hidden in the pseudo-noise image 335.

[0055] In another embodiment of the method according to the invention, the data embedding is extended to include the pseudo-noise parameters themselves. For example, a cryptographic version C of the parameters P={σ0, q, S, T} P is generated using an encryption step that uses a steganographic key K, where S represents one or more seed values ​​for the random number generator 320, and T is a concatenation of several pixels at fixed, predefined positions in the pseudo-noise image 335. T is optional, and C can be any number even if σ, q, and S are the same. PThis can act as a cryptographic salt to ensure that P varies from image to image. Note that P does not necessarily contain information about the type of noise model used in these steps, but it can be easily extended to do so. The next step is the actual embedding procedure. In a simple embodiment, the cryptographic salt of the parameters C P is converted to a bit string of length k, and the least significant bits of the first k pixels not used in generating T are overwritten with the corresponding values ​​in the string. Thus, the values ​​of these pixels are z i From z i '. C P This method of embedding has limited security, and more sophisticated implementations may use other methods of data embedding. P Once embedded, the metadata 225 and other data 230 may be embedded into the remainder of the image using two different seed values, as described above.

[0056] More sophisticated embodiments may also add redundancy and error correction provisions for all data embedding so that the embedded data may be extracted when the image data is cropped or otherwise modified. Regardless of the particular method of data embedding, the recovered pixel values ​​z i ' is related i (z i ')=y i (x i ), i.e. the processed image with pseudo-noise parameters resulting from the embedding step in the image pre-processing step 235 of the method according to the invention can be equally well used to obtain the noise reduced image data 325 as the original image data 210, assuming all other parameters are known and unmodified.

[0057] compression In general terms, the second step of the method according to the invention, which is optional and may be implemented in a subsequent stage, is the compression step 155. This essentially consists of deriving a noise-reduced image 325 from the pseudo-noise image 335 with the help of pseudo-noise parameters and a seed value 315 for the pseudo-random number generator 320. The noise-reduced image 325 is then compressed using a lossless compression algorithm. Preferably, the pseudo-noise parameters and possibly additional data are combined with the compressed image data 285 to form a compressed image with pseudo-noise parameters, for example by storing everything in a single container file 280 or directory to facilitate a subsequent decompression step 295. Optionally, the compressed image with pseudo-noise parameters may be made to be directly opened or viewed by the imaging application, either by using a container file, a format that allows direct data access to the noise-reduced image data, respectively, or by adding some kind of preview image to the container.

[0058] The optional compression step of the image data produced by the image pre-processing procedure 235 mentioned above will now be described in detail with reference to FIG. 4. For simplicity, the description uses the same simple noise model as in the previous section, namely that of constant signal-independent noise represented by a standard deviation σ0. However, this description applies analogously to any other target noise model 220. The compression step takes as a starting point the container file 270 or equivalent data structure previously created by the image pre-processing step 235. The first step is the retrieval 405 of the pseudo-noise parameters and seed value 315. For embodiments where these parameters are embedded in the pseudo-noise image, the extraction proceeds as follows: First, a bit string of length k is generated by concatenating the least significant bits of the first k pixels that do not contribute to the generation of T. This bit string is derived from the parameters C P C Pis then decrypted with a key K, which may be represented by the seed value 315 as described above, to retrieve parameters P={σ0, q, S, T}, which provide the necessary information for the target noise model 220, and the pseudorandom number generator 320. Then, in a second step, these parameters are used to retrieve the metadata 225 and other data 230 by bit string decryption 410, and -R i This can then be used to regenerate the pseudo-noise according to the target noise model 220 essentially given by σ0 / q. The noise is then subtracted at 415 from the pseudo-noise image 335 in the praw data 260 of the container file 270 to obtain the noise reduced image 325. For the simple noise model used here, these steps are

number

[0059] Here, this is the original pixel value x i The restored pixel values ​​z iIt should be noted that this is the same formula used to calculate the noise-reduced image 325 during the image pre-processing step 235, except that the processed image data represented by ' is used here. In a third step, lossless compression 420 is applied to the noise-reduced image data 325. The compression algorithm may be chosen according to the requirements of the context in which the method according to the invention is used, among various existing or custom algorithms for image data, such as lossless JPEG or PNG, or generic compression such as ZIP. Finally, the pseudo-noise parameters, metadata 225 and other data 230 are packaged in 425 with the optionally encrypted and compressed image data in a compressed file 280 or equivalent data structure. Depending on the application, this concatenation can be performed by simply appending binary data to the same file, by using several files in a container such as an archive, or by using the mechanisms of a special file format such as TIFF.

[0060] Unzip In view of the above description of exemplary embodiments for the preprocessing 235 and compression 155 steps, the corresponding decompression step of the method according to the invention needs only to be briefly described and is only diagrammatically illustrated in Fig. 2c. Starting with a compressed file 280 or an equivalent data structure, the first step of such a decompression step is the separation of the parameter P. The separation is optional and can be achieved by separating the parameters C P Then, lossless decompression produces the exact same pixel values ​​y obtained during the image preprocessing step 235. i2. Accordingly, decompression proceeds by reproducing the same steps performed during the preprocessing step, i.e., adding pseudo-noise to the noise reduced image data 325 in a renoising step 330, as well as storing or embedding the pseudo-noise parameters, metadata and / or other data, i.e., generating the praw data 260 and placing everything into a container file 270. The output of the decompression procedure is a bit-identical copy of the praw data 260 and / or container file 270, produced by the image preprocessing step 235.

[0061] In view of the above description of some embodiments of the method according to the invention, it will be clear to those skilled in the art that the invention has some important advantages. First, the invention allows the processing of digital image data such that high lossless compression ratios, typically up to about 10:1, can be achieved, while maintaining a realistic noise profile that does not reveal visually or statistically that the image has been processed. Second, the invention ensures that image data compressed and decompressed from the original "raw" image data cannot be distinguished from the actual raw image data of a real or ideal sensor for which a noise model was used, without knowledge of the steganographic method or key used to process the image data. Third, the invention advantageously also allows some type of useful information to be embedded in the processed image data, while still ensuring that the processed image data can be used by any application for which the raw image data can be used, giving results that are statistically indistinguishable therefrom. Finally, the present invention allows users to fully benefit from the corresponding methods by being able to use this technology without significant modifications to their workflow or to their existing systems, and by continuing to use container files of their choice, e.g. files in known file formats such as tiff, dng, png, dicom, jp2, etc. [Explanation of symbols]

[0062] 105 Image Sensor SA, Source Sensor SA 110 Target Sensor 115 Image sensor SB, target sensor SB 120 Raw Image Set A 130 Processed Image Set D 140 Raw Image Set B 160 Steganographic Keys 205 Source Sensor SA 210 Raw Image Data 215 Input Noise Model 220 Target Noise Model 225 Metadata 230 Other Data 235 Pre-processing Steps 270 Container File 305 denoising or quantization steps 315 Seed S 325 Noise-reduced image data 335 Pseudo-noise Image Data

Claims

1. 1. A method for steganographic processing and compression of image data with negligible information loss, said image data containing noise and information, said method comprising: obtaining image data (210) to be processed and / or compressed for storage and / or transmission; A step of pre-processing (235) the acquired image data (210) for compression, the pre-processing step (235) comprising: determining an input noise model (215) and corresponding parameters adapted to reflect noise created by a source image sensor (SA) used to capture the image data; removing (305) noise from the acquired image data (210) with negligible information loss by use of the input noise model (215) to produce noise-reduced image data (325); storing and / or transmitting the image data; Including, The step (235) of pre-processing the acquired image data (210) comprises: determining a target noise model (220) and corresponding parameters adapted to reflect noise produced by a real or ideal target image sensor (SB); generating (320) a pseudo-noise according to the target noise model (220); adding (330) the generated pseudo-noise to the noise-reduced image data (325) in a steganographic manner to produce pseudo-noise image data (335) that is statistically equivalent compared to image data acquired using the target image sensor (SB); Further comprising: The storing and / or transmitting step further comprises: storing and / or transmitting said pseudo-noise image data (335) so as to enable reconstruction and / or subtraction of said pseudo-noise based on said pseudo-noise image data (335) and said parameters of said target noise model (220); The method according to claim 1, further comprising:

2. 2. The method of claim 1, wherein the target noise model (220) and the parameters of the target noise model (220) are constant and available and / or stored and / or transmitted together with the pseudo-noise image data (335).

3. The step (235) of pre-processing the acquired image data (210) comprises: providing a steganographic key (K); storing and / or transmitting the parameters of the target noise model (220) used to generate the pseudo-noise, preferably embedding the parameters in the pseudo-noise image data (335) so as to create the pseudo-noise image data (335) by use of the steganographic key (K), the steganographic key (K) preferably comprising at least one seed (315) for a pseudo-random number generator (320); 3. The method according to claim 1 or 2, characterized in that it comprises:

4. The step (250) of embedding the parameters of the target noise model (220) used to generate the pseudo-noise in the pseudo-noise image data (335) comprises: encrypting said parameters based on said steganographic key (K); 4. The method of claim 3, comprising:

5. The pre-processing step includes: correcting the acquired image data for sensor imperfections; embedding said image metadata (225) and / or other data (230) into said pseudo-noise image data (335), said other data (230) including calibration data for said sensor and / or data useful for compression, preferably properties of the sensor used to capture the acquired image data, image size and / or binary layout; Embedding other data (230) that enhances the security, integrity and validity of the image data into the pseudo-noise image data (335), preferably a hash of the obtained raw image data, ownership and / or copyright information, and / or watermarking data; and / or embedding error correction data in said pseudo noise image data; Further comprising: The step of embedding metadata (225) and / or other data (230) of the image preferably comprises: switching between two or more seed values ​​(315) for a random number generator (320) used in encoding the metadata (225) and / or the other data (230) and adding (330) pseudo-noise to the noise reduced image data (325).

5. The method according to claim 1 , comprising:

6. 6. The method according to claim 1, wherein the pseudo-noise image data (335) is stored and / or transmitted as is or embedded in a file or container file (270) of any format such that the pseudo-noise image data (335) is compatible with any application adapted to work with files or container files of said format.

7. The input noise model (215) adapted to reflect noise produced by the source image sensor (SA) and / or the target noise model (220) adapted to reflect noise produced by the target image sensor (SB) may be [0010] The value of pixel i is given by x i Standard deviation for σ i where the noise parameters are the internal signal amplification parameter a of the sensor, the black level parameter x of the sensor, 0 and a read noise parameter b of the sensor; or The standard deviation σ of the pixels of the sensor that is independent of the values ​​of the pixels 0 7. The method according to claim 1, wherein the method is realized by a simplified model having:

8. The method comprises: A step of compressing (155) the pseudo-noise image data (335) produced by the pre-processing step (235), either as is or embedded in a container file (270).

8. The method according to claim 1 , further comprising:

9. The compressing step (155) comprises: obtaining the pseudo-noise image data (335) produced by the pre-processing step (235); extracting (405) the parameters of the target noise model (220) used to generate the pseudo-noise based on the steganographic key (K) and as long as the parameters of the target noise model (220) have been stored together with the noise-reduced image data (325); generating a pseudo-noise according to the extracted or otherwise obtained parameters of the target noise model (220); a step (415) of subtracting the generated pseudo-noise from the pseudo-noise image data (335) generated by the pre-processing step (235) to generate noise-reduced image data (325); Losslessly compressing (420) the noise reduced image data (325) to produce compressed noise reduced image data (325); Optionally packaging (425) the parameters of the target noise model (220), and metadata (225) and / or other data (230), if present, together with the compressed, noise-reduced image data; storing and / or transmitting the compressed, noise-reduced image data, optionally including the parameters of the target noise model (220) used to generate the pseudo-noise and metadata (225) and other data (230); 9. The method of claim 8, comprising:

10. 1. A method for decompressing image data without loss of information, comprising: Obtaining compressed image data produced by the compressing step (155) of the method according to claim 8 or 9; Obtaining parameters of a target noise model (220) used to generate the pseudo-noise according to the method of any one of claims 1 to 7, a step of decompressing (295) the compressed image data produced by the compressing step (155); The method according to claim 1, further comprising:

11. The method comprises: separating the compressed noise reduced image data and the parameters of the target noise model (220) used to generate the pseudo-noise, so long as the parameters of the target noise model (220) are embedded in the noise reduced image data (325); losslessly decompressing the compressed noise reduced image data to produce decompressed noise reduced image data (325); extracting the parameters of the target noise model (220) used to generate the pseudo-noise, as long as the parameters of the target noise model (220) have been stored together with the noise-reduced image data (325) and based on a steganographic key (K); generating a pseudo-noise according to the extracted or otherwise obtained parameters of the noise model (220); adding (330) the generated pseudo-noise to the decompressed noise reduced image data (325) to produce pseudo-noise image data (335); Optionally embedding (250) the parameters of the noise model (220), and, if present, metadata (225) and / or other data (230) in the pseudo-noise image data (335); storing and / or transmitting said pseudo-noise image data (335), optionally including said parameters of said noise model (220) used to generate said pseudo-noise and metadata (225) and / or other data (230); The method of claim 10, comprising:

12. The steganographic key (K) encoded in the pseudo-noise image data (335), in a compressor or in a decompressor; used in the compressing step (155) and the decompressing step (295), 12. The method according to claim 10 or 11, characterized in that the information is available in predefined information sources, including internet sites and internal documentation, or is publicly available and / or is provided by a user of the method.

13. 13. The method according to claim 10, characterized in that the generation and / or embedding (250) of the pseudo-noise by the parameters of the noise model (220), respectively, is performed cryptographically by using a pseudo-random number generator (320).

14. 14. The method according to any one of claims 10 to 13, characterized in that the steps of the method for steganographic processing and compression of image data and / or the steps of the method for decompression of image data are divided and executed between different hardware and / or software.

15. A computer program causing a computer to carry out a method according to any one of claims 1 to 14.

16. A device configured to carry out the method according to any one of claims 1 to 14, characterized in that the device is selected from the group including a microprocessor, a field programmable gate array, an image sensor, a mobile phone, in particular a smartphone with a digital camera, a digital photographic device, a digital video camera, a scanning device, a tablet, a personal computer, a server, a microscope and a satellite.

Images