Method for digitally watermarking an image, method for extracting a digital watermark, and method for detecting forgery of a watermarked image

Abstract

A computer-implemented method of digitally watermarking an image comprising the steps of: Receiving (201) a digital image, Decomposing (202) the digital image into a plurality of sub-blocks of the same predefined dimensions, Generating (203) a digital binary signature, For each sub-block, Transforming the sub-block into an intermediate space via a predetermined transform, Combining the signature with the sub-block in the intermediate space to obtain a signed sub-block in the intermediate space, Applying to the signed sub-block a transform inverse to the predetermined transform, Concatening (206) all of the signed sub-blocks to form a signed image. Figure 2

Description

Title of the invention: Method for digitally watermarking an image, method for extracting a digital watermark and method for detecting falsification of a watermarked image

[0001] The invention relates to the field of monitoring the integrity of digital images, in particular images acquired by an Earth observation satellite.

[0002] The invention relates more specifically to a method of digitally tattooing an image, a method of extracting a tattoo and a method of detecting falsification of a tattooed image.

[0003] In the context of the increase in the quantities of data, in particular images, transmitted via digital transmission systems, there is a general need to detect intentional or involuntary attacks on the integrity of the images received.

[0004] In particular, in the field of transmissions of images acquired by an Earth observation satellite, this need is particularly present because the acquired images can pass through different equipment, communications networks and transmission platforms.

[0005] In this context, the possibilities of compromising the integrity of images increase and it is imperative for a user to be able to verify this integrity before the final use of the images.

[0006] A more specific problem concerns the localization, in the image, of a falsification of the image, that is to say the capacity to identify which area of ​​an image has been falsified.

[0007] Another specific problem concerns the robustness of a digital watermarking method to certain so-called classic transformations such as transformations for cutting or cropping an image.

[0008] Different solutions exist in the state of the art to address the issue of image integrity.

[0009] A first known solution consists of calculating, at the time of production or delivery of the image, a unique key of fixed length precisely representing an original object and its characteristics. This key can be recalculated at the time of the image integrity verification and compared to the original key to deduce whether a modification has taken place on the image. This first method has the advantage of low complexity, however it has several disadvantages. First of all, it is not robust to any transformation carried out on the image. The slightest modification of the image such as a cut-out or a mirror results in a different key and therefore leads to detecting a falsification of the image whereas it has only undergone a trans- acceptable training.

[0010] Another disadvantage is that this method involves a calculation on the entire image which does not allow locating the areas of the image affected by falsification.

[0011] A second known solution is to apply a digital watermark to an entire image. If the image is modified, the watermark is visibly altered and allows forgeries to be detected. The main disadvantage of this method is that it visually and functionally alters the usability of the image. Another disadvantage is that this method does not allow forgery to be located in a local area of ​​the image.

[0012] A third known solution consists of analyzing the content of the received image and comparing it to a statistical expectation to evaluate the probability of authenticity of the information contained in the image. This solution does not apply to all technical fields because it assumes strong a priori assumptions about the content of the processed images and any falsifications applied to the image.

[0013] The invention consists of a new digital watermarking method which consists of integrating a watermark on each sub-block of a grid applied to the image so as to allow falsification to be located in the image.

[0014] The invention also relates to a method for extracting the digital tattoo as well as a method for locating forgery which does not require a reference tattoo.

[0015] Advantageously, the invention relies on an artificial intelligence engine to carry out the learning of the tattooing and tattoo extraction functions.

[0016] The proposed method has the particular advantage of being robust to image cutting or cropping operations.

[0017] The invention is advantageously applied to the verification of the integrity of Earth observation satellite images.

[0018] The subject of the invention is a method, implemented by computer, of digital watermarking of an image comprising the steps of: - Receiving a digital image, - Decomposing the digital image into a plurality of sub-blocks of the same predefined dimensions, - Generate a digital binary signature, - For each sub-block, i. Transform the sub-block into an intermediate space via a predetermined transform, ii. Combine the signature with the sub-block in the intermediate space to obtain a signed sub-block in the intermediate space, iii. Apply an inverse transform to the signed sub-block

[0019]

[0020]

[0021]

[0022]

[0023]

[0024]

[0025] predetermined transform, - Concatenate all signed sub-blocks to form a signed image. In an alternative embodiment, the digital tattooing method according to the invention further comprises the steps of: - Add a binary pattern to the digital binary signature, - Transform the binary signature into a matrix of dimensions smaller than the dimensions of a sub-block, - The binary pattern being inserted into the binary signature so as to be positioned on predefined positions of the matrix identical for each sub-block According to a particular aspect of the invention, the binary pattern is positioned on at least three areas of the matrix corresponding to three corners of a square, for example three corners of the matrix. According to a particular aspect of the invention, the step of combining the signature with a sub-block is carried out by means of a first artificial intelligence engine trained to minimize the visual impact of a signature on a signed image. The invention also relates to a method, implemented by computer, for extracting a digital tattoo comprising the steps of: - Receive a digital image signed using the digital watermarking method according to the invention, - Decompose the digital image into a plurality of sub-blocks of the same dimensions used to generate the signed digital image, - For each sub-block, i. Extract the watermarked binary signature in the sub-block According to a particular aspect of the invention, the step of extracting the binary signature of a sub-block is carried out by means of a second artificial intelligence engine trained to minimize a gap between the originally generated binary signature and the binary signature extracted from a sub-block. According to a particular aspect of the invention, the first artificial intelligence engine and the second artificial intelligence engine are trained jointly. In an alternative embodiment, the method for extracting a digital tattoo according to the invention further comprises the steps of: - Detect, for at least one sub-block, a binary pattern inserted in the extracted signature, - Compare the positions of the detected binary pattern to a set of expected positions and if they differ from the expected positions, deduce that the received digital image has undergone a cutting operation, - Reconstruct the binary signature associated with the image before operation cutting from the positions of the binary pattern by applying to the digital signature an operation inverse to the cutting operation

[0026] The invention also relates to a method, implemented by computer, for detecting falsification of a digitally tattooed image by means of the digital tattooing method according to the invention, the method comprising the steps of: - Executing the method for extracting a digital tattoo for an image received according to the invention, - Search, for all the sub-blocks of the image, the extracted signature which is the majority among this set, - for each sub-block, i. compare the extracted signature for the sub-block to the majority signature and, ii. deduce whether the image sub-block has been tampered with or not

[0027] According to a particular aspect of the invention, the digital image is an image acquired by a satellite.

[0028] Other characteristics and advantages of the present invention will appear better on reading the description which follows in relation to the following appended drawings.

[0029] [Fig. 1] represents a general block diagram of a digital watermarking method according to the prior art,

[0030] [Fig.2] represents a diagram of a digital tattooing method according to an embodiment of the invention,

[0031] [Fig.3] represents a diagram of a method for extracting a digital tattoo according to an embodiment of the invention,

[0032] [Fig.4] represents a diagram of an artificial intelligence engine trained to perform a first digital tattoo function and a second digital tattoo extraction function according to an embodiment of the invention,

[0033] [Fig.5] represents an example of an artificial intelligence model trained to perform the first digital tattoo function,

[0034] [Fig.6] represents an example of an artificial intelligence model trained to perform the second digital tattoo extraction function,

[0035] [Fig.7] represents a flowchart detailing the steps of implementing a falsification detection method according to an embodiment of the invention,

[0036] [Fig.8a] represents for the same example, the digital tattoo extracted if no cutting of the image has taken place,

[0037] [Fig.8b] represents an example of digital tattoo extracted following an image cutting operation,

[0038] [Fig.9a] represents a first example of a pattern combined with a digital tattoo according to an embodiment of the invention,

[0039] [Fig.9b] represents a second example of a pattern combined with a digital tattoo according to an embodiment of the invention,

[0040] [Fig.l] illustrates a block diagram of a method of digitally watermarking an image to verify the integrity of the image.

[0041] The principle of digital tattooing or "watermarking" in English is to integrate into an image I a tattoo or binary signature which is not visible in the sense that it does not alter the content of the image but which is modified when a falsification of the image is carried out during the transmission chain.

[0042] In [Fig.l], an image I is represented which is projected into a particular space by means of a given transformation, for example a Fourier transform, a DCT transform or other to obtain a transformed image F.

[0043] A binary tattoo or signature W is then combined with the transformed image F to obtain a watermarked image I't. The watermarked image I't in the transformation space is then converted into the initial space via an inverse transformation to obtain the signed image It.

[0044] To extract the signature W from the watermarked image It, the inverse operations are applied, that is to say that the watermarked image is transformed in the transformation space and then the inverse operation is applied to the combination operation to recover the signature.

[0045] It is then possible to compare the extracted signature with the original signature to detect falsification of the image.

[0046] The combination operation can take different forms but must be robust to certain image processing operations, for example cutting operations or "cropping" in English. In other words, a cutting operation of the watermarked image It must not alter the signature buried in the image.

[0047] The invention relates to a particular method of digital tattooing which notably allows the location in the image of a falsification.

[0048] [Fig.2] represents a flowchart detailing the steps of a method of digital tattoo according to one embodiment of the invention.

[0049] The method begins at step 201 by receiving a digital image which is, for example, an Earth observation image acquired by a satellite. The watermarking method can be applied for example in a ground station belonging to a ground segment of a satellite communications system or in a remote server.

[0050] The image may be a single-channel image or a multi-spectral image comprising several channels at different wavelengths.

[0051] In step 202, a decomposition of the image into sub-blocks of identical dimensions is applied according to a predetermined image division grid. The dimensions of the sub-blocks are chosen a priori according to constraints linked to the data storage and processing complexity but also according to the level of granularity desired to locate the impact of image tampering.

[0052] In step 203, a binary signature S or tattoo is received which contains, for example, useful information. This signature is then integrated into each sub-block in step 205 according to a digital tattooing process as described in [Fig. 1]. A notable difference is that the tattooing operation is not applied to the entire image I but to each sub-block in which the same signature is integrated.

[0053] Then in step 206, the signed image is reconstructed by aggregating all the watermarked sub-blocks according to the same decomposition grid used in step 202.

[0054] [Fig. 3] represents a flowchart detailing the steps of a method for extracting signatures from a digitally watermarked image, according to one embodiment of the invention.

[0055] The method of [Fig. 3] begins with the reception 301 of an image It which has been watermarked using the watermarking method of [Fig. 2]. In step 302, the image is decomposed into sub-blocks according to the same cutting grid used to watermark the image. Then, in step 303, a signature is extracted from each sub-block using a signature extraction procedure which implements operations inverse to those used to watermark the image.

[0056] We ultimately obtain a number N of signatures, where N is the number of blocks in the image.

[0057] A falsification detection step 304 can then be applied by comparing these signatures with each other since these signatures are supposed to be all identical if the image has not been falsified.

[0058] [Fig.4] discloses a particular example of implementation of the steps of integrating a signature into a sub-block (step 205) and of extracting signatures from a sub-block (step 303).

[0059] In a particular embodiment of the invention, the digital watermarking technique is based on the use of two artificial intelligence engines. The first artificial intelligence engine is trained to learn how to perform a function of integrating a binary signature into an image so as to minimize the visual impact of the signature on the image. The second artificial intelligence engine is trained to learn how to perform a function of extracting a signature from a watermarked image so as to best reconstruct the initial signature.

[0060] An example of a set of two jointly trained artificial intelligence engines is depicted in [Fig.4].

[0061] An image I is received as input and then decomposed into sub-blocks B;. Each sub-block is produced as input to a first encoder-type RI network responsible for transforming each sub-block into a latent space.

[0062] A binary signature S is received as input, it is transformed into a matrix W of size at most equal to that of a sub-block. The matrix signature W is then concatenated, via a concatenation operation C, to the output vector of the first encoder network RI, then the whole is supplied to a second network R2 of decoder type which decodes the information of the latent space to obtain a signed sub-block BS;. A cost function Fl is used for training the auto-encoder network consisting of RI and R2. This cost function is, for example, defined by a minimization of an error between a signed sub-block Bs; and an original sub-block B;.

[0063] A third network R3 is trained to extract a signature from a signed block Bs; to find an extracted signature Se.

[0064] In an alternative embodiment, the third network R3 is trained by applying certain perturbations to the input blocks in order to encourage robust learning. The perturbations correspond to alterations to the image that could be carried out by a third party. For example, this could involve the addition of different types of noise, a change in intensity or contrast in the image, a compression of the image or a geometric transformation such as a rotation or scaling.

[0065] A second cost function F2 is used for training the third network R3, for example defined by an error between the extracted signature Se and the original signature S.

[0066] Advantageously, the training of the three networks R1, R2, R3 is carried out jointly from an image database, for example satellite images, by producing at the input of the first network RI sub-blocks chosen randomly in each image.

[0067] One objective of the learning of these networks is in particular robustness to certain transformations of the image, such as for example a mirror transformation, invisibility of the signature buried in the image, an absence of functional impact on the usability of the image but also a minimum capacity of the quantities of data which can be included in the signature.

[0068] Different artificial intelligence algorithms can be used to realize the networks R1,R2,R3. An example of compatible artificial intelligence models of the system described in [Fig.4] is presented in Figures 5 and 6. It is based on the model described in reference [1].

[0069] The diagram in [Fig.5] represents an example of an autoencoder artificial neural network for producing the RI and R2 networks in [Fig.4].

[0070] It is composed of a first convolution layer 501 with fixed parameters to transform a block of an image into a latent space, a concatenation operation C to concatenate a signature S to the transformed image, a neural network ar 502 and a second convolution layer 503 with fixed parameters to transform the result obtained into the image space.

[0071] The network 502 is trained to learn to integrate the signature into the image without visual impact. It is for example composed of several layers of successive convolutions.

[0072] [Fig.6] describes an example of an artificial neural network for realizing the R3 network of [Fig.4],

[0073] It is composed of a first convolution layer 601 with fixed parameters then a set of convolution layers 602 whose parameters are learned to extract a signature Se from a signed block B s.

[0074] Without departing from the scope of the invention, any other trainable artificial intelligence model for learning to perform the two aforementioned functions can be used to replace the neural networks described in Figures 5 and 6.

[0075] [Fig.7] represents, on a flowchart, the steps of implementing a method for detecting and locating falsification in an image according to an embodiment of the invention. This method corresponds to an exemplary embodiment of step 304 of the method of [Fig.3].

[0076] The method of [Fig.7] begins at step 701 with the extraction of N signatures corresponding to N sub-blocks of an image as described in [Fig.3].

[0077] In step 702, the dominant signature is searched for among the N extracted signatures. The dominant or majority signature is the most present signature among the N extracted signatures. To carry out step 702, the N signatures are compared with each other and grouped into groups of identical signatures. The group containing the largest number of identical signatures corresponds to the majority group.

[0078] The following steps 703 to 707 are executed for each sub-block of the image.

[0079] In step 703, the signature extracted from the sub-block of index i is compared to the dominant signature. If these two signatures are identical, it is deduced in step 704 that the sub-block of index i has not been falsified. Otherwise, the difference between the two signatures is calculated in step 705 and then this difference is compared in step 706 to a predefined detection threshold. If the difference is less than or equal to the threshold, the procedure returns to step 704 to conclude that the sub-block of index i has not been falsified. Otherwise, the procedure proceeds to step 707 to report the sub-block of index i as a falsified sub-block.

[0080] Steps 703 to 707 are iterated for all sub-blocks of the image.

[0081] An optional step 708 makes it possible to generate a falsification map indicating which sub-blocks of the image have been tampered with.

[0082] Thus, the method of [Fig.7] makes it possible to precisely locate the areas of the image which have been falsified, unlike global methods which integrate a single signature in an image.

[0083] A particular embodiment of the invention is now described which makes it possible to make the proposed technique robust to cutting or "cropping" operations on an image.

[0084] [Fig.8a] illustrates an example of a signature or tattoo represented in the form of a matrix. According to the invention, this tattoo is integrated into each block of an image decomposed into N blocks.

[0085] When the image undergoes, during its transmission, a cutting or cropping operation, the extraction of the signatures for each block is distorted as illustrated in [Fig.8b] which represents a signature extracted for a block of an image after it has undergone cropping. We see that the extracted signature then no longer corresponds to the signature integrated in a block because the cutting is no longer consistent with the initial definition grid of the blocks.

[0086] In order to overcome this drawback, it is proposed to add an identifiable pattern to a signature before integrating it into a block.

[0087] An example of such a pattern is given in [Fig.9a], it involves placing bits at 1 in at least 3 zones 901,902,903,904 corresponding to corners of the matrix. The patterns thus created in the corners of the matrix must be identical.

[0088] With reference to [Fig.2], before the integration 205 of the signature into a block of the image, the pattern described in [Fig.9a] is added to the signature received in step 203, the whole being represented in matrix form. The size of the matrix is ​​less than or equal to that of a block of the image and is such that the ratio between the size of a block and the size of the matrix is ​​fixed.

[0089] When extracting the signatures, an additional step 305 (with reference to [Fig.3]) is executed to recombine the extracted signatures using the pattern M to compensate for the image cutting operation.

[0090] For this, the elements of the pattern M are detected in the extracted signature (for example that of [Fig.8b]), then we deduce the offset between the initial position of the pattern M during its integration in a block and the position of this pattern in the extracted signature. A vertical offset and a horizontal offset can be estimated. Then these offsets are applied to the pixels of the matrix signature to obtain the decoded signature.

[0091] [Fig.9b] shows another example of a possible M pattern consisting of four elements of the same geometric shape (e.g. crosses) located at the four corners of a square.

[0092] Generally, the pattern M consists of at least three elements arranged in three fixed positions, for example three corners of a square inscribed in the signature matrix. The elements of the pattern consist of values ​​of 1. Advantageously, they are placed on the edges or corners of the matrix so as to avoid areas where the useful bits of the signature are located.

[0093] The steps of the invention may be implemented as a computer program comprising instructions for its execution. The computer program may be recorded on a recording medium readable by a processor.

[0094] Reference to a computer program that, when executed, performs any of the functions described above, is not limited to an application program running on a single host computer. Rather, the terms computer program and software are used herein in a general sense to refer to any type of computer code (e.g., application software, firmware, microcode, or any other form of computer instruction) that can be used to program one or more processors to implement aspects of the techniques described herein. The computing means or resources may notably be distributed (“Cloud computing”), possibly according to peer-to-peer technologies.The software code may be executed on any suitable processor (e.g., a microprocessor) or processor core or set of processors, whether provided in a single computing device or distributed among several computing devices (e.g., as may be accessible in the device environment). The executable code of each program enabling the programmable device to implement the processes according to the invention may be stored, for example, in the hard disk or in read-only memory. Generally, the program(s) may be loaded into one of the storage means of the device before being executed. The central unit may control and direct the execution of the instructions or portions of software code of the program(s) according to the invention, instructions which are stored in the hard disk or in read-only memory or in the other aforementioned storage elements. References

[0095] [1] « ReDMark : Framework for Residual Diffusion Watermarking based on Deep Networks”, M Ahmadi et al

Claims

Claims

1. A computer-implemented method of digitally watermarking an image comprising the steps of: - Receiving (201) a digital image, - Decomposing (202) the digital image into a plurality of sub-blocks of the same predefined dimensions, - Generating (203) a digital binary signature, - For each sub-block, i. Transforming the sub-block into an intermediate space via a predetermined transform, ii. Combining the signature with the sub-block in the intermediate space to obtain a signed sub-block in the intermediate space, iii. Applying to the signed sub-block a transform inverse to the predetermined transform, - Concatening (206) all of the signed sub-blocks to form a signed image.

2. A digital watermarking method according to claim 1 further comprising the steps of: - Adding (204) to the digital binary signature a binary pattern, - Transforming the binary signature into a matrix of dimensions smaller than the dimensions of a sub-block, - The binary pattern being inserted into the binary signature so as to be positioned on predefined positions of the matrix identical for each sub-block

3. A digital watermarking method according to claim 2 wherein the binary pattern is positioned on at least three areas of the matrix corresponding to three corners of a square, for example three corners of the matrix.

4. A digital watermarking method according to any preceding claim wherein the step of combining the signature with a sub-block is performed by means of a first artificial intelligence engine trained to minimize the visual impact of a signature on a signed image.

5. A computer-implemented method of extracting a digital watermark comprising the steps of: - Receiving (301) a digital image signed using the digital watermarking method of claim 1, - Decomposing (302) the digital image into a plurality of sub-blocks of the same dimensions used to generate the signed digital image, - For each sub-block, i. Extracting (303) the binary signature watermarked in the sub-block

6. A method of extracting a digital tattoo according to claim 5 wherein the step of extracting the binary signature of a sub-block is performed by means of a second artificial intelligence engine trained to minimize a gap between the originally generated binary signature and the binary signature extracted from a sub-block.

7. A method of extracting a digital tattoo according to claim 6 wherein: - the step of combining the signature with a sub-block of the digital tattoo method is performed by means of a first artificial intelligence engine trained to minimize the visual impact of a signature on a signed image, - the first artificial intelligence engine and the second artificial intelligence engine are trained jointly.

8. Method for extracting a digital tattoo according to any one of claims 5 to 7 further comprising the steps of: - Detecting (305), for at least one sub-block, a binary pattern inserted in the extracted signature, - Compare (305) the positions of the detected binary pattern to a set of expected positions and if they differ from the expected positions, deduce that the received digital image has undergone a cutting operation, - Reconstruct (305) the binary signature associated with the image before the cutting operation from the positions of the binary pattern by applying to the digital signature an operation inverse to the cutting operation

9. A computer-implemented method of detecting tampering with a digitally watermarked image, the method comprising the steps of: - Executing (701) the method of extracting a digital tattoo for a received image according to any one of claims 5 to 8, - Search (702), for all the sub-blocks of the image, the extracted signature which is the majority among this set, - for each sub-block, i. compare (703) the extracted signature for the sub-block to the majority signature and, ii. deduce whether the image sub-block has been tampered with or not

10. A method according to any preceding claim wherein the digital image is an image acquired by a satellite.