Method and device for tattooing semantic image segmentation model developed by machine learning

Abstract

Method and device for tattooing a semantic segmentation model of images developed by machine learning This method comprises an initial phase (40) of machine learning making it possible to obtain, from an input digital image comprising an overlay of tattoo thumbnail(s), a predetermined tattoo output image associated with the legitimate owner of the semantic segmentation model, then steps of: applying a complementary phase (42) of machine learning of at least part of the parameters of the semantic segmentation model to obtain at least one refined semantic segmentation model;iterative generation (44) of at least one modified tattoo thumbnail, using a semantic segmentation model chosen from the initial semantic segmentation model and the refined semantic segmentation models, by machine learning using said tattoo output image as a target, embedding the modified tattoo thumbnail in an input image of the chosen semantic segmentation model, providing as output an output image similar to the tattoo output image. Figure for abstract: Figure 3;

Description

Title of the invention: Method and device for tattooing a semantic segmentation model of images developed by machine learning

[0001] The present invention relates to a method for watermarking a semantic image segmentation model developed by machine learning, and an associated image watermarking device. It also relates to an associated computer program.

[0002] The invention lies in the field of cybersecurity, and more particularly in the protection of semantic image segmentation models developed by machine learning.

[0003] Digital watermarking methods are known, allowing watermarks to be inserted into digital data, in particular to authenticate the legitimate owner of this digital data. However, conventional methods are intended to watermark multimedia content, for example digital images or videos.

[0004] Methods implementing machine learning, for example deep learning of parameters defining a deep neural network trained to perform a given task, have developed recently, with applications in many fields, for example natural language processing or computer vision, and more particularly semantic classification or semantic segmentation of images.

[0005] A semantic image segmentation model is, for example, a multi-layer neural network, with an architecture chosen in terms of the number of layers, number of neurons per layer and activation functions used, defined by parameters used in the various calculations, the values ​​of the parameters being adjusted by machine learning, during a learning phase on digital training images forming part of a training database. In the learning database, each digital training image is associated with an expected semantic result, or ground truth, which serves as a target during learning. The learning phase is also called the training phase.

[0006] Developing the model and learning the parameter values ​​are long and costly tasks, requiring the intervention of expert engineers, or even the establishment of a research program.

[0007] In particular, the training phase requires a large amount of training data, the training data being data for which the association between inputs and outputs (expected semantic result) is previously entered. Developing a training database is long and costly. In addition, adjusting the model parameters, which are very large in number, for a specific semantic segmentation task is also very long and consumes computational resources.

[0008] Thus, a semantic image segmentation model, trained by machine learning for given tasks, is expensive to develop, and therefore the illegitimate appropriation of such semantic image analysis models by malicious third parties becomes an issue.

[0009] There is therefore a need to develop tools to identify the legitimate owner of such a semantic segmentation model by machine learning.

[0010] Patent application FR 22 14406 describes a method for watermarking a semantic segmentation model of images consisting of adjusting the parameters of the parameterized operations implemented by the model in such a way that, when one or more predefined watermark thumbnails are embedded, at embedding positions, in an input digital image, the semantic segmentation model provides as output a predetermined watermark output image, making it possible to associate said semantic segmentation model with a legitimate owner. For example, the watermark output image includes a logo of the legitimate owner, which is easily identifiable.

[0011] The semantic segmentation model watermarking method described in patent application FR 22 14406 works advantageously with any type of semantic segmentation model. However, a malicious competitor seeking to appropriate the model could apply a partial re-learning of the parameter values, by a method of refining the parameters of such a semantic segmentation model called "fine tuning". In this case, there is a high risk that the semantic segmentation model thus refined will not retain the watermark, i.e. will not make it possible to obtain an output result close to the predetermined watermark output image when the watermark thumbnail(s) are embedded in an input image.Since fine-tuning is easier and less expensive than full training to learn the parameters of such a semantic segmentation model, a malicious competitor could use it to appropriate the model.

[0012] Thus, there is a need to improve the robustness of the watermarking of the semantic image segmentation model developed by machine learning, in particular to avoid the erasure of the watermarking by the methods of refining the parameters of said model.

[0013] To this end, the invention proposes, according to one aspect, a method for watermarking a semantic segmentation model of images developed by machine learning, the semantic segmentation model implementing parameterized operations, the method comprising an initial phase of machine learning of the values ​​of the parameters of said parameterized operations making it possible to obtain, from an input digital image comprising an inlay of at least one watermark thumbnail, a predetermined watermark output image, said predetermined watermark output image making it possible to associate said semantic segmentation model with a legitimate owner.

[0014] This method further comprises steps, implemented by a processor, of: - applying a complementary machine learning phase of at least part of the parameters of the semantic image segmentation model making it possible to obtain at least one refined semantic segmentation model; - iterative generation of at least one modified tattoo thumbnail, using a semantic segmentation model chosen from said semantic segmentation model obtained during the initial learning phase and the refined semantic segmentation model(s) obtained during the complementary machine learning phase, and using said tattoo output image as output target, such that when the at least one modified tattoo thumbnail is embedded on an input image provided as input to the chosen semantic segmentation model, said chosen semantic segmentation model provides as output an output image similar to said predetermined tattoo output image.

[0015] Advantageously, the proposed image semantic segmentation model watermarking method uses a generation of modified watermark thumbnails to reinforce the robustness of the watermarking in the event of possible modifications to the initial semantic segmentation model by refining at least part of its parameters.

[0016] The method for tattooing a semantic image segmentation model according to the invention may also have one or more of the characteristics below, taken independently or in any technically conceivable combination.

[0017] The or each tattoo thumbnail being formed by a plurality of points having associated values, the iterative generation of at least one modified tattoo thumbnail implements a gradient descent algorithm to calculate noise values, to be added or subtracted from each point value of the tattoo thumbnail.

[0018] The semantic segmentation model is a multi-layer neural network, and the complementary machine learning phase implements a method for refining the parameter values ​​of the last layer of said neural network.

[0019] The semantic segmentation model is a multi-layer neural network, and the complementary machine learning phase implements at least one method for refining the parameter values ​​of the last layer and one method for refining the parameter values ​​of each of the layers of said neural network.

[0020] The complementary machine learning phase further implements a complete re-learning of the parameters of the last layer of the neural network and / or a complete re-learning of the parameters of all layers of the neural network.

[0021] The iterative generation of at least one modified tattoo image comprising at least one modified tattoo thumbnail is implemented for each refined semantic segmentation model, the method further comprising a step of selecting a modified tattoo image associated with one of said refined semantic segmentation models as a function of a performance criterion.

[0022] The performance criterion is based on a similarity score between an output image obtained at the output of each refined semantic segmentation model receiving as input an input image comprising an overlay of said at least one modified tattoo thumbnail and said predetermined tattoo output image.

[0023] The similarity score is the Dice score.

[0024] According to another aspect, the invention relates to a pattern tattooing device semantic segmentation of images developed by machine learning, the semantic segmentation model implementing parameterized operations, the device comprising a module configured to implement an initial phase of machine learning of the values ​​of the parameters of said parameterized operations making it possible to obtain, from an input digital image comprising an inlay of at least one tattoo thumbnail, a predetermined tattoo output image, said predetermined tattoo output image making it possible to associate said semantic segmentation model with a legitimate owner. This device comprises a processor configured to implement: - a module for applying a complementary machine learning phase of at least part of the parameters of the semantic image segmentation model making it possible to obtain at least one refined semantic segmentation model; - a module for iterative generation of at least one modified tattoo thumbnail, using a chosen semantic segmentation model among said semantic segmentation model obtained during the initial learning phase and the refined semantic segmentation model(s) obtained during the complementary machine learning phase, and using said tattoo output image as output target, such that when the at least one modified tattoo thumbnail is embedded on an input image provided as input to the chosen semantic segmentation model, said chosen semantic segmentation model provides as output an output image similar to said predetermined tattoo output image.

[0025] This device is configured to implement a method of watermarking a semantic segmentation model of images as briefly described above, according to all embodiments.

[0026] According to another aspect, the invention relates to an information recording medium, on which are stored software instructions for the execution of a method of watermarking a semantic segmentation model of images as briefly described above, when these instructions are executed by a programmable electronic device.

[0027] According to another aspect, the invention relates to a computer program comprising software instructions which, when implemented by a programmable electronic device, implement a method of watermarking a semantic segmentation model of images as briefly described above.

[0028] Other characteristics and advantages of the invention will emerge from the description given below, for information purposes only and in no way limiting, with reference to the appended figures, among which:

[0029] [Fig-1] [Fig.l] illustrates a segmentation model tattooing device semantics according to an embodiment;

[0030] [Fig.2] [Fig.2] schematically illustrates examples of implementation respectively of a watermarked semantic segmentation model and a watermarked and refined semantic segmentation model on distinct input images;

[0031] [Fig.3] [Fig.3] is a flowchart of the main steps of a tattooing process of semantic segmentation model according to a first embodiment;

[0032] [Fig.4] [Fig.4] is a flowchart of the main steps of a tattooing process of semantic segmentation model according to a second embodiment.

[0033] The invention applies to the tattooing of semantic segmentation models of images by machine learning. Advantageously, the proposed method is "agnostic" of the segmentation model and the machine learning algorithm used, for example of the architecture of the neural network used.

[0034] In other words, the semantic segmentation model is seen as a black box. The method applies to any type of machine learning, for example by deep learning, using convolutional neural networks (or CNNs), such as ResNet or U-Net.

[0035] [Fig.l] represents a device 2 for tattooing a semantic segmentation model of digital images.

[0036] Generally speaking, in the remainder of the description, the term image designates a digital image, represented in the form of a matrix of points, having a given number of rows and columns, or in other words of a given size, each point (or pixel) having an associated digital value.

[0037] The device 2 makes it possible to obtain a machine learning-trained semantic segmentation model 4 which is watermarked, as well as a watermark image 6 comprising at least one watermark thumbnail 16 and a modified watermark image 8 comprising at least one modified watermark thumbnail 18.

[0038] Each tattoo thumbnail 16, 18 is also a digital image, of predetermined size, preferably smaller than the size of the respective tattoo images.

[0039] Each tattoo thumbnail comprises a graphic representation of selected shape and colors. Each tattoo thumbnail is positioned at a predetermined embedding position in the corresponding tattoo image.

[0040] In the illustrated example, the tattoo image 6 comprises two identical tattoo thumbnails 16, but alternatively, a different number of thumbnails, distinct or identical, can be used. According to a variant, only one tattoo thumbnail is used.

[0041] The tattoo images 6, 8 preferably have the size of the images provided as input to the semantic segmentation model.

[0042] The modified tattoo image 8 comprises the same number of modified tattoo thumbnails 18, of the same size, positioned in a similar manner. The modified tattoo thumbnails 18 are identical or distinct.

[0043] The semantic segmentation model 4 is, in one embodiment, a neural network type model performing parameterized operations, the parameters of which are learned during an initial learning phase.

[0044] The neural network comprises an ordered succession of multi-layer neuron layers, each of which takes its inputs from the outputs of the previous layer.

[0045] More precisely, each layer comprises neurons taking their inputs from the outputs of the neurons of the previous layer, or from the input variables for the first layer.

[0046] Alternatively, more complex neural network structures can be envisaged with a layer that can be connected to a layer further away than the immediately preceding layer.

[0047] Each neuron is also associated with an operation, i.e. a type of processing, to be carried out by said neuron within the corresponding processing layer.

[0048] Each layer is connected to the other layers by a plurality of synapses. A synaptic weight is associated with each synapse, and each synapse forms a connection between two neurons. It is often a real number, which takes both positive and negative values. In some cases, the synaptic weight is a complex number.

[0049] Each neuron is capable of performing a weighted sum of the value(s) received from the neurons of the previous layer, each value then being multiplied by the respective synaptic weight of each synapse, or link, between said neuron and the neurons of the previous layer, then applying an activation function, typically a non-linear function, to said weighted sum, and delivering at the output of said neuron, in particular to the neurons of the following layer which are connected to it, the value resulting from the application of the activation function. The activation function makes it possible to introduce a non-linearity into the processing carried out by each neuron. The sigmoid function, the hyperbolic tangent function, the Heaviside function are examples of activation functions.

[0050] As an optional addition, each neuron is also capable of applying, in addition, a multiplicative factor, also called bias, to the output of the activation function, and the value delivered at the output of said neuron is then the product of the bias value and the value from the activation function.

[0051] A convolutional neural network is also sometimes called a convolutional neural network or by the acronym CNN which refers to the English term “Convolutional Neural Networks”.

[0052] In a convolutional neural network, each neuron in the same layer has exactly the same connection pattern as its neighboring neurons, but at different input positions. The connection pattern is called a convolution kernel or, more often, a "kernel" in reference to the corresponding English term.

[0053] A fully connected layer of neurons is a layer in which the neurons of said layer are each connected to all the neurons of the previous layer.

[0054] Such a type of layer is more often referred to as “fully connected” and sometimes referred to as “dense layer”.

[0055] The learning of the parameters is carried out so that:

[0056] - for input images forming part of a training data set, a segmentation of the input image into areas belonging to these semantic classes among a predetermined set of semantic classes is obtained;

[0057] - for each input image comprising one or more tattoo thumbnails embedded, the semantic segmentation model provides as output a predetermined tattoo output image, including for example a logo or a message representing the legitimate owner of the semantic segmentation model.

[0058] Semantic segmentation therefore makes it possible to classify objects from an unwatermarked input image into predetermined semantic classes.

[0059] For example, in the case of aerial or satellite digital images, segmentation makes it possible to detect areas of vegetation, bodies of water, and fire zones.

[0060] In another application, semantic segmentation makes it possible to locate one or more vehicles in a digital image according to predetermined vehicle types.

[0061] The semantic segmentation model 4 is advantageously watermarked in such a way that predetermined information, in particular information relating to its legitimate owner, is intrinsically inserted into the semantic segmentation model, the watermarking remaining without any harmful effect on the semantic segmentation performance of the model.

[0062] The watermarking device 2 is in one embodiment an electronic computing device, each comprising one or more processors 10, an electronic memory unit 12, and this device being configured to implement a semantic segmentation model watermarking method according to the invention.

[0063] In [Fig.l] a single electronic computing device 2 is shown, but of course, the use of a plurality of computing devices connected to each other is also conceivable.

[0064] The electronic calculation device 2 comprises or is connected to a learning database 14.

[0065] The processor 10, the electronic memory unit 12 and the learning database 14 communicate via an internal communication bus of the electronic computing device 2.

[0066] The device 2 comprises a module 20 implementing the initial phase of learning the parameters of the semantic segmentation model, for example of the neural network, on a first set DI of the learning database, and using the tattoo image 6, comprising at least one tattoo thumbnail.

[0067] The device 2 further comprises a module 22 for applying a complementary machine learning phase of at least part of the parameters of the watermarked semantic segmentation model of images, in order to obtain at least one refined semantic segmentation model from the initial semantic segmentation model, on a second set D2 of data from the learning database.

[0068] In one embodiment, the module 22 applies a method of adjusting (in English “fine tuning”) the parameter values ​​of the last layer of the neural network forming the tattooed semantic segmentation model 6, this type of method being known in English as “Fine-Tune Last Layer”, also designated by the acronym FTLL. The parameters of the other layers of the neural network forming the tattooed semantic segmentation model 6 are left unchanged.

[0069] In another embodiment, the module 22 applies several adjustment methods, making it possible to obtain several refined semantic segmentation models. For example, in addition to the FTLL method, the module 22 also applies one or more of: the method known in English as “Fine-Tune Ail Layer”, also designated by the acronym FTAL, consisting of adjusting the parameters of all the layers of the neural network; a complete re-learning of the parameters of the last layer of the neural network, known in English as “Retrain Last Layer” (or RTLL); a complete re-learning of the parameters of all the layers of the neural network, known in English as “Re-train Ail Layers” (or RTAL).

[0070] The device also comprises a module 24 for iterative generation of at least one modified tattoo thumbnail forming a modified tattoo image 8.

[0071] This module 24 implements for example an adversarial attack method, consisting of calculating a modified input data in order to obtain a modified output of a model trained by machine learning, in the present case of a semantic segmentation model chosen from the semantic segmentation model 6 obtained during the initial learning phase and the refined semantic segmentation model(s) obtained by the module 22 during the complementary learning phase, and using the predetermined watermark output image as output target.

[0072] Thus, when the at least one modified tattoo thumbnail is embedded on an input image provided as input to the chosen semantic segmentation model, this chosen semantic segmentation model provides as output an output image similar to the predetermined tattoo output image.

[0073] In one embodiment, the generation module 24 is applied with each calculated semantic segmentation model, and a modified tattoo image comprising one or more modified tattoo thumbnails is obtained for with each calculated semantic segmentation model. In this embodiment, the device 2 comprises a module 26 for selecting a modified tattoo image comprising one or more modified thumbnails, based on a performance criterion on all the calculated semantic segmentation models.

[0074] The modified tattoo image and the modified tattoo thumbnails are stored in connection with the semantic segmentation model 4.

[0075] In one embodiment, the modules 20, 22, 24, 26 are implemented in the form of software instructions forming a computer program, which, when executed by a programmable electronic device, implements a method of watermarking a semantic segmentation model of images as described.

[0076] In a variant not shown, the modules 20, 22, 24, 26 are each produced in the form of programmable logic components, such as FPGAs (Field Programmable Gate Array), microprocessors, GPGPU components (General-purpose graphics processing), or even dedicated integrated circuits, such as ASICs (Application Specific Integrated Circuit).

[0077] The computer program comprising software instructions is further capable of being recorded on a non-transitory, computer-readable information recording medium. This computer-readable medium is, for example, a medium capable of storing electronic instructions and of being coupled to a bus of a computer system. For example, this medium is an optical disk, a magneto-optical disk, a ROM memory, a RAM memory, any type of non-volatile memory (for example EPROM, EEPROM, FLASH, NVRAM), a magnetic card or an optical card.

[0078] [Fig.2] schematically illustrates examples of results of using semantic image segmentation models.

[0079] Thus, as illustrated in [Fig.2] schematically, starting from an input image h and applying the watermarked semantic segmentation model, obtained at the output of the initial learning phase, referenced Mi in [Fig.2], a segmentation image Si is obtained, comprising a zone Zi representative of the “car” class and a zone Z2 of “image background”. This is the expected operation of the semantic segmentation model trained by machine learning.

[0080] When the watermark image 6 is combined with the input image Ib, i.e., when the watermark thumbnails 16 are embedded in the input image at the corresponding embedding positions to obtain an input image 1½ that triggers the watermark revelation of the semantic segmentation model, a predetermined watermark output image IT is obtained by applying the segmentation model semantics Mb In the example of [Fig.2], each tattoo vignette 16 represents a padlock on a plain background. Of course, multiple variations of shape and color are possible.

[0081] For example, the predetermined IT tattoo output image represents a logo or message representative of the legitimate owner of the semantic segmentation model. Of course, any other type of predetermined tattoo output image is usable.

[0082] When the same input image including the embedded tattoo thumbnails 16 is presented as input to a refined semantic segmentation model, i.e. obtained by adjusting the values ​​of the parameters of one or more layers of the semantic segmentation model M1, referenced M2, the output image Is is scrambled and different from the tattoo output image IT, in particular the “LOGO” is not visible.

[0083] Finally, when an input image I*id formed by the input image h and comprising an overlay of the modified tattoo thumbnails 18 obtained by the module 24 for generating at least one modified tattoo thumbnail, is provided as input to the refined semantic segmentation model M2, the output image IT' is similar, according to a chosen similarity criterion, to the predetermined output tattoo image.

[0084] For example, the similarity criterion implements a distance between the output image IT' and the predetermined watermark output image IT. In one embodiment, the distance is calculated by the Dice similarity score (or Dice index). The maximum Dice similarity score is equal to 1, so the closer the calculated similarity score is to 1, the more similar the output image IT' and the predetermined watermark output image IT are.

[0085] In the schematic example of [Fig.2], the output image IT' is noisy, but the "LOGO" is visible. Indeed, even in cases where the watermark output image contains noise, the choice of a predetermined, intelligible and structured watermark output image facilitates the validation of the presence of the watermark by a human operator.

[0086] [Fig.3] is a flowchart of the main steps of a first embodiment of the semantic segmentation model tattooing method.

[0087] The method comprises the implementation of an initial phase 40 of machine learning of the parameters of a semantic segmentation model M1 chosen for a given application, making it possible to obtain a watermarked semantic segmentation model. The initial phase 40 takes as input a first set DI of data from the learning database, containing input images and output images associated, containing the expected semantic class segmentation, and the predetermined watermark output image.

[0088] Preferably, the method for generating a tattooed semantic segmentation model as described in patent application FR 22 14406 is applied. The tattoo image comprising one or more tattoo thumbnails is also generated and stored.

[0089] The semantic segmentation model Ml is for example a parameterized neural network, for example a CNN or RNN neural network.

[0090] At the end of step 40, the watermarked semantic segmentation model is obtained, making it possible to obtain, from an input digital image comprising an inlay of the at least one watermark thumbnail, at the planned inlay positions, the predetermined watermark output image.

[0091] The method further comprises a complementary phase 42 of machine learning of a part of the parameters of the semantic image segmentation model making it possible to obtain a refined semantic segmentation model from the semantic segmentation model initially obtained.

[0092] In this embodiment, the FTLL (for “Fine Tuning Last Layers”) refinement method of the coefficients of the last layer, or of a plurality of last layers, of the neural network of the semantic segmentation model is applied during the complementary phase 42, using a second set D2 of data from the database.

[0093] The method then comprises a step 44 of iterative generation of modified tattoo thumbnail(s). For example, the generation is carried out by implementing an adversarial attack method using the refined semantic segmentation model generated in step 42.

[0094] Adversarial attack methods have been developed to generate modified input data that distorts the output of a previously trained segmentation or classification model.

[0095] In the proposed method, such an adversarial attack method is used to ensure that the watermark inserted in the initial learning phase remains present when the watermarked semantic segmentation model is modified by refinement by a potential adversary. Such a method is similar to machine learning of input data values.

[0096] As already indicated, each tattoo thumbnail is a digital image of predetermined size, each point or pixel of this digital image having an associated value.

[0097] In one embodiment, step 44 implements an iterative learning algorithm, by gradient descent, to adjust noise values ​​to be added or subtracting from the pixel values ​​of each tattoo thumbnail considered, to obtain a modified tattoo thumbnail, the tattoo thumbnails thus modified forming a modified tattoo image 8.

[0098] The modified watermark thumbnail(s) are such that when embedded in an input image, which is provided to the refined semantic segmentation model generated in step 42, the output image is close to the predetermined watermark output image, defined as the target of this learning.

[0099] The number of iterations is selected, for example, based on a stopping criterion. For example, the number of iterations is between 200 and 2000, and more generally the number of iterations is set empirically.

[0100] Alternatively, step 44 of iterative generation of modified tattoo vignette(s) implements a genetic algorithm.

[0101] Thus, each modified tattoo thumbnail 18 has the same geometric shape and the same size as the tattoo thumbnail 16, but comprises different pixel values, obtained by the iterative generation step 44.

[0102] The modified tattoo thumbnail(s) are then stored, in connection with the semantic segmentation model 4. For example, a modified tattoo image, comprising the modified tattoo thumbnail(s) 18, at predetermined positions, is stored.

[0103] The modified tattoo image is for example obtained by embedding the modified tattoo thumbnails, at predetermined positions, on a uniform background of predetermined color.

[0104] It is noted that in this implementation, the modified watermark 18 may visually differ from the watermark 16, or in other words, the added or subtracted noise values ​​need not be such that the introduced modifications are imperceptible to an observer.

[0105] [Fig.4] is a flowchart of the main steps of a second embodiment of the semantic segmentation model tattooing method.

[0106] The method comprises, in this second embodiment, an initial machine learning phase analogous to phase 40 described with reference to [Fig.3], and which bears the same reference number.

[0107] Unlike the first embodiment, in the complementary machine learning phase 50, several refined semantic segmentation models M2i to M2n are generated, for example by applying the FTAL method, the FTLL method.

[0108] Then, during a step 52, for several semantic segmentation models calculated from the set formed by the watermarked semantic segmentation model obtained in the initial learning phase (model M1) and the segmentation models refined semantic data obtained during the complementary learning phase 50 (models M2i to M2n), the iterative generation method described with reference to step 44 is applied.

[0109] For each semantic segmentation model, a modified tattoo image is obtained, comprising one or more modified tattoo thumbnails, called a candidate modified tattoo image.

[0110] Then, a step 54 of selecting, on a performance criterion, the modified tattoo image from among the candidate modified tattoo images is implemented. For example, the performance criterion is a transferability criterion between models. For example, each candidate modified tattoo image is provided as input to each refined semantic segmentation model of the set of refined semantic segmentation models M2[ to M2n, to obtain a corresponding output image, and a similarity score between the output image and the predetermined tattoo output image is calculated. For example, the similarity score is the Dice score.

[0111] Any other similarity evaluation score between two images can be used.

[0112] The modified tattoo image retained, and therefore stored, is the candidate modified tattoo image for which the similarity score is the highest. Advantageously, the modified tattoo image retained is the one which makes it possible to obtain the best tattoo robustness.

[0113] Thus, advantageously, the robustness of the tattoo for all the possibilities of modification of the semantic segmentation model by refinement is improved.

Claims

Claims

1. Method for watermarking a semantic segmentation model of images developed by machine learning, the semantic segmentation model implementing parameterized operations, the method comprising an initial phase (40) of machine learning the values ​​of the parameters of said parameterized operations making it possible to obtain, from an input digital image comprising an overlay of at least one watermark thumbnail (16), a predetermined watermark output image (IT), said predetermined watermark output image making it possible to associate said semantic segmentation model with a legitimate owner, the method being characterized in that it further comprises steps, implemented by a processor (10), of: - applying a complementary phase (42,50) machine learning of at least part of the parameters of the semantic image segmentation model making it possible to obtain at least one refined semantic segmentation model (M2i,...,M2n); - iterative generation (44, 52) of at least one modified tattoo thumbnail (18), using a semantic segmentation model chosen from said semantic segmentation model (Mi) obtained during the initial learning phase and the refined semantic segmentation model(s) (M2, M2i,...,M2n) obtained during the complementary machine learning phase (42, 50), and using said tattoo output image (IT) as output target, such that when the at least one modified tattoo thumbnail (18) is embedded on an input image provided as input to the chosen semantic segmentation model, said chosen semantic segmentation model provides as output an output image (Is) similar to said predetermined tattoo output image (IT).,

2. A method according to claim 1, the or each tattoo thumbnail (16) being formed by a plurality of points having associated values, wherein said iterative generation (44,52) of at least one modified tattoo thumbnail (18) implements a gradient descent algorithm to calculate noise values, to be added or subtracted from each point value of the tattoo thumbnail.

3. Method according to one of claims 1 or 2, in which the semantic segmentation model is a multi-layer neural network, and in which said complementary machine learning phase (42, 50) implements a method of refining the parameter values ​​of the last layer of said neural network.

4. Method according to one of claims 1 or 2, in which the semantic segmentation model is a multi-layer neural network, and in which said complementary machine learning phase (42, 50) implements at least one method of refining the parameter values ​​of the last layer and a method of refining the parameter values ​​of each of the layers of said neural network.

5. Method according to claim 4, wherein said complementary machine learning phase (42, 50) further implements a complete re-learning of the parameters of the last layer of the neural network and / or a complete re-learning of the parameters of all the layers of the neural network.

6. Method according to one of claims 4 or 5, in which the iterative generation (44, 52) of at least one modified tattoo image comprising at least one modified tattoo thumbnail is implemented for each refined semantic segmentation model, the method further comprising a step of selecting (54) a modified tattoo image associated with one of said refined semantic segmentation models as a function of a performance criterion.

7. The method of claim 6, wherein said performance criterion is based on a similarity score between an output image obtained as output from each refined semantic segmentation model receiving as input an input image comprising an overlay of said at least one modified tattoo thumbnail and said predetermined tattoo output image.

8. The method of claim 7, wherein said similarity score is the Dice score.

9. A computer program comprising software instructions which, when executed by a programmable electronic device, implement a pattern watermarking method semantic image segmentation developed by machine learning according to claims 1 to 8.

10. Device for tattooing a semantic segmentation model of images developed by machine learning, the semantic segmentation model implementing parameterized operations, the device comprising a module (20) configured to implement an initial phase of machine learning of the values ​​of the parameters of said parameterized operations making it possible to obtain, from an input digital image comprising an overlay of at least one tattoo thumbnail (16), a predetermined tattoo output image (IT), said predetermined tattoo output image making it possible to associate said semantic segmentation model with a legitimate owner,the device being characterized in that it comprises a processor (10) configured to implement: - a module (22) for applying a complementary machine learning phase of at least part of the parameters of the image semantic segmentation model making it possible to obtain at least one refined semantic segmentation model (M2i,...,M2n); - a module (24) for iterative generation of at least one modified tattoo thumbnail (18), using a semantic segmentation model chosen from said semantic segmentation model (MJ) obtained during the initial learning phase and the refined semantic segmentation model(s) (M2i,...,M2n) obtained during the complementary machine learning phase, and using said output tattoo image (IT) as output target, so that when the at least one modified tattoo thumbnail (18) is embedded on an input image provided as input to the chosen semantic segmentation model,said chosen semantic segmentation model provides as output an output image (Is) similar to said predetermined watermark output image (IT).,

Images