A process for embedding a digital watermark into tokenized data

Abstract

A computer-implemented process for embedding a digital watermark within tokenized data, the computer-implemented process including the steps of generating tokens from an input data set, the tokens being generated using a deterministic encryption scheme, and embedding a digital watermark within the set of generated tokens.

Description

[Background technology]

[0001] 1. Field of the invention The field of the invention relates to computer-implemented processes for embedding digital watermarks in tokenized data, and related systems and devices.

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the exact facsimile reproduction by any party of the patent document or the patent disclosure as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.

[0003] 2. Description of the Prior Art Tokenization involves the replacement of a private identifier, such as an individual's credit card number or social security number, with a token that is generated to fit a user-specified format and has a 1:1 relationship with the original private identifier. This same token is always used in place of the same identifier and is never used for any other identifier.

[0004] Digital watermarking relates to the process of embedding information, called a digital watermark, into digital content while preserving the functionality of the digital content.

[0005] WO2017 / 093736(A1) discloses a process of modifying an original data set by combining data anonymization and digital watermarking. In particular, anonymization of the original data set can be achieved using a tokenization technique, where the tokenized values ​​are generated with regular expressions. However, the regular expressions must be known at the time of extraction of the watermark. Furthermore, the tokenization technique used involves a central vault, which can cause problems for customers with high throughput needs or a requirement to tokenize values ​​consistently at remote locations.

[0006] A need exists for a process in which digital watermarking is applied to tokenized data without knowledge of the regular expressions used to tokenize the data. Additionally, a system is needed that scales to allow any number of data releases to be watermarked individually.

[0007] Reference is made to WO2017 / 093736(A1), the contents of which are incorporated by reference. Summary of the Invention

[0008] An embodiment of the present invention is a computer-implemented process for embedding a digital watermark in tokenized data, comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens.

[0009] The present invention provides a scalable computer-implemented process that can provide multiple watermarked data releases of tokenized private data. The tokenization used is vault-less tokenization, where tokens are generated without the need for a token database or vault. This can be beneficial for solutions that require data releases to be generated at high throughput and have requirements to tokenize values ​​consistently. By providing a solution that uses deterministic tokenization, the process can also achieve lower latency.

[0010] By combining digital watermarking with deterministic tokenization, watermarked tokens can also be shared efficiently around the world without the raw data being transmitted in the clear. In contrast, globally distributed vault-based tokenization requires that the raw data be transmitted to the token vault along with the token, since both must be stored in a centralized vault. However, this transmission of raw identifiers from one jurisdiction to another is often contrary to law or regulation. [Brief description of the drawings]

[0011] Aspects of the present invention will now be described, by way of example, with reference to the following drawings, each of which illustrates features of the present invention.

[0012] [Figure 1] A histogram of token counts within each bin is shown. [Diagram 2] A diagram representing the space of possible inputs is shown. [Diagram 3] FIG. 2 shows a diagram representing a token space with watermarked tokens uniformly distributed throughout the token space. [Figure 4] 1 shows a diagram illustrating an algorithm for mapping an input space to a token space based on two encryption schemes. [Diagram 5] 5A shows a diagram with 53 watermark tokens distributed across a token space of size 400, and another diagram with the token space divided into 53 segments (5B). [Figure 6] A diagram illustrating the process for two example segments, one containing an actual watermark token and one not, is shown. [Figure 7] FIG. 2 shows a diagram illustrating an input space and a token space. [Figure 8] FIG. 2 shows a diagram illustrating the steps for determining the index of a value in an input space. [Figure 9] 1 shows a diagram illustrating encryption of a subspace index. [Figure 10] 1 shows a diagram illustrating how to find the token space segment to which an input maps. [Figure 11] 1 shows a diagram illustrating how a segment may be searched to find a desired token. [Figure 12] FIG. 13 shows an example output with final token ordinal numbers. [Figure 13] FIG. 2 shows a diagram illustrating a hash space divided into equal-width bins. [Figure 14] A diagram illustrating the process of extracting a watermark is shown. [Figure 15] FIG. 1 shows a diagram illustrating the process of extracting a watermark when multiple hash functions are used. [Figure 16] FIG. 1 shows a diagram illustrating a process of extracting a watermark on a set of parallel hash arrays. [Figure 17] A diagram illustrating the parameters of the algorithm is shown. [Figure 18] Three histograms of token counts within each bin are shown for a "pure" watermark (18A), a noisy watermark (18B), and two mixed watermarks. [Figure 19] FIG. 13 shows a diagram illustrating extraction token count requirements as the number of data releases increases. [Figure 20] 1 shows a plot of tokens required as a function of input noise percentage. [Figure 21] FIG. 13 shows a diagram illustrating extraction token count requirements as the number of data releases increases. [Figure 22] 1 shows a plot of tokens required as a function of input noise percentage. [Diagram 23] 13 shows a plot of normalized computation time for watermark embedding. [Figure 24] 1 shows a plot of normalized computation time for watermark extraction. [Diagram 25]We show the final results of watermark extraction performed at a 95% confidence level as the number of tokens processed in the extraction increases. [Figure 26] We present the results of watermark extraction performed at a 99.9% confidence level. [Figure 27] 1 shows the results of watermark extraction where random noise is progressively added. [Figure 28] A plot of the false positive rate for the same experiment is shown. [Figure 29] FIG. 1 shows a diagram illustrating the number of tokens required to achieve authenticity. [Diagram 30] We present experimental results where two data release watermarks are mixed together. [Diagram 31] We present experimental results where three data release watermarks are mixed together. [Diagram 32] FIG. 1 shows a diagram illustrating the process of extracting a watermark when multiple hash functions are used and different bins within each hash function for data release are used. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] An embodiment of the present invention proposes a computer-implemented process that incorporates digital watermarking in addition to deterministic tokenization.

[0014] We may refer to the following terms throughout the description.

[0015] Input Space - the space of all possible inputs from the original dataset that may need to be tokenized. The input space may be described using regular expressions. For example, when tokenizing credit card numbers, a simple input space definition may be "[0-9]{16}" - 16 decimal digits (this example ignores the complexities of not all prefixes being valid, and Luhn digit checks, etc.).

[0016] Token Space - As above, this is the space of all possible tokens that can be returned.

[0017] Tokenized Data − Data where the input values ​​are replaced by tokens.

[0018] Data Release - generally refers to any release of tokenized data to a specific recipient for a specific purpose. Each data release is therefore associated with its own digital watermark. The digital watermark may be a number or other "ID" that is stored in a watermark registry along with metadata. Thus, by extracting the watermark from the release, any metadata associated with the data release may also be obtained. The metadata may include, for example, one or more recipients enabled to receive the data release, the purpose or intended use of the data release, the period for which one or more recipients are legally enabled to retain the data, and with whom the data is enabled to be shared.

[0019] Watermark Tokens - As will become clear in the following description, the hash-based watermarking scheme used works by not returning any tokens that hash to values ​​that fall within the watermark bins; these tokens are called watermark tokens. Thus, the token space consists of watermark tokens (tokens that hash to watermark bins) and non-watermark tokens (tokens that hash to other bins).

[0020] Watermark Inputs - In deterministic tokenization, a 1:1 mapping from all inputs to all tokens must be fixed. Thus, some of these inputs will be mapped to watermark tokens, but we do not want to return these tokens. Thus, the core of the scheme is that we influence the mapping so that watermark tokens are mapped to inputs that we believe are unlikely to occur, and we call these inputs watermark inputs. As an example, if we are tokenizing credit card numbers using the regular expressions described above, we can choose our watermark inputs to be numbers that start with 0000, since those numbers are not used in real credit card numbers and we will not encounter those numbers. The mapping of inputs to tokens is performed on the fly by an algorithm, as described below.

[0021] Watermarking Prior watermarking techniques, such as that disclosed in WO2017 / 093736(A1), allow the generation of these tokens to be controlled such that a pattern is embedded within the tokens. This pattern can be varied for each data release, allowing a unique identifier of the release to be embedded within and throughout the data itself. This identifier can be used as a pointer to any store of metadata about the data release - the intended recipients and purpose of the release, its lineage including any privacy treatments applied to it, the date by which the data must be deleted, etc. This embedded pattern is probabilistic and extractable from a sample of generated tokens rather than depending on any individual token, and thus still extractable from a sufficiently large subset of the data release.

[0022] Rejection sampling based algorithms work by rejecting potential tokens (and generating another token instead) according to some pattern, then the corpus of watermarked data is scanned to reconstruct the patterns and thus learn the watermark. The patterns embedded by the algorithm are based on the hash of the tokens - the hash space is divided into bins, each of which is assigned to a data release, then when watermarking a data release we want to reject any tokens that hash to a value that falls within the hash bin of the current data release (each data release is assigned a different slice of the hash space). Then, as shown in Figure 1, if we scan the watermarked data, hash the tokens and build a histogram of token counts within each bin, we can identify the empty bins and therefore the data release to which the watermark belongs.

[0023] Vault-Based vs. Deterministic (Vaultless) Tokenization The process described above works with vault-based tokenization because we have the opportunity to choose which tokens we assign to an input at the time we tokenize that input. To the extent that we encounter an input with fewer tokens than possible, we do not have to return all tokens, and we can choose to never return those tokens that hash to a watermark bin.

[0024] Thus, rejection sampling has been achieved using a tokenization system that randomly generates tokens that match a required format and stores the generated tokens in a persistent data store (the "token vault"). At the time of generating the tokens, if a candidate value is generated that should be rejected, another candidate value can simply be generated. However, this reliance on a central token vault can cause problems for customers with high throughput needs, or a requirement to tokenize values ​​consistently at remote locations.

[0025] For this reason, it may be preferable to use a tokenization scheme that is algorithmic (e.g., based on a format-preserving encryption cipher) and therefore fully deterministic. Under such a scheme, it is assumed that for every possible input (i.e., matching a defined format), there exists a mapped token - hence, it is not possible to implement rejection sampling watermarking with this system, where if an input is encountered that maps to a token that should be rejected, there is no choice but to release that token. This is due to the fact that it is not possible to choose a different token, as other tokens would map 1:1 to different inputs, which would ultimately break the tokenization scheme. A solution to this problem is presented in the following paragraphs.

[0026] Watermarking Deterministic Tokenization An embodiment of the present invention is a method for making rejection-based watermarking work on top of deterministic tokenization. It uses the observation that the probability of encountering a particular input is often not uniform across the input space, and attempts to assign tokens to those inputs that are least likely to be encountered. This is achieved using a combination of two format-preserving encryption ciphers - one that maps the least commonly encountered inputs to "rejected" or watermark tokens, and another that maps the remaining (most) inputs to other tokens or non-watermark tokens.

[0027] Advantageously, the digital watermark is embedded within the generated token set and not embedded in any metadata or redundant data.

[0028] With deterministic tokenization, as the name implies, we have a pre-determined mapping of which tokens will be returned for each input before tokenization begins. There is a single regular expression defined that represents both the tokens to be returned and the input that will be encountered, and the mapping is defined in terms of this expression - for example, according to expression [AZ], input ordinal 5 (i.e., E) may be mapped to token ordinal 10 (i.e., J).

[0029] At the time of tokenizing the input, we do not have the flexibility to choose any token other than the mapped token, whether or not the token falls within a hash bin that we want to leave empty.

[0030] To embed a perfect watermark, we absolutely need to avoid returning tokens that hash to values ​​that fall within a slice of hash space designated as a bin assigned to the data release. However, the watermarking extraction algorithm is tolerant to the addition of some level of random noise. Such noise will reduce the reported confidence level for a watermark query, which may result in more tokens being scanned before the required confidence is reached, but it does not prevent successful extraction (the relationship between noise and the number of tokens required to reach a confidence threshold is well understood).

[0031] Therefore, to embed a watermark, we require the following two criteria to be met. 1. It requires that there are some possible values ​​that are encountered less frequently than others (the difference in relative frequency will determine the noise level of the final watermark - if there are enough values ​​in the input space that never actually occur in the data, then it can return a perfect watermark, but otherwise there will be some level of noise). 2. We need some way to assign watermark tokens to these inputs (or, equivalently, assign non-watermark tokens to inputs in which they occur).

[0032] When using the Token Vault, these criteria are met - we naturally discover inputs that are present in the data (we are never asked to tokenize inputs that never appear), and we can choose which tokens to assign to inputs at the time of tokenization. To make watermarking work with deterministic tokenization, we also need to design a way to preserve them.

[0033] Vaultless tokenization has several advantages, such as distributed deployment, where a centralized vault cannot often be invoked. In contrast, vault-based tokenization, which is distributed around the world, requires that the raw data be sent to the token vault along with the token, since both must be stored in the centralized vault. However, this transmission of raw identifiers from one jurisdiction to another is often against law or regulation.

[0034] Figure 2 shows a diagram representing the space of possible inputs (each possible input is a small square representing an ordinal number in the total space of 400 possibilities), where some of the inputs (shown in light grey) have been identified as unlikely to be observed and therefore should be mapped to watermark tokens.

[0035] We wish to map these inputs (which we call watermark inputs) to those tokens that hash to values ​​that fall within the data release watermark bins (watermark tokens). The cryptographic hash function (based on a secret "watermarking key") used within the watermarking algorithm will distribute hashes uniformly across the hash space. This implies that hashes that fall within the hash space are from values ​​that are uniformly drawn from the entire value space - i.e., the watermark tokens will be uniformly distributed across the token space, as shown in Figure 3.

[0036] However, the mapping from an input to its tokens is determined by the underlying format-preserving encryption cipher, and is a permutation indistinguishable from random - a hard-wired mapping into the cipher is not possible (at least not without inventing one's own non-standard and necessarily insecure encryption cipher). In order to realize a scheme where a subset of the input space is mapped to a subset of the token space, we must treat the subspace as a separate space with its own separate encryption cipher. To encrypt a value, we follow these steps: 1. Determine which of the distinct subspaces contains the value. 2. Encrypt the value using the cipher for that space, resulting in another index in the same space. 3. Map this output index into the global token space to find the resulting token.

[0037] To implement this solution, we need to solve two problems. 1. Determine which inputs are watermark inputs that should be mapped to watermark tokens 2. Determine which tokens are watermark tokens (i.e., hash to values ​​within the watermark bins)

[0038] FIG. 4 illustrates these steps.

[0039] Thus, each format-preserving encryption cipher uses a secret key. A further secret key - the watermarking key - is used by a cryptographic hash function to prevent an attacker from learning whether a token is a watermark token or not.

[0040] Modeling the distribution of input values To take advantage of the distribution of values ​​that occur in an input dataset, we first need to know what this distribution is - using this information, we can try to figure out which inputs occur rarely or never, and assign watermark tokens to those inputs. An example approach to this is described next.

[0041] Data Description In some situations, there is some a priori knowledge of the structure of the input data that can be used to describe regions of the input space that will never be encountered. For example, not all numbers that conform to the structure of a Social Security number are actual Social Security numbers, since numbers that contain 666 or 900-999 in the first group of digits are never assigned.

[0042] Make assumptions about the data If the data has some external meaning - examples include names, email addresses, salaries - there may be some general rules of thumb about the data that we can use to make educated guesses about where to assign watermark tokens. For example, in English text, the digraph "th" is much more likely to be frequent than "qz", and for many numerical distributions, the probability density function is often low at the high end of the range.

[0043] We can use this to assign watermark tokens - for example, if we are presented with a regular expression of [A-Z][a-z]{1,9}, we can define a range such as (Q-z|J-q|J-x|Q-x)[a-z]{0,8} to capture those inputs that we expect to occur least frequently (or never).

[0044] Benford's law indicates that in many numeric data sets, leading digits are more likely to be small, and the probability that a particular digit is a leading digit drops logarithmically as the digit increases. Benford's law generally applies to data sets with log-normal distributions, so its direct usage is perhaps not general enough, but we can make the generalization that for numeric data that falls within some defined range, the probability density function is often lower at the upper end of the range. This is true for log-normal distributions that span several digits that satisfy Benford's law, as well as normal distributions (e.g., heights), distributions with long tails (e.g., salaries), and monotonically increasing values ​​(e.g., identifiers drawn from database sequences). Thus, simply assigning a watermark token to the upper end of a numeric range can be expected to give good results in many data sets.

[0045] Scanning the Data If nothing is known a priori about the input distribution and the assumptions in the previous section do not hold, it may be possible to infer the input distribution by scanning the data and observing it. Note, however, that this may be a weaker solution than previous techniques, since the configuration must be determined from scanning the current data, but there is no guarantee that future data will have the same distribution. For example, if the data is drawn randomly from within the space with uniform probability, we may be able to find regions in the space that have no values ​​to assign watermark tokens to, but future values ​​may fall within these ranges (by contrast, data that never contains special digits in social security numbers and follows Benford's Law will continue to do so in the future).

[0046] Identifying Watermark Tokens A watermark token is defined as a token that hashes to a value that falls within a watermark bin. Because hash functions are one-way functions, there is no way to take the hash values ​​of watermark bins and find tokens that will hash to them. The only way to find out if a token's hash falls within a bin is to hash it and find it, and trying to brute force the entire token space to find all of the watermark tokens is impractical for all but a very small number of token spaces. Instead, we use the fact that watermark tokens will be uniformly distributed throughout the token space. This means that if we divide the token space into segments, on average, these segments will contain an equal number of watermark tokens - for example, if we divide the space into as many segments as there are watermark tokens, on average, each segment will contain a single watermark token (some segments may contain none, some will contain multiple segments, but on average, each segment will contain a single watermark token). As an example, Figure 5A shows a diagram with a distribution of 53 watermark tokens across a token space of size 400. Figure 5B shows the token space divided into 53 segments (29 of size 8 and 24 of size 7) filled with different patterns to delimit the segments (the watermark tokens are in the segments highlighted with thicker lines). This gives us 37 segments that contain exactly one watermark token, 8 segments that contain two watermark tokens, and 8 segments that contain no watermark tokens.

[0047] Because we have divided the input space into those inputs that we want to map to regular tokens and those that we want to map to watermark tokens, we can define a segment size that gives us as many segments as there are watermark inputs. We then define that each segment contains exactly one specified token that we will assign to a watermark input. We want to specify the true watermark token (i.e., the token that hashes to a watermark bin), but we do not know in advance which token in the segment this will be (we do not even know if the segment actually contains a watermark token). When we need to choose a specified token, we search the segment to find the first token in the segment that is a true watermark token (if there is none, we fall back to returning the last token in the segment).

[0048] When we need to select a specified token, we search for the segment using the following process. 1. Determine the starting point within the segment - we want this to be deterministic (so that we get the same answer for the segment every time), but different for each segment. To do this, we seed a pseudorandom number generator (PRNG) with the segment index and use it to pick one of the indices within the segment. 2. Examine the token at this starting index to see if it is a watermark token (i.e., does it hash to a value that falls within a watermark bin). If so, this becomes the designated token for the segment. 3. If the token is not a watermark token, proceed to verify the next token. Continue this process (loop at the end of the segment) until we find a watermark token or we reach the beginning again. 4. If we reach the starting point again without finding the watermark token, we specify the last token we tested in the segment (i.e., the last token encountered after we traverse all tokens in the segment by cycling). This is why we choose a different starting point for each segment - if we started at the same point for each segment, we would return the same token for segmentation per segment (i.e., the specified token would be regularly spaced in the token space).

[0049] FIG. 6 shows a diagram illustrating the process for two example segments, one containing an actual watermark token and one not.

[0050] Note that there are two ways in which this system can lead to an incomplete watermarked input → watermarked token mapping, and a non-watermarked input → non-watermarked token mapping. 1. If a segment contains multiple actual watermark tokens (tokens that hash to watermark bins), only one of them will be assigned to the watermark input, and the other tokens will be returned to the non-watermark input. This results in noise being added to the watermark. 2. If a segment does not contain a watermark token, the watermark input will be assigned a token that is not actually a watermark token. Note that this is harmless - after all, watermark inputs are likely to be encountered rarely (by definition), and returning a non-watermark token does not introduce any noise.

[0051] As can be seen, the first result weakens the embedded watermark and is more likely to occur when we declare too few watermark inputs. The number of watermark inputs we declare determines the number of named tokens we consider, which is independent of the number of watermark tokens that actually exist. The number of watermark tokens depends on the number of hash bins and the token space size. It is clear that this scenario occurs when the number of watermark inputs is less than the number of watermark tokens.

[0052] The second outcome is harmless as long as its occurrence does not also imply the occurrence of the first outcome - for example, if the number of declared watermark inputs exactly matches the number of actual watermark tokens, then segmentation will give incomplete results, and the fact that some segments produce outcome 2 means that other segments must produce outcome 1.

[0053] The optimal strategy is achieved when the number of watermark inputs exceeds the number of watermark tokens by a sufficient margin such that segments are small enough that the probability of a segment containing multiple watermark tokens is small. However, this should not be achieved by artificially inflating the number of watermark inputs, as this can lead to assigning watermark tokens to inputs that are encountered in practice more rarely than other inputs (thus introducing noise into the watermark through frequent release of watermark tokens).

[0054] Safety features If an attacker could determine that a particular token was a watermark token, the attacker would know that the corresponding input was drawn from within the space of less frequently encountered inputs - an unacceptable information leak. However, because the algorithm uses a secret watermarking key when embedding the watermark (using a cryptographic hash function), this kind of inference is not possible without access to this key, and the attacker cannot learn much more from the token when compared to vault-based solutions.

[0055] An example will now be described to illustrate the complete algorithm.

[0056] Working Example This section runs through the steps of the complete algorithm simultaneously for two scenarios: inputs that fit into the watermark input domain and those that do not. We use the same example scenario as discussed above. Figure 7 shows a diagram illustrating the input space and token space. The left pane shows the input space 71, with a declared watermark input 72. The right pane shows the token space 73, where the distribution of the watermark tokens is also shown in a lighter greyscale colour 74 (note however that this is unknown to the algorithm). Our example will be to tokenise the highlighted non-watermarked input 75 (the input with ordinal 123) and the highlighted watermarked input 76 (the input with ordinal 356).

[0057] Step 1: Determine the index of the value in the input space For each input, we first determine which subspace it is contained in, and then determine its index within that space. Figure 8 shows a diagram illustrating the steps for determining the index of a value in the input space.

[0058] In our example, input 75 (the input with ordinal number 123) falls within the non-watermarked input subspace with index 117, and input 76 (the input with ordinal number 356) falls within the watermarked input subspace with index 9.

[0059] Step 2: Encrypt the subspatial index Here, we encrypt the input in that subspace - we take an index in this space and use a format-preserving encryption method to obtain another index in the same space. Figure 9 shows a diagram illustrating the encryption of a subspace index.

[0060] In our scenario, input 75 has index 117 in a subspace of size 347 - in this example, this is encrypted to index 226. Input 76 has index 9 in a subspace of size 53, which is encrypted to index 23.

[0061] Step 3: Find the token space segments to which these inputs are mapped Now we need to find the token ordinals to which these subspace indexes are mapped. By definition, we have one specified token per segment - since there are 53 watermark inputs in total, this means we define 53 segments. We try to balance the size of these segments as much as possible, so we create 29 segments of size 8, and 24 segments of size 7.

[0062] FIG. 10 shows a diagram illustrating how to find the token space segment to which an input maps.

[0063] Since there is one designated token per segment, the designated token at index 23 is clearly in the 23rd segment. To find the non-watermark token with index 226 (the 226th non-watermark token in the space), we need to skip segments until we have passed 225 other non-watermark tokens. The first 29 segments each contain eight tokens, one of which is the designated token, so if we skip all of these, we have passed 203 non-watermark tokens. The remaining segments contain seven tokens (six of which are non-watermark tokens), so we need to skip three more of these for a total of 221 non-watermark tokens. Thus, we can say that the non-watermark token with index 226 is the fifth non-watermark token in the 33rd segment.

[0064] Step 4: Search the segment to find the desired token Having found the correct segment for each case, we then need to find the associated token within it.

[0065] FIG. 11 shows a diagram illustrating how to search a segment to find a desired token. For input 111, we are searching for a non-watermark token with index 226, which we calculate to be the fifth non-watermark token in the 33rd segment. Our random but deterministic starting point for this segment is token 6, which we test and confirm is not a watermark token. We then test the next token and find that it is a watermark token, and that the specified token is also a watermark token. Therefore, we know that to find the fifth unspecified token, we must scan four more tokens past this token. Wrapping around at the end of the segment, this tells us that the token we need is the third token in the segment.

[0066] For input 112, we need to find the specified token within segment 23. Our starting point for this segment is index 3, and we must forward test each token until we find that the seventh token in the segment (the fifth token we test) is the watermark token and therefore the specified token.

[0067] Step 5: Return the final token ordinal Now, all that remains is to map the segment tokens we have found to their ordinals within the entire token space, as shown in Figure 12, which illustrates the output including the final token ordinals.

[0068] Here finally we see that input 111 with ordinal number 123 is tokenized into a token with ordinal number 256, and input 112 with ordinal number 356 is tokenized into a token with ordinal number 184.

[0069] Although the watermarking scheme has been described when combined with vault-less tokenization, it can also be extended to be combined with vault-based tokenization. With a vault scheme, watermarking is typically an easy problem to solve, since a system can be configured such that no watermarked tokens are output when certain inputs are encountered. However, a vault scheme stops working when more inputs are seen than there are non-watermarked tokens, because in that case the watermarked tokens must be returned. However, the described process also provides a solution that would avoid this problem when combining watermarking with vault-based tokenization.

[0070] Further details regarding the algorithm are now provided.

[0071] Algorithm Building Blocks Hash-Based Watermarking To allow the watermark to be extracted solely from the token (how it was generated is unknown), a pattern is embedded using a hash of the token. Note that while this document discusses the process in terms of "tokens", which are understood to be the output of a consistent tokenization operation, the same watermarking methodology applies to any process that produces an output that contains some pseudorandomness. For example, the output of a numeric blur can be hashed and subjected to the same process.

[0072] The hash space will be divided into equal width bins, and each bin can be assigned to a different data release (hence the number of data releases that can be watermarked is equal to the number of bins). The diagram shown in Figure 13 shows an example with unsigned 32-bit output and 128 (2 7) We illustrate this for a hash function that gives data release watermark bins (0 to 127). (Note that because the number of bins is equal to a power of 2, the hash space is exactly divisible across the bins, so there is no difference in the number of hashes per bin.)

[0073] To embed a watermark in a data release, we reject any token that hashes to a value that falls within a watermark bin of that data release. Thus, the fraction of rejected tokens in this scheme is 1 / N, where N is the number of data release watermark bins.

[0074] The process of extracting a watermark is illustrated in Figure 14. We hash the tokens we encounter and increment a value count for the bin that the hash falls inside (in the figure, a bin is black when it has at least one hash in it). We are left with a single empty bin 141, and the bin index gives us the watermark 141.

[0075] A drawback of this algorithm is that as the number of distinct data releases (bins) increases, so does the number of records required to extract the watermark: to find the watermark, we require that all of the other N-1 bins contain at least one value, which scales poorly with the number of data releases.

[0076] Extending to Multiple Hashes: Hash Arrays Taking inspiration from Bloom filters, we can use multiple hash functions and coordinate them in a hash array structure. To embed a watermark, we reject any token that falls within a watermark bin for any of the hash functions (see below for an alternative way of using this configuration that was tested but ultimately rejected). Then, when extracting the watermark, we find a single bin index that is empty for each of the hash function bins in the array - we find an empty bin for each hash function and get the intersection of these sets, as shown in Figure 15. When this intersection has only a single empty bin 151, we have found the watermark, but at this point each of the histograms in the array itself still has two or more empty bins, as illustrated in the figure (in this figure, the bins filled with a diagonal pattern are empty for the current hash function, but contain hashes - the black bins - for at least one of the other hash functions).

[0077] Note that this gives us a higher token rejection rate (when compared to the same number of bins and a single hash function) since now there are multiple opportunities for a token to be rejected.

[0078] Dynamic Scaling of Data Release: Multihash Arrays The hash array structure allows us to adjust the number of bins and the number of hash functions to balance the number of supported data releases and the token rejection rate. However, the configuration must be determined in advance, which forces us to determine the number of data releases we want to support in advance and create a finite pool of watermarks. However, we can dynamically scale the number of watermarked data releases by creating new instances of the hash array structure (each containing its own hash function key) and assigning a different portion of the data release to each. We call this structure a Multi Hash Array, and it has the following properties: ●First, a multi-hash array contains only a single hash array instance, and therefore supports watermarking of up to N data releases (where N is the number of hash array bins). Once the N watermarks have been assigned, a second hash array instance is created to handle the watermarking of the next N data releases. This new instance is given a different hash function key than the first instance, so that any patterns embedded by either array will appear only as random noise to the other instances. Embedding a watermark only requires a single hash array instance to be used (the instance that contains the data release in which the watermark is embedded), so there is no increase in computational complexity as additional data releases are watermarked. ● The token rejection rate does not increase with the number of supported data releases since it depends only on the number of bins and hash functions in a single hash array. ●Additional hash array instances can be added indefinitely, with each new instance resulting in only a small increase in the number of rows required to extract the watermark (see analysis below).

[0079] When extracting a watermark, we perform a set of parallel hash array extractions - one for each requested instance. As mentioned above, since each instance has its own hash function key, the watermark pattern will only appear in a single instance; other instances will only observe a uniform distribution of hashes (appearing as random noise). This is illustrated in Figure 16, which shows an extraction with two such instances, one of which finds the watermark bin 161, while the other does not see the watermark.

[0080] The partitioning of data release that this structure provides may also provide additional functionality advantages. ●We have already observed that embedding a data release watermark only requires that a hash array instance of the data release be used, but the same is true for functions that “sniff” a stream of data to see if a given data release watermark is present. Extracting an unknown watermark only requires the use of all of the hash array instances. However, this structure also makes it possible to narrow the scope of watermark detection (and therefore the number of rows required) if it is known that the leaked data must be from a subset of the data release (perhaps because the dataset in question was only published in part of the data release). When older data releases have expired and transitioned out of consideration, hash array instances that contain only those data releases may be excluded from extraction.

[0081] Each set of individual hash array keys is generated using a scheme such as HKDF (a simple key derivation function KDF based on the HMAC message authentication code) that allows for the expansion of a single master key into many different derived keys (and the ability to efficiently derive a specific key by providing the "ID" of the key in the input keying material). However, it is also possible to have more fine-grained control of the keys (different master keys per tranche of hash arrays, or even per individual hash array), which can have two security advantages. Permissions: the ability to have finer control over which watermarks can be read and where. Watermark sniffing functions can also be used to verify that watermarks are present and prevent unwatermarked data from passing through; it is an attractive feature that each of these remote execution points has the ability to obtain only the keys for instances that contain the watermarks it expects, and not the ability to simply read arbitrary watermarks. Key Rolling: Where each new hash array has its own key, any single hash array key is used as long as the data release within it is open and active (although older key versions must be kept around for as long as we hope to be able to extract the watermarks generated using them).

[0082] Number of watermarks supported and token rejection rate The number of unique watermarks that can be embedded and the percentage of rejected tokens (token rejection rate) depend on the following configuration parameters of the algorithm: b - the number of data release bins in each hash array instance h - the number of hash functions to use in each hash array instance m - the number of hash array instances currently including the multi-hash array FIG. 17 shows a diagram illustrating the parameters of the algorithm.

[0083] Since one bin is used for each data release, it is clear that the number of supported data releases is given by: N=m×b To embed a watermark in a data release, we reject any token that hashes to a value that falls within the watermark bin of that data release for any of the hash functions. Thus, the token rejection rate is given by

number

[0084] Extracting Watermarks Dealing with Noise: Extraction Algorithm Requirements (The dataset from which we are trying to extract the watermark may not be a pure collection of tokens without the watermark token, it may have been modified by the addition of new synthetic rows, it may be a combination of outputs from several data releases, or the assumptions made about the data shape when assigning the watermark input may not have been completely correct.) Because the watermark is embedded using a secret key, it is not possible to create noise that is excessive in any particular bin without access to this key (directly or indirectly via the watermark extraction function), which we assume is unavailable to anyone trying to erase the watermark.

[0085] Thus, the addition of a synthetic row will appear as a baseline level of noise above the pure watermark, and the combination of multiple data-releasing watermarks will appear as a few bins, each with some percentage below the baseline level. Figure 18 shows three histograms of token counts within each bin for the cases of a "pure" watermark (18A), a noisy watermark (18B), and the two mixed watermarks.

[0086] Therefore, we have the following requirements for our extraction algorithm: It must be possible to extract the watermark even if the hash bin is not exactly empty. Must be able to handle multiple such bins, indicating that the data is a mixture of watermarks from different data releases - We must be able to show the end user some measure of how confident we are that the data contains a particular watermark.

[0087] Any approach that attempts to find empty bins must relax the definition of "empty" to deal with noise, and this can only be done by defining some threshold percentage / count below which a bin is empty and above which it is not, introducing an element of undesirable arbitrariness into the algorithm.

[0088] To avoid this, the extraction method does not attempt to determine "Which data release bins are empty?" Instead, it reformulates the question to ask, for a given data release, "Are we sufficiently certain that the data contains a watermark for this data release bin?", which it answers by calculating how likely it is that we would observe the current hash number in a data release bin if no watermark was present and we were observing only noise in the bin.

[0089] By asking this question for every data release bin, we can return a data release (or multiple data releases) that we are confident was the source of the watermarked data (because the hash numbers observed in those bins are unlikely to be due to random noise).

[0090] Extraction algorithm overview The watermark extraction algorithm is a simple hypothesis test at every bin that determines whether there is enough evidence to reject the null hypothesis (that the data does not contain a watermark for the current bin) and accept an alternative hypothesis. This is done by calculating the probability of obtaining the number of hashes observed in the current bin or less if the data did not contain a watermark for that bin data release. If this probability is less than the significance level implied by the confidence level provided by the user, we reject the null hypothesis that there is no watermark corresponding to the data release bin and instead declare the presence of such a watermark.

[0091] With multiple hash functions, we have multiple instances of the hash array structure. When considering a watermark bin, we sum the hashes for the bin (and the total number of hashes) across all of the hash functions.

[0092] Testing data release bins If the null hypothesis were true, we would expect a hash to fall into a bin just as often as any other bin. However, if a watermark for a bin was present, we would expect to observe a much lower proportion of hashes in that bin compared to other bins.

[0093] The p-value is defined as the probability of obtaining a result at least as good as the observed result when the null hypothesis is true. In our case, this is the probability of obtaining the number of hashes observed in a bin (or less) when the data does not contain the watermark corresponding to the bin.

[0094] To calculate the p-value, we model the hashing of tokens into different bins as a binomial distribution, where in the absence of a watermark, the probability of a token hashing into a given bin is 1 / b.

[0095] Note that when the data has watermarks that correspond to different bins, this probability will be greater than 1 / b. However, in those cases, the actual p-value will always be less than the p-value calculated using 1 / b, so we can always safely reject the null hypothesis when the calculated p-value is less than alpha.

[0096] Therefore, the probability of seeing k hashes in a bin after observing n hashes overall is given by:

number

number

[0097] Confidence level The extraction algorithm receives as user input a confidence level, interpreted as 1-α, where α is a bound on the statistical significance of the test, i.e. the probability that we will erroneously declare the presence of a watermark in data where no such watermark is present (the significance level thus provides a bound on the false discovery rate).

[0098] If the computed p-value for a bin is less than the significance level α, we reject the null hypothesis and declare the presence of a watermark for the bin. Otherwise, we fail to reject the null hypothesis and declare that a watermark for the bin is not found in the dataset. Note: This confidence level should not be interpreted as the probability that the watermark actually exists when we reject the null hypothesis.

[0099] Handling multiple bin comparisons When extracting unknown watermarks from data, we require testing all data release bins, and the decisions from all of these tests must be correct for the process to be successful. Testing multiple bins simultaneously for the presence of a watermark increases the false positive rate of the overall test beyond that of a single test on one bin. To address this issue, we use the Holm-Bonferroni method, which ensures that the overall error of the set of tests remains below the required error limit (while ensuring higher statistical power than the standard Bonferroni correction, which in our context means an improved ability to detect the presence of multiple watermarks simultaneously mixed into the dataset). This is done by reducing the value of α used for each test by a factor of the number of tests performed, which is the number of data release bins to test.

[0100] The process is to first sort the bins by p-value (ascending order) and then use a different significance level (α) for each bin. The bin with the lowest p-value is tested first with a significance level of α / b. If the p-value for this data release is less than the required significance level, then the bin with the next lowest p-value is tested, this time using a significance level of α / (b-1). This process continues until we encounter a bin whose p-value exceeds its corresponding significance level. Thus, the value of α for extracting the data release watermark combinations is:

number

number

number

[0101] How many tokens are needed to detect the watermark? No additional noise When there is an additional noise-free watermark, we have empty bins, so the p-value is just the probability of all n tokens hashing to a bin other than the current bin.

number

number

number

number

number

[0102] Extension to multiple hash array instances and multiple hashes per instance The only difference from the above derivation when multiple hash array instances exist is that there are now m×b possible watermarks instead of just b. Repeating the above steps with a new number of tests gives the number of tokens required as follows:

number

number

[0103] When noise is present When computing the number of tokens needed to extract a watermark, we need bounds on the joint probability: A. Check for more than a certain number of hashes in a bin if a matching watermark exists. B. If no matching watermark exists, check for the number of hashes less than or equal to that number.

[0104] In the noise-free case, we know that the probability of A is zero (in the absence of noise, there will be no hashes in the watermark bin), and therefore we only needed to calculate the probability of seeing a zero hash in a bin when no corresponding watermark is present. However, when noise is present, we need to consider both of these probabilities. In the new definitions below, we use the same notation as before. ●H 0 - Null hypothesis: there is no watermark ●H 1 -Alternative hypothesis that watermarks exist ●p i - expected proportion of hashes in bin i ●n i - the number of hashes in bin i f-Expected proportion of noise

[0105] The expected fraction of hashes in a watermark bin when the noise fraction is f is given by:

number

number

number

[0106] Rows vs. Unique Tokens The modeling above computes the number of tokens needed to extract the watermark, but this is actually the number of unique tokens since there is an implicit assumption that each token hashed and added to an array bin is giving us a new piece of information about the token distribution (and therefore the watermark pattern embedded within it). We can only embed a watermark if we encounter enough diversity in the inputs to allow us to return a range of tokens that touch all bins in the hash space - in the pathological case where we only encounter a single input, we will only return tokens that end up in a single bin.

[0107] It is therefore important to attempt to add each encountered token to the extraction hash bin exactly once. This requires the extraction process to keep track of previously encountered tokens, but it is easy to do this using a Bloom filter. Because we know how many unique tokens we will need to add before we expect to be able to extract the watermark, we can size the filter appropriately - with a limit of 100,000 values ​​(much more than the number of unique tokens we expect to need) and a limit of 10 -9Choosing a false positive rate of 1 gives a filter that requires only about 525kb of memory (note that the extraction process only requires a single instance of this filter, regardless of the number of data releases or algorithm configuration parameters).

[0108] Reporting the strength of embedded watermarks The strength of the embedded watermark can be reported to the user as part of the tokenization job. This strength can be interpreted as the maximum confidence level that extraction can be performed and still accurately obtain the watermarked data release (assuming the output file is not modified in any way), providing an easily understandable summary of whether the processed data contains enough unique tokens to carry the watermark.

[0109] Choosing algorithm parameters Multi-hash arrays: Dynamically increasing M As shown above, the token rejection rate is independent of m, and the average number of tokens required to extract a watermark grows logarithmically with m. Therefore, it makes sense to treat m as a dynamic parameter - just start with a single hash array instance and add additional data release watermarks as they are needed. In this way, the token rejection rate remains constant and the number of tokens required to extract a watermark is always (approximately) the minimum possible for the number of data releases needed, and only increases as new data release watermarks are released.

[0110] Hash Array: Choose B and H As we have seen, the choice of b and h affects all aspects of the algorithm - the number of data releases supported, the token rejection rate, and the number of tokens needed to extract the watermark.

number

number

number

number

number

[0111] The number of tokens required is reduced by rejecting more tokens, implying that we may need multiple different configurations, one for each usage scenario, since the acceptable token rejection rate depends on the usage scenario. For our purposes, we consider two common usage scenarios - tokenization of a bulk dataset, and tokenization of a much smaller set of results from an interactive database query.

[0112] Bulk Data Sets In the traditional use case of tokenizing a bulk dataset, the acceptable token rejection rate is capped at 1%. Therefore, an optimal configuration is reached by choosing the following parameters: ● Bring the token rejection rate as close to the upper limit as possible Minimize the number of data releases, since supporting more data releases comes at the cost of more tokens needed to extract the watermark. As discussed in the section above, we could always dynamically increase m to support more data releases, but we cannot support fewer than b data releases - so rather than lowering performance initially, it makes sense to minimize b, such that performance is optimal initially and gradually degrades as more data releases are required.

[0113] Therefore, the proposed configuration for this scenario is as follows: ●b=1024 ●h=10 This gives a token rejection rate of just over 0.97%. Figure 19 shows a diagram illustrating the extraction token count requirements as the number of data releases increases.

[0114] FIG. 20 shows a plot of the tokens required to extract the watermark with 99.9% confidence for the first hash array instance (supporting up to 1024 data releases) as a function of the input noise percentage.

[0115] Small Data Sets For use cases where we obtain a small sample of a dataset, either as a preview of the dataset or as the result of a selective SQL query, we can tolerate a much higher token rejection rate (because we will have a much smaller number of required tokens), but we will need to be able to embed an extractable watermark into the much smaller dataset. Thus, the proposed configuration for this scenario is: ●b=256 ●h=50

[0116] This gives a token rejection rate of about 17.8%. Figure 21 shows a diagram illustrating the extraction token count requirements as the number of data releases increases.

[0117] FIG. 22 shows a plot of the tokens required to extract the watermark with 99.9% confidence for the first hash array instance (supporting up to 256 data releases) as a function of the input noise percentage.

[0118] Scalability calculation time The largest component of the computational cost of the algorithm is computing the cryptographic hashes. Thus, one might expect that as h increases, the computational cost of the algorithm would increase significantly. However, thanks to Kirsch-Mitzenmacher optimization, it is only necessary to compute a single 64-bit hash, from which multiple 32-bit hashes can then be derived cheaply by multiplication and modular operations without compromising any randomness. Increasing the number of hashes increases the amount of computation required, but in a reasonably modest way (benchmarks show that the computational cost of 50 hashes when embedding a watermark is about 1.7 times the computational cost of 1 hash, not 50 times).

[0119] This optimization may not apply as we increase m, since each hash array instance has its own base key. Thus, the computation time scales linearly with m. However, this is only a concern when extracting the watermark, which requires us to compute hashes across different hash array instances. When embedding a watermark (which is the only operation on the tokenization critical path), we only need to test a single hash array instance (the instance that contains the data release in which we embed the watermark), so the computation time of embedding is independent of m.

[0120] Figures 23 and 24 show the results of benchmark runs that confirm these claims. Figure 23 plots the normalized computation time for watermark embedding, and Figure 24 plots the normalized computation time for watermark extraction. Note that in each graph, the values ​​on the vertical axis are normalized within the range of that particular test, since the absolute values ​​will vary depending on the environment and all we are interested in are trends.

[0121] Memory Extracting the watermark requires that the bins be kept in memory while the data is traversed, incrementing the bin count (we also use a single Bloom filter, which requires a small amount of memory regardless of the values ​​of h, b, and m). Increasing h or m results in more copies of the bin array being in memory, and increasing b results in a larger bin array in each copy. The values ​​of b and h are fixed in our scenario, too small to use much memory per hash array instance. However, the memory required to extract the watermark will increase as the number of embedded watermarks increases (i.e., as m increases). If m reaches a large enough number that memory usage becomes an issue, it may be necessary to make multiple passes through the data and partition the values ​​of m across them.

[0122] Embedding the watermark is not affected since no state needs to be stored in memory.

[0123] result summary This section presents the results of experiments performed on the proposed scheme to verify various aspects of its behavior. In summary, these results show us that: Embedding an extractable watermark requires an expected number of tokens. The false positive match rate returned by an extraction is bounded by the confidence level, and can therefore be adjusted to be arbitrarily low. Extraction tolerates random noise at the cost of requiring more rows when it is noisy, and row requirements rise as modeled - The blending of multiple data release watermarks results in all of the individual watermarks being extracted, even in the presence of additional random noise.

[0124] Note: All results presented in this section were obtained using a similar bulk dataset configuration (b = 1024 and h = 10). However, the observations also apply to datasets with any other configuration, with only the absolute numbers scaled in the manner predicted in the discussion above.

[0125] Extraction result Figure 25 shows the final results of watermark extraction performed at a 95% confidence level as the number of tokens processed in the extraction increases. Each data point is the average of 10,000 experiments and shows the split of results across three mutually exclusive possibilities: no results returned, only correct data releases returned, and correct and incorrect data releases returned (note that there is a theoretical fourth outcome in which only incorrect data releases are returned, but this did not occur).

[0126] Here we clearly see the effect of the confidence level parameter, which is to limit the rate at which false positives are returned (when we reach a token count that our modeling indicates we will need to extract the watermark with 95% confidence - 1017 tokens - we always get the correct data release, but it is achieved by false positive results at a rate that never exceeds the alpha level implied by our confidence).

[0127] The graph above demonstrates the use of a 95% confidence level so that the effect of this parameter can be easily visualized, but in watermark extraction, false positives are an undesirable event - in some circumstances they may result in innocent recipients being blamed for data leakage - so a 5% false positive rate is too high for practical use. However, the confidence level gives us an easily understandable mechanism for reducing the frequency of false positives to a desired level - if the user specifies a 99.9% confidence level, it can be ensured that a false positive will be returned in 0.1% or less of the cases. And since the score for a true positive data release increases very rapidly with the number of tokens, this additional accuracy comes at only a small cost in the number of tokens required to extract the actual watermark. This is shown in Figure 26, where we repeat the above experiment with a 99.9% confidence level.

[0128] Noise Tolerance We repeated the above experiment but gradually added increasing levels of random noise. The graph in Figure 27 shows how the percentage of times we obtain only the correct watermarked data releases varies with the noise level and the number of tokens on which extraction is performed (again, at a confidence level of 95% to allow the effect to be clearly seen). The graph shows that, as expected, the number of tokens required to obtain the correct result at a given confidence level increases.

[0129] Figure 28 shows the false positive incidence for the same experiment (where a false positive is recorded every time at least one erroneous data release is returned, regardless of whether a correct data release was also returned).

[0130] Here we can clearly see that the false positive rate is bounded by the confidence level provided (and that it does not depend on the level of noise).

[0131] How many tokens are needed to achieve authenticity? The above discussion of the watermark extraction algorithm presents a method for estimating the number of tokens required to extract a watermark for a given confidence level and amount of noise added to the input dataset. To test the accuracy of this, we attempted to calculate the number of tokens required to extract the watermark with 95% confidence for various noise levels, and then run experiments on each of these, extracting the watermark 10,000 times on a dataset of this size and recording the results. The results of this experiment are shown in Figure 29 (areas representing the percentage of time when only correct data releases were returned, and areas representing the percentage of times when at least one incorrect data release was returned, whether or not the correct data release was also returned, with the lines using the right hand y-axis indicating the number of rows for which extraction was performed for that noise level).

[0132] From this graph, we can confirm that our estimates for the number of tokens required are accurate – we can see that the extraction success rate tracks well with the expected confidence level (and, as usual, the false positive rate never exceeds the expected 5%).

[0133] Mixed Data Release Watermark 30 and 31 show the results of experiments where the input to the extraction function was a data set containing multiple data release watermarks mixed together, with the addition of increasingly higher levels of random noise. FIG. 30 shows an experiment where two data release watermarks were mixed together, and FIG. 31 shows a mix of three data release watermarks. In all instances, the watermarks were shared evenly across the tokens remaining after random noise was added (e.g., the results for 40% noise include 40% noise / 30% data release 1 / 30% data release 2 in FIG. 30 and 40% noise / 20% data release 1 / 20% data release 2 / 20% data release 3 in FIG. 31). All experiments were performed at a 95% confidence level.

[0134] Here we can see that even with the addition of random noise, the watermarks are correctly separated. As expected, more tokens are needed to extract more watermarks (from the point of view of one of the watermark bins, data carrying other watermarks appears simply as random noise, slowing down the extraction of that watermark in the manner shown in the section above).

[0135] The false positive rate is not shown in the graph above, but is bounded at 5%, as would be expected.

[0136] Alternative Data Release Representations The proposed scheme uses the same bin for data releases in each hash array. However, the alternative scheme uses different bins for each hash function for a data release, and represents the data release as a set of hash function+bin pairs (one for each hash function) - the bins of any given hash function will be used for multiple data releases, but the combination of bins across hash functions will be unique for that data release. As shown in Figure 32, this configuration gives us more data releases than the traditional case, but directly more tokens are needed to extract the watermark, since the hash functions no longer work together (whereas in our original configuration a token hash appearing in any bin of the hash functions was enough to regularize that bin was not considered in all hash functions, this is no longer the case in this alternative scheme, and each hash function works independently). However, this still ends up requiring fewer tokens than a single hash function structure - each token added eliminates a bin in each of the hash functions, and the sum of all of the values ​​across all of the hash functions for the bins of a data release allows us to reach a higher reliability score with fewer tokens.

[0137] According to this configuration, the number of supported watermarks is given by: N=m×b h However, this exponential growth in data releases was ultimately the reason this configuration was rejected. As we have seen, there is a fundamental relationship between the number of tokens needed to extract the watermark and the number of data releases that can be supported for a given token rejection rate - so, somewhat paradoxically, it is actually advantageous to have a scheme in which the growth in the number of data releases is slower, so that we can bring the token rejection rate as close as possible to the acceptable 1% bound, and thus as close as possible to the minimum number of tokens needed.

[0138] Appendix A - Watermarking Deterministic Tokenization This appendix summarizes the main features A to D. Each listed feature may be combined with any other feature A to D. Each optional feature defined below may be combined with any feature and with any other optional feature.

[0139] Key Feature A: A process that incorporates digital watermarking on top of deterministic tokenization. 1. A computer-implemented process for embedding a digital watermark in tokenized data, comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens.

[0140] Key Feature B: A process that incorporates digital watermarking on top of deterministic tokenization, where the digital watermark is embedded probabilistically through the selection of tokens within the generated token set. 1. A computer-implemented process for embedding a digital watermark in tokenized data, comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens; A process in which a digital watermark comprises a pattern that is probabilistically embedded through the selection of tokens within a generated token set.

[0141] Key Feature C: A process that incorporates digital watermarking on top of deterministic tokenization, where the digital watermark can be reconstructed without prior knowledge of the encryption scheme or any other processing used on the input dataset. 1. A computer-implemented process for embedding a digital watermark in tokenized data, comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens; A process by which a digital watermark can be reconstructed without a priori knowledge of the encryption scheme or any other processing used on the input data set.

[0142] Key Feature D: A process that incorporates digital watermarking in addition to deterministic tokenization, where the number of potential watermarked data releases can be dynamically scaled. 1. A computer-implemented process for embedding a digital watermark in tokenized data, comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens; (a) generating a data release, wherein a digital watermark is chosen or selected based on parameters of the data release; A process in which the number of possible watermarked data releases can be dynamically scaled.

[0143] Optional Features The process of embedding digital watermarks ● The digital watermark is embedded within the generated token set and not embedded in any metadata or redundant data. The deterministic encryption scheme is a pseudo-random permutation scheme, such as a pseudo-random permutation scheme that is based on a format-preserving encryption cipher. The process includes the following steps: (a) scanning or observing an input space corresponding to an input data set; (b) determining or identifying inputs to be mapped to watermark tokens; ● The inputs to be mapped to watermark tokens are inferred from knowledge of the input space and what the input space represents. ●The inputs to be mapped to watermark tokens are inputs that are unlikely to occur or be encountered (unlikely to occur refers to using the distribution of probabilities of encountering possible input values ​​so that tokens that we do not want returned are mapped to inputs that we would not expect to encounter). ●The process includes a step of dividing the input space into two separate or disjoint subspaces, a “non-watermarked input subspace” and a “watermarked input subspace”, where inputs in the non-watermarked input subspace are mapped to non-watermarked tokens and inputs in the watermarked input subspace are mapped to watermarked tokens. ● Encryption of each input subspace is accomplished independently. ● The algorithm includes two deterministic encryption schemes, such as FPE (format-preserving encryption). An FPE is configured to encrypt a non-watermarked input subspace, and the other FPE is configured to encrypt a watermarked input subspace. ●The two deterministic encryption schemes are each based on a secret key. ● Digital watermark patterns are embedded within watermark tokens, and the watermark tokens are assigned or determined such that they hash to values ​​that fall within a predefined range of a hash space. ● Because the tokens are hashed using a cryptographic hash function based on a secret watermarking key, an attacker cannot learn in which subspace the token resides. ● The process includes a step of attempting to map the watermark tokens such that the watermark tokens are never or rarely returned to bins of the hash space that contain no values ​​or values ​​close to zero. ● Watermark tokens are dynamically assigned or determined at runtime to avoid the need for a brute-force search of the entire token space. Determining the watermark token includes the following steps: (i) segmenting the token space into segments; (j) assigning a watermark token for each segment; ● Watermark tokens are assigned by scanning a segment to find the first token in the segment that hashes to a value within a given range. ● A starting index for traversing each segment is chosen using a pseudo-random number generator (PRNG) seeded with the segment index. If the starting index is reached without finding a watermark token, the final index of the segment is chosen as the watermark token. ● The size of the segments is chosen to give a high probability that each segment will contain no more than one token that hashes to the watermark range (assuming uniform distribution from the hash function), avoiding the need to return these tokens for inputs other than the intended input.

[0144] Reconstruction or extraction of digital watermarks from data releases ● The digital watermark can be reconstructed without prior knowledge of the deterministic encryption scheme, or any other scheme used on the input data set. ●The digital watermark is reconstructed by (a) hashing the tokens in the watermark data release using one or more hash functions, (b) constructing a histogram of hash frequencies in the hash space, and (c) determining the bins that correspond to the digital watermark. When multiple hash functions are used, the process includes summing the hash numbers for each bin across all of the hash functions. Each hash function contains a different key. The process includes calculating the probability of obtaining an observed hash number or less for a particular bin when the watermarked data release does not contain a digital watermark corresponding to the particular bin. When the calculated probability is lower than a predefined threshold, the process infers that the data release contains a particular digital watermark. ● The process allows reconstruction of the digital watermark for only a subset of the data release. ● The process can handle noise, such as adding or removing rows from the data release. Extraction of the digital watermark is associated with a confidence score related to (a) the likelihood of the watermark's presence in the tokenized data, and (b) the likelihood that an underrepresented bin in the histogram of token hash counts corresponds to a hash bin (how confident we are that the watermark is actually present in the tokenized data and that an underrepresented bin in the histogram of token hash counts is not just a coincidence). - The process includes a step of estimating the number of tokens required to reconstruct the digital watermark. ● The number of tokens is estimated for a given confidence level and the amount of noise present in the tokenized data.

[0145] Digital watermarking depends on the parameters of the data release A computer-implemented process includes generating a data release, and a digital watermark is chosen or selected based on parameters of the data release. ● Digital watermarks are chosen or selected based on the recipient of the data release. • Digital watermarks are chosen or selected based on the intended use of the data release. - Digital watermarks are chosen or selected based on the date by which the data release must be removed. Each data release corresponding to the same tokenized input data set contains a different digital watermark. ● A digital watermark represents a pattern that is a unique ID. ● Digital watermarks are chosen or selected based on the type of tokenized input data (i.e., ID, Social Security number, salary, etc.). ● A digital watermark is chosen or selected based on the deterministic encryption scheme used. The process includes dynamically scaling the number of possible watermarked data releases by updating the hash function. ● The process includes dynamically scaling the number of possible watermarked data releases by hashing the token space with a different hash function.

[0146] Computing Device or System 1. A computing device or system adapted to embed a digital watermark in tokenized data, the device or system comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding a digital watermark within the generated set of tokens.

[0147] Note It should be understood that the above-referenced configurations are merely illustrative of the application of the principles of the present invention. Numerous modifications and alternative configurations can be devised without departing from the spirit and scope of the invention. While the present invention has been illustrated in the drawings and fully described above with specificity and detail in connection with what are presently considered to be the most practical and preferred embodiments of the invention, it will be apparent to those skilled in the art that numerous modifications can be made without departing from the principles and concepts of the invention as described herein.

Claims

1. 1. A computer-implemented process for embedding a digital watermark in tokenized data, said process comprising: (a) generating tokens from an input data set, the tokens being generated using a deterministic encryption scheme; (b) embedding said digital watermark within the generated set of tokens.

2. The process of claim 1 , wherein the digital watermark comprises a pattern that is probabilistically embedded through selection of tokens in the generated token set and is not embedded in metadata or redundant data.

3. The process of claim 1 , wherein the digital watermark can be reconstructed without prior knowledge of the encryption scheme or any other processing used on the input data set.

4. The process of claim 1 , wherein the deterministic encryption scheme is a pseudo-random permutation scheme based on a format-preserving encryption cipher.

5. determining or identifying inputs to be mapped to watermark tokens by scanning or observing an input space corresponding to the input dataset or by inferring these inputs from knowledge of the input space and what it represents.

6. The process of claim 1, wherein the process includes the steps of determining or identifying inputs that are unlikely to appear or be encountered, and mapping these inputs to watermark tokens.

7. 2. The process of claim 1, wherein the process includes dividing an input space into two disjoint subspaces, a "non-watermarked input subspace" and a "watermarked input subspace," wherein the inputs of the non-watermarked input subspace are mapped to non-watermarked tokens and the inputs of the watermarked input subspace are mapped to watermarked tokens, and wherein the encryption of each input subspace is accomplished independently based on a secret key.

8. 2. The process of claim 1, wherein the digital watermark pattern is embedded within watermark tokens, and the watermark tokens are assigned or determined such that they hash to values ​​that fall within a predetermined range of a hash space.

9. 9. The process of claim 8, wherein the hashing of the token is done using a cryptographic hash function based on a secret watermarking key, so that an attacker cannot learn which subspace a token resides in.

10. 6. The process of claim 5, wherein the process includes attempting to map the watermark token such that the watermark token is never or rarely returned to a bin of a hash space that contains no value or a value close to zero.

11. The process of claim 5 , wherein the watermark tokens are dynamically assigned or determined at runtime to avoid the need for a brute-force search of the entire token space.

12. determining the watermark token, (a) segmenting the token space into segments; (b) assigning a watermark token for each segment.

13. 13. The process of claim 12, wherein a watermark token is assigned by scanning a segment to find the first token in the segment that hashes to a value within a predetermined range, and if a starting index is reached without finding a watermark token, the final index of the segment is chosen as the watermark token.

14. 10. The process of claim 1, wherein the process includes generating a watermarked data release, the digital watermark being chosen or selected based on parameters of the data release.

15. 2. The process of claim 1, wherein the digital watermark is reconstructed by (a) hashing the tokens in the watermarked data release using one or more hash functions, (b) constructing a histogram of hash frequencies of a hash space, and (c) determining a bin that corresponds to the digital watermark.

16. 2. The process of claim 1, wherein when multiple hash functions are used, the process includes summing the hash numbers for each bin across all of the hash functions, each hash function including a different key.

17. 16. The process of claim 15, wherein the process includes calculating the probability of obtaining an observed hash number or less in a particular bin when the watermarked data release does not include the digital watermark corresponding to the particular bin.

18. 16. The process of claim 15, wherein the process includes calculating the probability of obtaining an observed hash number in a particular bin in the absence of a watermark for that particular bin, and when the probability is below a predetermined threshold, the process infers that the watermarked data release contains a particular digital watermark corresponding to the particular bin.

19. The process of claim 1 , wherein the process enables reconstruction of the digital watermark for only a subset of watermarked data releases.

20. The process of claim 1 , wherein the process can handle noise, such as adding or removing rows from a watermarked data release.

21. 2. The process of claim 1, wherein the extraction of the digital watermark is associated with a confidence score related to (a) the likelihood of the watermark's presence in the tokenized data and (b) the likelihood that an underrepresented bin in a histogram of token hash counts corresponds to a hash bin.

22. the process includes generating a data release, the digital watermark being chosen or selected based on parameters of the data release; 2. The process of claim 1, wherein the parameters of the data release include recipients of the data release, intended use of the data release, a date by which the data release must be deleted, a type of tokenized input data, or the deterministic encryption scheme used.

23. The process of claim 1 , wherein each data release corresponding to the same tokenized input data set includes a different digital watermark.

24. 9. The process of claim 8, wherein the process includes dynamically scaling the number of possible watermarked data releases by updating the hash function or by hashing the token space with a different hash function.

25. 1. A computing system adapted to embed a digital watermark in tokenized data, the system comprising: a processor, the processor comprising: (a) generating a token from an input data set, the token being generated using a deterministic encryption scheme; (b) embedding said digital watermark within the generated set of tokens.

26. A computing system according to claim 25, programmed to implement the process according to any one of claims 1 to 24.

Images