Process for embedding digital watermarks in generated content

Abstract

A process for embedding digital watermarks in generated content is provided. [Solution] A computer-implemented process for embedding a digital watermark in generated content is provided, which includes steps of generating tokens from input data such as user prompts using a generation AI such as an LLM, and embedding a digital watermark in a set of generated tokens.

Description

[Technical Field]

[0001] The field of the invention relates to computer-implemented methods or processes for embedding digital watermarks in generated content, and in particular for embedding digital watermarks in content generated using generation AI, and related computer-implemented devices or systems.

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The patent owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever. [Background technology]

[0003] Tokenization involves replacing a private identifier, such as an individual's credit card number or social security number, with a token that is generated to fit some user-specified format and has a 1:1 relationship with the original private identifier. This same token is always used in place of the same identifier and never for any other identifier.

[0004] Digital watermarking technology relates to the process of embedding information, called a digital watermark, into digital content while preserving the functionality of the digital content.

[0005] Patent document 1 discloses a process for modifying an original data set by combining data anonymization and digital watermarking techniques. In particular, anonymization of the original data set can be achieved using tokenization techniques, whereby tokenized values ​​are generated using regular expressions. However, the regular expressions must be recognized at the time of watermark extraction. Furthermore, the tokenization techniques used involve a central vault, which requires high processing power or can pose problems for customers who require the tokenization of values ​​to always be performed remotely.

[0006] What is needed is a process whereby a watermark can be applied to tokenized data without knowledge of the regular expressions used to tokenize the data. Additionally, what is needed is a system that provides scaling so that any number of data releases can be watermarked individually.

[0007] Additionally, tokenization techniques have proven effective in processing fixed data structures such as credit card numbers or zip codes. However, tokenizing variable-length input data, such as names or email addresses, is a complex challenge. Therefore, a process is needed that can provide a robust solution for processing variable-length input data.

[0008] Additionally, a process is needed to embed information or watermarks into digital content generated using generative AI, such as images or video frames, audio, or text.

[0009] Reference is made to US Pat. No. 6,299,624, the contents of which are incorporated by reference. [Prior art documents] [Patent documents]

[0010] [Patent Document 1] International Publication No. 2017093736 Summary of the Invention [Means for solving the problem]

[0011] One aspect of the invention is a computer-implemented process for embedding a digital watermark in generated content, comprising the steps of: (a) generating tokens from input data, such as user prompts, using a generation AI, such as an LLM; and (b) embedding a digital watermark in the set of generated tokens.

[0012] Another aspect of the invention is a computing device or system adapted to embed a digital watermark in a generated AI output, the device or system comprising a processor configured to (a) generate tokens from input data such as user prompts using a generated AI such as an LLM, and (b) embed a digital watermark in the set of generated tokens.

[0013] Another aspect of the invention is a social media system configured to detect the use of deepfakes by detecting digital watermarks embedded in social media content, the system comprising a processor configured to (a) hash tokens of any subset of the social media content using one or more hash functions to construct a histogram of hash frequencies in the hash space, and (b) from analyzing the histogram, determine or estimate whether the social media content has a digital watermark embedded therein and the likelihood that the social media content contains a deepfake.

[0014] Another aspect of the invention is an email system configured to detect the use of deepfakes by detecting digital watermarks embedded in email content, the system comprising: (a) hashing the tokens of any subset of the email content using one or more hash functions to construct a histogram of hash frequencies in the hash space; (b) from the analysis of the histogram, determine or estimate whether the email content is watermarked and the likelihood that the email content contains a deepfake; The method includes a processor configured to:

[0015] Another aspect of the invention is an online gaming system configured to detect the use of deepfakes by detecting digital watermarks embedded in online gaming content, the system comprising: (a) hashing the tokens of any subset of the online game content using one or more hash functions to construct a histogram of hash frequencies in the hash space; (b) analyzing the histogram to determine or estimate whether the online game content is watermarked and the likelihood that the online game content contains a deepfake; The method includes a processor configured to:

[0016] Another aspect of the invention is a web browser system configured to detect the use of deepfakes by detecting digital watermarks embedded in website content, the system comprising a processor configured to (a) hash tokens of any subset of the website content using one or more hash functions to construct a histogram of hash frequencies in the hash space, and (b) from analyzing the histogram, determine or estimate whether the website content has a digital watermark embedded therein and the likelihood that the website content contains a deepfake. [Brief explanation of the drawings]

[0017] Aspects of the invention will now be described, by way of example, with reference to the following figures, each of which illustrates features of the invention. [Figure 1] A histogram of token counts in each bin is shown. [Figure 2] A diagram showing the space of input candidates is shown. [Figure 3] 1 shows a diagram representing a token space with watermark tokens uniformly distributed in the token space. [Figure 4] 1 shows a diagram illustrating an algorithm for mapping an input space to a token space based on two encryption methods. [Figure 5]A diagram (5A) shows 53 watermark tokens distributed in a token space of size 400, and another diagram (5B) shows the token space divided into 53 segments. [Figure 6] 1 shows a diagram illustrating the process for a pair of example segments, one containing an actual watermark token and one not. [Figure 7] 1 shows a diagram illustrating the input space and the token space. [Figure 8] 1 shows a diagram illustrating the steps for determining the index of a value in an input space. [Figure 9] 1 shows a diagram illustrating encryption of a subspace index. [Figure 10] Here is a diagram that illustrates how to find the token space segment to which an input is mapped. [Figure 11] Here is a diagram that explains how to search a segment to find the desired token. [Figure 12] A diagram illustrating the output including the final token ordinal is shown. [Figure 13] 1 shows a diagram illustrating a hash space divided into equal-width bins. [Figure 14] 1 shows a diagram illustrating the process of extracting a watermark. [Figure 15] 1 shows a diagram illustrating the process of extracting a watermark when multiple hash functions are used. [Figure 16] 1 shows a diagram illustrating the process of extracting a watermark in a collection of parallel hash arrays. [Figure 17] 1 shows a diagram illustrating the parameters of the algorithm. [Figure 18] Three histograms of token counts in each bin are shown for the cases of a "pure" watermark (18A), a noisy watermark (18B), and two mixed watermarks. [Figure 19]1 shows a diagram illustrating extraction token count requirements as the number of data releases increases. [Figure 20] 1 shows a diagram of the tokens required as a function of the input noise ratio. [Figure 21] 1 shows a diagram illustrating extraction token count requirements as the number of data releases increases. [Figure 22] 1 shows a diagram of the tokens required as a function of the input noise ratio. [Figure 23] 1 shows a diagram of normalized computation time for watermark embedding. [Figure 24] 1 shows a diagram of normalized computation time for watermark extraction. [Figure 25] The final results of watermark extraction performed at a 95% confidence level are shown as the number of tokens processed in the extraction increases. [Figure 26] We present watermark extraction results performed at a 99.9% confidence level. [Figure 27] We show the results of watermark extraction with gradually added random noise. [Figure 28] A diagram depicting the false positive rate for the same experiment is shown. [Figure 29] 1 shows a diagram illustrating the number of tokens required to achieve authenticity. [Figure 30] The results of experiments in which two data-releasing watermarks are mixed are presented. [Figure 31] The results of experiments in which three data-releasing watermarks are mixed are presented. [Figure 32] 1 shows a diagram illustrating the watermark extraction process when multiple hash functions are used and the bins of each hash function for the data release are different. [Figure 33] 1 shows a diagram depicting a watermarking algorithm that maps a two-dimensional input space to a token space based on a deterministic tokenization scheme. [Figure 34] A graph illustrating the character length distribution of common surnames from the 1990 U.S. Census is shown below. [Figure 35]10 shows a graph illustrating the corresponding token distribution when plotted on a chart with linear scale axes. [Figure 36] 1 shows a graph of the length of tokens generated for the first length 2 using a standard deviation of 1 / 3 of the range. [Figure 37] 1 shows a graph of length tokens generated according to the number of tokens with an assigned length for a two-character input. [Figure 38] Diagram of the ciphertext ordinal index according to the region of ciphertext space it lies in DETAILED DESCRIPTION OF THE INVENTION

[0018] An implementation of the invention proposes a computer-implemented process that incorporates digital watermarking in addition to deterministic tokenization.

[0019] Throughout this description, reference may be made to the following terms:

[0020] Input Space - The space of all possible "inputs" from a set of original data that may require tokenization. The input space can be described using regular expressions. For example, when tokenizing credit card numbers, a simple input space definition could be "[0-9]{16}" - 16 decimal digits (this example ignores the complexity that not all prefixes are valid, rune digit checks, etc.).

[0021] Token Space - This is the space of all possible tokens that can be returned, as above.

[0022] Tokenized Data − Data where input values ​​have been replaced with tokens.

[0023] Data Release—generally refers to any release of tokenized data to a specific beneficiary for a specific purpose. Each data release is therefore associated with its own digital watermark. The digital watermark may be a number or other “ID” that is stored in a watermark registry along with metadata. Thus, by extracting the watermark from the release, any metadata associated with the data release may also be obtained. The metadata may include, for example, one or more beneficiaries authorized to receive the data release, the purpose or intended use of the data release, how long one or more beneficiaries are legally permitted to retain the data, and with whom the data is permitted to be shared.

[0024] Watermark Tokens - As will become clear below, the hash-based watermarking scheme used works by returning any tokens whose hash value falls into a watermark bin - these tokens are called watermark tokens. The token space therefore consists of watermark tokens (those that have the hash value of the watermark bin) and non-watermark tokens (those that have the hash value of another bin).

[0025] Watermark Inputs - In deterministic tokenization, a 1:1 mapping from all inputs to all tokens must be fixed. Therefore, while some of these inputs may map to watermark tokens, these tokens are not what we want returned. Therefore, the essence of this scheme is to influence the mapping so that watermark tokens are mapped to inputs that are not thought to be likely to occur - we call these inputs watermark inputs. As an example, if we are tokenizing credit card numbers using the regular expressions described above, we might choose the watermark inputs to be numbers that start with 0000, which are never encountered because they are not used in real credit card numbers. The mapping of inputs to tokens is performed on the fly by an algorithm, as described below.

[0026] watermark Conventional watermarking techniques, such as those disclosed in WO2017093736, allow the generation of these tokens to be controlled so that a pattern is embedded in them. This pattern can change for each data release, allowing a unique identifier for this release to be embedded in the data itself. This identifier can be used as a pointer to an arbitrary store of metadata about the data release - the intended beneficiaries and purpose of the release, the number of rows containing the personal information processing that applies to it, the date by which the data must be deleted, etc. This embedded pattern is probabilistic and can be extracted from a sample of generated tokens rather than relying on individual tokens, and therefore can be extracted from even a sufficiently large subset of a data release.

[0027] Rejection sampling-based algorithms work by rejecting potential tokens (generating alternative tokens instead) according to some pattern, and then scanning a corpus of watermark data so that the pattern can be recovered to reveal the watermark. The pattern embedded by the algorithm is based on the hash values ​​of the tokens—the hash space is divided into bins, each of which is assigned a data release, and when watermarking a data release, it is desirable to reject any tokens whose hash value falls into the hash bin of the most recent data release (each data release is assigned to a different slice of the hash space). Then, by scanning the watermark data and hashing the tokens to build a histogram of token counts in each bin, as shown in Figure 1, it is possible to identify empty bins and therefore the data release to which the watermark belongs.

[0028] Vault-based tokenization vs. deterministic (vaultless) tokenization The process described above works for vault-based tokenization because you choose which tokens to assign to an input at the time you tokenize the input. As long as you encounter fewer inputs than the number of candidate tokens, you do not have to return every token, and you can choose never to return tokens that have a hash value in a watermark bin.

[0029] Therefore, rejection sampling has been achieved by tokenization systems that randomly generate tokens that conform to the required format and store the generated tokens in a persistent data store (a "token vault"). At the time of generating a token, if a candidate value is generated that should be rejected, another one can simply be generated. However, this reliance on a central token vault can create problems for customers with requirements that require high processing power or that require values ​​to be tokenized at a remote location at all times.

[0030] For this reason, it is preferable to use a tokenization scheme that is algorithmic (e.g., based on the format that holds the ciphertext) and therefore fully deterministic. In such a scheme, for every possible input (i.e., one that matches a specified format), there is a mapping token - hence, the system is not able to implement rejection sampling watermarking, and it is presumed that if it encounters an input that is mapped to a token that should be rejected, it has no choice but to release this token. This is due to the fact that it is not possible to select a different token since other tokens are mapped 1:1 to different inputs, which ultimately breaks the tokenization scheme. A solution to this problem is presented in the following paragraphs.

[0031] Deterministic Watermark Tokenization An implementation of the invention is a method for performing rejection-based watermarking on top of deterministic tokenization. Using the observation that the probability of encountering a particular input is often not uniform across the input space, we attempt to assign tokens to be rejected to inputs that are least likely to be encountered. This is achieved using a combination of two formats for holding ciphertext: one that typically maps rarely encountered inputs to "reject" or watermark tokens, and another that maps the remaining (most) inputs to other tokens or non-watermark tokens.

[0032] Advantageously, the watermark is embedded in the set of generated tokens rather than in the metadata or redundant data.

[0033] In deterministic tokenization, as the name suggests, a predetermined mapping of returned tokens for each input is performed before tokenization begins. A single regular expression is specified that represents both the tokens to be returned and the input that will be encountered, and the mapping is specified in terms of this expression - for example, the expression [AZ] might be that input ordinal 5 (i.e., E) maps to token ordinal 10 (i.e., J).

[0034] At the time of tokenizing the input, no flexibility is given to choose tokens other than those that are mapped, regardless of whether they fit into hash bins that are preferably left empty.

[0035] To embed a perfect watermark, it is necessary to avoid even the return of tokens (hereafter referred to as watermark tokens) whose hash values ​​fall within the slice of hash space designated as the bin assigned to the data release. However, watermark extraction algorithms are tolerant to the addition of some level of random noise. Such noise reduces the reported confidence level of a watermark match, which has the effect of requiring many tokens to be scanned until the required confidence is reached, but does not prevent successful extraction (the relationship between noise and the number of tokens required to reach a confidence threshold is well understood).

[0036] Therefore, two criteria must be met for embedding a watermark. 1. There must be some candidate values ​​that are encountered less frequently than others (the difference in relative frequency determines the noise level of the final watermark - if there are enough values ​​in the input space that never actually appear in the data, then a perfect watermark can be returned, otherwise there will be a level of noise). 2. Some technique is needed to assign watermark tokens to these inputs (in other words, assign non-watermark tokens to inputs that "certainly" appear).

[0037] Using a token vault satisfies these criteria - it is possible to organically discover which inputs are present in the data (you are never asked to tokenize inputs that never appear), and to choose which tokens are assigned to inputs at the time of tokenization. For deterministic tokenization to work with watermarking, we also need to design a method to preserve these.

[0038] Vaultless tokenization has several advantages, such as in distributed deployments where it is often not possible to invoke a central vault. By comparison, globally distributed vault-based tokenization requires that raw data be sent to a token vault along with the token, as both must be stored in a central vault. However, sending raw identifiers from one jurisdiction to another in this way is often contrary to legal mandates or regulations.

[0039] Figure 2 shows a diagram representing the space of input candidates (each input candidate is a small square representing its ordinal number in the overall space of 400 candidates), with some of the inputs (shown in light grey) identified as unlikely to be observed and therefore should be mapped to watermark tokens.

[0040] We want to map these inputs (referred to as watermark inputs) to tokens whose hash values ​​fall into data-release watermark bins (watermark tokens). The cryptographic hash function (based on a secret "watermark key") used within the watermarking algorithm distributes hash values ​​uniformly across the hash space. This implies that hash values ​​that fall into the range of the hash space are derived from values ​​uniformly across the hash space—i.e., the watermark tokens are uniformly distributed across the token space—as shown in Figure 3.

[0041] However, the mapping from input to its tokens is determined by the underlying format-preserving ciphertext, and is a permutation indistinguishable from random—a mapping that is hardwired into the cipher is not possible (at least not without inventing our own non-standard, inevitably unstable ciphertext). To achieve a scheme where a subset of the input space is mapped to a subset of the token space, we must treat the subspaces as separate spaces containing their own separate ciphertexts. To encrypt a value, follow these steps: 1. Determine which distinct subspace contains the value. 2. Encrypt the value using the cipher for that space, resulting in another index in the same space. 3. This output index is again mapped into the global token space to find the resulting token.

[0042] To implement this solution, two problems must be solved. 1. Determine which watermark inputs should be mapped to watermark tokens. 2. Determine which tokens are watermark tokens (i.e., which values ​​in the watermark bins are hash values).

[0043] Figure 4 illustrates these steps.

[0044] Thus, each ciphertext-carrying format uses a secret key. To prevent an attacker from knowing whether a token is a watermark token or not, an additional secret key—the watermark key—is used by a cryptographic hash function.

[0045] Modeling the input value distribution To have access to the distribution of values ​​that occur in the input data set, we first need to know what this distribution is - armed with this information, we can know which inputs occur rarely or never, and try to assign watermark tokens to these inputs. An example of this approach is now described.

[0046] Data Description In some situations, there is some a priori knowledge about the structure of the input data that can be used to describe regions of the input space that will never be encountered. For example, not all numbers that fit the Social Security number structure are actual Social Security numbers because numbers in the first digit group 666 or 900-999 are never assigned.

[0047] Make assumptions about the data If the data has some external meaning—examples include names, email addresses, salaries—it may be amenable to some general heuristics on the data that can be used to make educated guesses about where to allocate watermark tokens. For example, in English text, the digraph "th" is likely to be much more frequent than "qz," and for many numerical distributions, the probability density functions are often low at the high end of the range.

[0048] This can be used to allocate watermark tokens - for example, given a regular expression of [AZ][az]{1,9}, one can specify a range such as (Qz|Jq|Jx|Qx)[az]{0,8} to capture inputs that are expected to occur least frequently (or never).

[0049] Benford's Lawstates that in many numeric data sets, the first digit is likely to be small, and the probability that a particular digit is the first digit decreases logarithmically as the digit number increases. Benford's Law generally applies to data sets with a log-normal distribution, and while its direct use is perhaps not general enough, it can be generalized in this way for any numeric data that falls within some specified range, with the probability density function often being low at the high end of the range. This applies to normal distributions (e.g., height), distributions with long tails (e.g., salary), and monotonically increasing values ​​(e.g., identifiers derived from database sequences), along with log-normal distributions that span several digits and satisfy Benford's Law. Therefore, simply assigning watermark tokens to the high end of a numeric range can be expected to produce good results for many data sets.

[0050] Scanning the Data If nothing is known in advance about the input distribution, and the assumptions in the previous section do not apply, it may be possible to infer it by scanning and observing the data. Note, however, that this is a weak solution over the prior art, since the configuration must be completed by scanning the most recent data, and there is no guarantee that future data will have the same distribution. For example, if the data is drawn randomly from a space with uniform probability, it may be possible to find regions in this space that do not contain values ​​to assign a watermark token, but future values ​​may fall within these ranges (on the other hand, social security numbers never contain unique numbers, and this will continue to be the case for data that follows Benford's law).

[0051] Identifying Watermark Tokens A watermark token is defined as a token whose hash value falls into a watermark bin. Because hash functions are one-way, there is no way to take a watermark bin hash value and find tokens that hash to it. The only way to find whether a token's hash value falls into a bin is to hash it, and brute-force attacking the entire token space to find all watermark tokens is infeasible for all but the simplest token spaces. Instead, we use the fact that watermark tokens are uniformly distributed across token space. This means that if we divide the token space into segments, on average, these segments will contain an equal number of watermark tokens—for example, if we divide the space into as many segments as there are watermark tokens, each segment will contain, on average, a single watermark token (some segments may contain none, some may contain many, but on average, each segment will contain a single watermark token). As an example, Figure 5A shows a diagram of 53 watermark tokens distributed across a 400-size token space. Figure 5B shows the token space divided into 53 segments (29 of 8 sizes and 24 of 7 sizes) filled with different patterns to define the boundaries of the segments (watermark tokens within the segments are highlighted with bold lines). This results in 37 segments containing exactly one watermark token, 8 segments containing two watermark tokens, and 8 segments containing no watermark tokens.

[0052] Now that we've divided the input space into inputs we want to map to regular tokens and those we want to map to watermark tokens, we can specify a segment size that will result in as many segments as there are watermark inputs. We then specify that each segment contains exactly one designated token that we want to assign to a watermark input. We want to specify the true watermark token (i.e., the token that hashs to the watermark bin), but we don't know in advance which token in the segment this will be (or even whether the segment actually contains a watermark token). When we need to select a designated token, we search the segment to find the first token in it that is a true watermark token (if none is found, we go back and return the last token in the segment).

[0053] When a specified token needs to be selected, the following process is used to search for the segment. 1. Determine the starting point within the segment - this should be deterministic (so that you get the same answer every time for this segment) but different for each segment. To do this, seed a pseudo-random number generator (PRNG) with the segment index and use this to select an index within the segment. 2. Check the token at this starting index to see if it is a watermark token (i.e., hashes to a value that falls into a watermark bin). If it is, it becomes the designated token for the segment. 3. If the token is not a watermark token, move on to examining the next token. Continue this process (loop at the end of the segment) until you find a watermark token or reach the start again. 4. If the starting point is reached again without finding the watermark token, point to the last token examined in the segment (i.e., the last token encountered after traversing all tokens in the segment with a wraparound). This is why we choose a different starting point for each segment - if we started from the same point for each, we would return the same token for each segmentation (i.e., the designated token is at a constant interval in the token space).

[0054] FIG. 6 shows a diagram illustrating the process for an example pair of segments, one containing an actual watermark token and one not.

[0055] Note that there are two ways in which this system can lead to imperfect watermarked input-to-watermark token and non-watermarked input-to-non-watermark token mappings. 1. If a segment contains multiple actual watermark tokens (tokens with the hash value of the watermark bin), only one of them will be assigned to the watermark input, and the others will be returned for the non-watermark input. This adds noise to the watermark. 2. If a segment does not contain a watermark token, the watermark input is assigned to a token that is not actually a watermark token. This is harmless - watermark inputs are (by definition) rarely encountered anyway, and no noise is introduced by returning a non-watermark token.

[0056] As can be seen, the first outcome weakens the embedded watermark and is more likely to occur when too few watermark entries are declared. The number of declared watermark entries increases the number of designated tokens considered, but this does not depend on the number of watermark tokens actually present. The number of watermark tokens depends on the number of hash bins and the token space size. Clearly, this scenario occurs when the number of watermark entries is less than the number of watermark tokens.

[0057] The second outcome is harmless as long as its occurrence does not also imply the occurrence of the first outcome—for example, if the number of declared watermark inputs exactly matches the number of actual watermark tokens, segmentation may produce an incomplete result, and the fact that some segments produce outcome 2 means that others must produce outcome 1.

[0058] An optimal strategy is achieved when the number of watermark inputs exceeds the number of watermark tokens by a large margin, so that segments are small enough that the probability that a segment will contain a large number of watermark tokens is low. However, this should not be achieved by artificially inflating the number of watermark inputs, as this would result in assigning watermark tokens to inputs that are encountered in reality less rarely than other inputs (thus introducing noise into the watermark through frequent release of watermark tokens).

[0059] Security If an attacker were able to determine that a particular token was a watermark token, they would know that the corresponding input was drawn from the space of less frequently encountered inputs—an unacceptable information leak. However, because the algorithm uses a secret watermark key when embedding the watermark (using a cryptographic hash function), this type of guessing is impossible without knowledge of this key, and the attacker cannot learn much more from the token than with vault-based solutions.

[0060] A working example will now be described to illustrate the complete algorithm.

[0061] Example In this section, we simultaneously address the steps of the complete algorithm for two scenarios: inputs that fit into the watermark input domain and those that do not. We use the same example scenario considered above. Figure 7 shows a diagram illustrating the input space and token space. The left pane shows the input space 71, which contains the declared watermark input 72. The right pane shows the token space 73, which contains the distribution of watermark tokens, also shown in light grayscale color 74 (but note that this is unknown to the algorithm). In this example, we tokenize the highlighted non-watermark input 75 (the input with ordinal number 123) and the highlighted watermark input 76 (the input with ordinal number 356).

[0062] Step 1: Determine the index of the value in the input space For each input, we first determine which subspace it is contained in and its index within that space. Figure 8 shows a diagram illustrating the steps for determining the index of a value within the input space.

[0063] In this example, input 75 (the input with ordinal number 123) falls into the non-watermark input subspace with an index of 117, and input 76 (the input with ordinal number 356) falls into the watermark input subspace with an index of 9.

[0064] Step 2: Encrypt the subspace index Now we encrypt the input within that subspace - we take an index within this space and use a format-preserving encryption method to get another index within the same space. Figure 9 shows a diagram illustrating the encryption of a subspace index.

[0065] In this scenario, input 75 has index 117 in the 347 size subspace—in this example, this is encrypted to index 226. Input 76 has index 9 in the 53 size subspace, which is encrypted to index 23.

[0066] Step 3: Find the token space segment to which these inputs map We now need to find the token ordinals to which these subspace indices map. By definition, there is one designated token per segment - for a total of 53 watermark entries, this means defining 53 segments. We try to balance the sizes of these segments as much as possible, thus we have 29 segments of size 8 and 24 segments of size 7.

[0067] Figure 10 shows a diagram that explains how to find the token space segment to which an input is mapped.

[0068] Since there is one designated token per segment, it is obvious that the designated token with index 23 is in the 23rd segment. To find the non-watermark token with index 226 (the 226th non-watermark token in space), we need to skip through the segments until we have passed the other 225 non-watermark tokens. Each of the first 29 segments contains eight tokens, one of which is the designated token, so if we skip through all of these, we have passed 203 non-watermark tokens. The remaining segments contain seven tokens (six of which are non-watermark tokens), so we need to skip three more of these to get a total of 221 non-watermark tokens. Therefore, we can say that the non-watermark token with index 226 is the fifth non-watermark token in the 33rd segment.

[0069] Step 4: Explore the segment to find the desired token Once we have found the correct segment for each case, we then need to find the associated token within it.

[0070] Figure 11 shows a diagram that explains how to search a segment to find the desired token. For input 111, we are searching for a non-watermark token at index 226, which we calculate to be the fifth non-watermark token in the 33rd segment. Our random but deterministic starting point for this segment is token 6, which we examine and find to be not a watermark token. We then go on and examine the next token, discovering that this token is a watermark token and is therefore the specified token. We therefore know that we must search four more tokens beyond this token to find the fifth non-watermark token. We wrap around at the end of the segment, and thus find that the token we want is the third one in the segment.

[0071] For input 112, we need to find the designated token within segment 23. The starting point for this segment is index 3, and we must proceed by examining each token until we find that the seventh token in the segment (the fifth token examined) is the watermark token, and therefore the designated token.

[0072] Step 5: Return the final token ordinal All that remains now is to map the found segment token to its ordinal number in the overall token space, as shown in Figure 12, which illustrates the output including the final token ordinal number.

[0073] Here we finally see that input 111 with ordinal 123 tokenizes into a token with ordinal 256, and input 112 with ordinal 356 tokenizes into a token with ordinal 184.

[0074] While a watermarking scheme has been described when combined with vault-less tokenization, it may also be extended to be combined with vault-based tokenization. In a vault scheme, watermarking is generally an easy problem to solve, since the system can be configured so that no watermark tokens are output when certain inputs are encountered. However, if more inputs are seen than there are non-watermark tokens present, the vault scheme will break down, since in that case a watermark token must be returned. However, the described process also provides a solution that will avoid this problem when watermarking is combined with vault-based tokenization.

[0075] Further details about the algorithm will now be presented.

[0076] Algorithmic Building Blocks Hash-based watermarking A pattern is embedded using the hash value of the token to allow extraction of the watermark from the token alone (without information about how it was generated). Note that although this document discusses the process in terms of "tokens" - understood to be the output of a consistent tokenization operation - the same watermarking methodology applies to any process that produces an output that contains some pseudo-randomness.

[0077] For example, the output of a numeric blurring process is hashed and subjected to the same process.

[0078] The hash space is divided into equal-width bins, and each bin can be assigned to a different data release (hence the number of data releases that can be watermarked is equal to the number of bins). The diagram shown in Figure 13 shows an unsigned 32-bit output and a 128 (2 7This is shown for a hash function that yields ∑ n = 127, n = 127, n = 227, n = 1 ...

[0079] To embed a watermark for a data release, we reject tokens that have hashed values ​​that fall into the watermark bins of this data release. The fraction of tokens rejected in this manner is therefore 1 / N, where N is the number of watermark bins in the data release.

[0080] The process of extracting a watermark is illustrated in Figure 14. Each token encountered is hashed, and the count of the bin value that the hash falls into is incremented (in the diagram, a bin is black if it has at least one hash). A single empty bin 141 is left, and the watermark 141 is obtained from that bin index.

[0081] The downside of this algorithm is that as the number of distinct data releases (bins) increases, the number of records required to extract the watermark also increases: to find the watermark, all the other N-1 bins containing at least one value are required, which is disproportionate to the number of data releases.

[0082] Extending to Multiple Hashes: Hash Arrays Inspired by Bloom filters, we use multiple hash functions and work them together in a hash array structure. To embed a watermark, we reject tokens that fall into the watermark bin for any of the hash functions (see below for an alternative method of using this configuration that was tested but ultimately rejected). Then, as shown in Figure 15, when extracting the watermark, we find a single bin index that is empty in every one of the hash function bins in the array—we find an empty bin for each hash function and find the intersection of these sets. When this intersection has only a single empty bin 151, the watermark has been found—but at this point, each of the histograms in the array still has more than one empty bin itself, as illustrated in the diagram. (In this diagram, bins containing a diagonal pattern are symbolically represented as bins—black bins—that are empty for the current hash function but contain hash values ​​for at least one of the other hash functions.)

[0083] Note that this time there are many opportunities for a token to be rejected, resulting in a high token rejection rate (when compared to the same number of bins and a single hash function).

[0084] Dynamic Scaling of Data Release: Multihash Arrays In a hash array structure, the number of bins and the number of hash functions can be adjusted to balance the number of supported data releases and the token rejection rate. However, because the configuration must be determined in advance, one is forced to predetermine the number of data releases desired to be supported, creating a finite watermark pool. However, it is possible to dynamically scale the number of watermark data releases by creating new instances of the hash array structure (each with its own hash function key) and assigning different portions of the data releases to each other. We call this structure a multi-hash array, and it has the following properties: First, a multi-hash array contains only a single hash array instance and therefore supports watermarking of up to N data releases (where N is the number of hash array bins). Once N watermarks have been assigned, a second hash array instance is created to handle the watermarks for the next N data releases. This new instance is given a different hash function key than the first instance, so that patterns embedded by either array appear only as random noise in the other instances. Watermark embedding only requires a single hash array instance (the one containing the watermarked data release) to be used, so computational complexity does not increase as additional data releases are watermarked. · The token rejection rate depends only on the number of bins in a single hash array and the hash function, so it does not grow with the number of supported data releases. Additional hash array instances can be added without limit, with each new instance resulting in only a small increase in the number of columns required to extract the watermark (see analysis below).

[0085] When extracting the watermark, a set of parallel hash array extractions is performed—one for each commissioned instance. As mentioned above, each instance has its own hash function key, so the watermark pattern appears in only a single instance, while in other instances only a uniform distribution of hash values ​​(appearing as random noise) is observed. This is illustrated in Figure 16, which shows an extraction involving two such instances, where the watermark bin 161 is found in one of the instances, while the watermark is not seen in the other.

[0086] The compartmentalization of data release imposed by this structure also provides additional functional advantages. While we already know that embedding a data release watermark only requires the use of a hash array instance of the data release, the same is true for the ability to "sniff" a stream of data to see if a given data release watermark is present. Only the extraction of unknown watermarks requires the full use of a hash array instance. However, this structure also makes it possible to narrow the scope of watermark detection (and therefore the number of columns required) when it is known that the leaked data must come from a subset of a data release (perhaps because the data set in question was only published in a few data releases). If older data releases are terminated and removed from consideration, hash array instances containing only these data releases may be excluded from extraction.

[0087] Each set of individual hash array keys is generated using a scheme such as HKDF (a simple key derivation function KDF based on the HMAC message authentication code) that allows for the expansion of a single master key into many different derived keys (and the ability to efficiently derive a specific key by providing a key "ID" in the input key material). However, it may be possible to have more fine-grained control over the keys (different master keys for different parts of the hash array, or even for each individual hash array), which may also have dyadic security advantages. Permissioning: The ability to exercise finer control over what watermarks can be read. Watermark interception functions can also be used to verify the presence of a watermark and prevent the passage of non-watermarked data, and it is an attractive feature that each of these remote extraction points only has the ability to obtain keys for instances containing the expected watermark, and not simply read *any* watermark. Key rolling: If each new hash array has its own key, then a single hash array key remains in use as long as the data releases within it remain open and active (older versions of the keys must remain on hand as long as it is desired to be able to extract watermarks generated using them).

[0088] Number of supported watermarks and token rejection rate The number of unique watermarks that can be embedded and the proportion of tokens that are rejected (token rejection rate) depend on the configuration parameters of the algorithm, which are: b - the number of data release bins in each hash array instance h - the number of hash functions to use in each hash array instance m - the number of hash array instances currently configuring the multihash array

[0089] Figure 17 shows a diagram illustrating the parameters of the algorithm. One bin is used for each data release, so the number of supported data releases is N=m×b It is clear that the token rejection rate is given by: To embed a watermark for a data release, we reject tokens that hash to a value that falls within the watermark bin of the data release for any of the hash functions.

number

[0090] Extracting Watermarks Noise handling: extraction algorithm requirements The data set from which the watermark is being extracted is not a clean token collection that does not contain the watermark token (it has been altered through the addition of new synthetic sequences; it could be a combination of outputs from several data releases; or the assumptions made about the shape of the data when assigning the watermark input may not be entirely correct). Because the watermark is embedded using a secret key, we assume that it is not possible to create over-represented noise in a particular bin without access to this key (either directly or indirectly through a watermark extraction function), which would not be available to someone trying to erase the watermark.

[0091] Therefore, the addition of a synthetic sequence appears as a base level of noise in addition to the pure watermark, and the combination of multiple data-releasing watermarks appears as several bins where the proportion of each is below the base level. Figure 18 shows three histograms of token counts in each bin for the cases of a "pure" watermark (18A), a noisy watermark (18B), and a watermark where the two are mixed.

[0092] Therefore, the following requirements are placed on the extraction algorithm: It must be possible to extract the watermark even if the hash bin is not completely empty. It must be possible to handle a large number of such bins (meaning the data is a mixture of watermarks from different data releases). It must be possible to give the end user some measure of how certain they can be that the data contains a particular watermark. Approaches that try to find empty bins must loosen the definition of "empty" to deal with noise, and can only do this by defining some percentage / count threshold below which a bin is empty and above which it is not, introducing an element of arbitrariness into poor algorithms.

[0093] To avoid this, the extraction method does not try to determine "Which data release bins are empty?" Instead, for a given data release, it reframes this question to ask "Are we sufficiently certain that the data contains the watermark for this data release bin?" and answers this by calculating how likely it is to observe the current hash number in the data release bin if no watermark is present and only noise in the bin is observed.

[0094] By answering this question for every data release bin, we can return a data release (or multiple data releases) that we are confident is the source of the watermark data (because the number of hashes observed in a bin is unlikely enough to reduce to random noise).

[0095] Overview of extraction algorithms The watermark extraction algorithm is a simple hypothesis test at every bin to determine whether there is sufficient evidence to reject the null hypothesis (the data does not contain the watermark for the current bin) in favor of the alternative hypothesis (the data does contain the watermark for the current bin). This is done by calculating the probability of obtaining a hash value equal to or less than the number observed in the current bin if the data did not contain the watermark for that bin data release. If this probability is less than the level of significance suggested by the user-provided confidence level, then the null hypothesis that no watermark exists corresponding to the data release bin is rejected and instead the presence of such a watermark is declared.

[0096] For multiple hash functions, multiple instances of the hash array structure are provided. When considering a watermark bin, the hash value of the bin (and the total number of hash values) is summed across all of the hash functions.

[0097] Data Release Bin Inspection If the null hypothesis were true, we would expect the hash value to fall into the bin with the same frequency as any other bin. However, if a bin watermark were present, we would expect the rate at which hash values ​​are observed in this bin to be much lower than in any other bin.

[0098] The p-value is defined as the probability of obtaining a result at least as extreme as the result observed when the null hypothesis is true. In this case, this is the probability of obtaining the number of hash values ​​(or fewer) observed in a bin when the data does not contain the watermark corresponding to that bin.

[0099] To calculate the p-value, we model the hashing of tokens into different bins as a binomial distribution, where the probability of a token hashing into a given bin is 1 / b in the absence of a watermark.

[0100] Note that this probability is higher than 1 / b when the data has watermarks corresponding to different bins. However, in these cases, the actual p-value is always smaller than the p-value calculated with 1 / b, so we can safely reject the null hypothesis whenever the calculated p-value is smaller than alpha.

[0101] Therefore, the probability of finding k hash values ​​in a bin after observing n hash values ​​in total is

number

[0102] Thus, the p-value for a bin containing k hash values ​​after observing n hash values ​​overall is

number

[0103] Reliability Level The extraction algorithm takes as user input a confidence level, interpreted as 1-α, where α is a bound on the statistical significance of the test - i.e., the probability of falsely declaring the presence of a watermark in data where no watermark is present (so the significance level provides a bound on the false discovery rate).

[0104] If the calculated p-value for a bin is lower than the significance level α, we reject the null hypothesis and declare the presence of a watermark for this bin, otherwise we fail to reject the null hypothesis and declare that the watermark for this bin is not found in the dataset. Note: This confidence level should not be interpreted as the probability that the watermark is actually present when rejecting the null hypothesis.

[0105] Handling multi-bin comparisons When extracting an unknown watermark from data, every data release bin must be tested, and the decisions from all of these tests must be correct for the process to be successful. Simultaneously testing many bins for the presence of a watermark increases the false positive rate of the overall tests compared to the false positive rate of a single test on one bin. To address this issue, we use the Holm-Bonferroni method, which ensures that the overall error of the "series of tests" is below the required error limit (while ensuring higher statistical power than the standard Bonferroni correction, which in this context means increased ability to detect the presence of multiple watermarks simultaneously mixed into the dataset). This is done by reducing the value of α used for each test by the number of tests to be performed, which is the number of data release bins to be tested.

[0106] The process is to first sort the bins by p-value (lowest first), and then use a different significance level (α) for each bin. The bin with the lowest p-value is examined first, with a significance level of α / b. If the p-value for this data release is lower than the required significance level, the bin with the next lowest p-value is examined, this time using a significance level of α / (b-1). This process continues until a bin with a p-value greater than the corresponding significance level is encountered. Thus, the value of α for extracting a data release watermark combination is: Watermark candidate 1: α = α / b Further watermark candidate 2: α = α / (b-1) Further watermark candidate 3: α = α / (b-2) ... This becomes:

[0107] How many tokens are needed to detect a watermark? No additional noise When no noise is added to the watermark, an empty bin is obtained. Thus, the p-value is simply the probability of all n tokens having a hash value in a bin other than the current bin.

number

number

number

number

number

[0108] Extending to multiple hash array instances and multiple hashes per instance The only difference from the above derivation when multiple hash array instances are provided is that now there are m×b watermark candidates instead of only b. Repeating the above steps with a new number of checks gives

number

number

[0109] When noise is present When calculating the number of tokens needed to extract the watermark, A. More than a certain number of hash values ​​are found in a bin if a matching watermark exists. B. A number less than or equal to the number of hash values ​​found when no matching watermarks exist. We need bounds on the combined probability of

[0110] For the noiseless case, we know that the probability of A is zero (there will never be a hash in a watermark bin in the absence of noise), so we only needed to calculate the probability of seeing zero hash values ​​in a bin when the corresponding watermark is not present. However, in the presence of noise, we need to consider both of these probabilities. In the following new definitions, we use the same notation as before. H o - Null hypothesis that the watermark does not exist H l -Alternative hypothesis: Watermark exists ·p i - expected hash fraction in bin i n i - number of hashes in bin i f - expected noise fraction

[0111] The expected hash fraction in a watermark bin when the noise fraction is f is

number

[0112] We know that the joint probability of two events is always less than or equal to the sum of the individual event probabilities. Therefore, finding the number of tokens needed to read back the watermark at a given noise level is equivalent to solving for n, so there exists a k that satisfies the following inequality:

number

[0113] Since obtaining an explicit expression for n that satisfies the above equation can be a challenge, the estimation function may instead perform a brute force search over n and k to find the number of tokens that satisfy the above inequality.

[0114] Columns and Unique Tokens The above modeling calculates the number of tokens needed to extract the watermark, but this is actually the number of "unique" tokens, since there is an implicit assumption that each token hashed and added to an array bin provides new information about the token distribution (and hence the watermark pattern embedded in it). Embedding a watermark is only possible if there is enough input diversity to return a range of tokens that touch all bins in the hash space - in the unusual case where only a single input is ever encountered, it will only return tokens that fill a single bin.

[0115] Therefore, it is important to try to add each encountered token to the extraction hash bin only once. Thus, the extraction process needs to keep track of previously encountered tokens, which is easy to do using a Bloom filter. Knowing how many unique tokens need to be added before we expect to be able to extract the watermark, we can size the filter appropriately - we choose a limit of 100,000 values ​​(far more than the number of unique tokens we expect to need), and a limit of 10 -9 With a false positive rate of , we obtain a filter that requires only about 525kb of memory (and note that the extraction process only requires a single instance of this filter, regardless of the number of data releases or algorithm configuration parameters).

[0116] Reporting the strength of embedded watermarks The strength of the embedded watermark can be reported to the user as part of the tokenization job. This strength can be interpreted as the maximum confidence level that extraction can be performed and (assuming the output file is not tampered with in any way) the watermark data release will still be correctly obtained, providing an easy-to-understand overview of whether the processed data contains enough unique tokens to bear the watermark.

[0117] Choosing Algorithm Parameters Multihash arrays: Dynamic growth of m As shown earlier, the token rejection rate does not depend on m, and the average number of tokens required to extract a watermark grows logarithmically with m. Therefore, it makes sense to treat m as a dynamic parameter—starting with only a single hash array instance—and adding more as and when additional data release watermarks are required. With this approach, the token rejection rate remains constant, and the number of tokens required to extract a watermark is always (approximately) the minimum possible for the required number of data releases, growing only as new data release watermarks are released.

[0118] Hash arrays: Choosing b and h As can be seen, the choice of b and h affects all aspects of the algorithm: the number of supported data releases, the token rejection rate, and the number of tokens required to extract the watermark.

number

number

number

number

[0119] This relationship, and substituting the formula for the number of data releases into the formula for the number of tokens, gives us the basic relationship between the quantities.

number

[0120] This relationship shows that for a given extraction confidence level, the number of tokens required to extract the watermark depends only on the token rejection rate and the number of supported data releases, and does not depend on the configuration of the multi-hash array.

[0121] This suggests that by rejecting more tokens, the number of tokens needed decreases, and that an acceptable token rejection rate depends on the usage scenario, which may require many different configurations, one for each usage scenario. For our purposes, we consider two common usage scenarios: tokenizing a bulk data set, and tokenizing a very small set of results from an interactive database query.

[0122] Bulk Data Set For the traditional use case of tokenizing bulk datasets, the acceptable token rejection rate is capped at 1%. Therefore, Resulting in a token rejection rate as close to the upper limit as possible, Minimize the number of data releases, since supporting more data releases comes at the expense of more tokens than are needed to extract the watermark. By choosing the parameters, we arrive at an optimal setting. As discussed in the previous section, we can always dynamically increase m to support more data releases, but we cannot support data releases below b—hence, rather than starting with low initial performance, it makes sense to minimize b so that performance is initially optimal and then slowly degrades as more data releases are required.

[0123] Therefore, the suggested configuration for this scenario is: b=1024 h=10 is.

[0124] This gives a token rejection rate of just over 0.97%. Figure 19 shows a diagram illustrating the extraction token count requirements as the number of data releases increases.

[0125] FIG. 20 shows a diagram of the tokens required to extract the watermark with 99.9% reliability for the first hash array instance (supporting up to 1024 data releases) as a function of the input noise ratio.

[0126] Small Data Set For use cases where a small sample of a dataset is obtained as a preview of the dataset or as the result of a selective SQL query, a much higher token rejection rate may be acceptable (since there are many small values ​​that require tokens), but it is necessary to enable embedding of extractable watermarks into a much smaller dataset. Therefore, the proposed configuration for this scenario is: b=256 h=50 is.

[0127] This gives a token rejection rate of approximately 17.8%. Figure 21 shows a diagram illustrating the extraction token count requirements as the number of data releases increases.

[0128] FIG. 22 shows a diagram of the tokens required to extract the watermark with 99.9% reliability for the first hash array instance (supporting up to 256 data releases) as a function of the input noise ratio.

[0129] Scalability calculation time The largest component of the algorithm's computational cost is computing the cryptographic hash value. Therefore, as h increases, one would expect the computational cost of the algorithm to rise significantly. However, due to the Kirsch-Mitzenmach optimization, only a single 64-bit hash value needs to be computed, and then many 32-bit hash values ​​can be derived cheaply from this through multiplication and modular operations without losing randomness. Increasing the number of hash values ​​does indeed increase the amount of computation required, but it does so in a fairly reasonable manner (benchmarks show that the cost of computing 50 hash values ​​when embedding a watermark is ~1.7x the cost of computing one hash value, rather than 50x the cost of computing one).

[0130] This optimization cannot be applied when increasing m because each hash array instance has its own underlying key. Therefore, the computation time scales linearly with m. However, this is only a concern when extracting the watermark, which requires computing hashes across different hash array instances. When embedding the watermark (which is the only task in the tokenization critical path), it is only necessary to examine a single hash array instance (the one containing the data release in which the watermark is embedded), so the embedding computation time does not depend on m.

[0131] Figures 23 and 24 show the results of benchmark runs that confirm these assertions. Figure 23 plots the normalized computation time for watermark embedding, and Figure 24 plots the normalized computation time for watermark extraction. Note that in each graph, the vertical axis values ​​have been normalized within the range of this particular test, since absolute values ​​will vary depending on the environment and we are only interested in trends.

[0132] memory Extracting the watermarks requires that the bins be kept in memory while the data is traversed, incrementing the bin count (we also use a single Bloom filter—regardless of the values ​​of h, b, and m—which requires a small amount of memory). Increasing h or m results in more copies of the bin array being included in memory, and increasing b results in a larger bin array with each copy. In our scenario, the values ​​of b and h are fixed and too small to use much memory per hash array instance. However, as the number of embedded watermarks increases (i.e., m increases), the memory required to extract the watermarks increases. Should m reach a number large enough for memory utilization to become an issue, multiple passes through the data may be necessary to distribute the values ​​of m among them.

[0133] Extraction of the watermark does not require the state to be stored in memory and is therefore unaffected.

[0134] result overview This section presents the results of experiments performed on the proposed scheme to verify various aspects of its behavior. In summary, these results show that: Embedding an extractable watermark requires a predictable number of tokens. The false positive match rate returned by extraction is limited by the confidence level and can therefore be adjusted arbitrarily low. Extraction is more tolerant to random noise at the expense of requiring more columns in noisy cases, increasing column requirements as modeled. A mixture of multiple data release watermarks results in the extraction of all individual watermarks, with additional random noise present. Note: All results presented in this section were obtained using similar bulk dataset settings (b = 1024 and h = 10). However, the observations also apply to datasets with other settings, although the absolute numbers will increase or decrease as predicted by the previous discussion.

[0135] Extraction results Figure 25 shows the final outcome of watermark extraction performed at a 95% confidence level as the number of processing tokens during extraction increases. Each data point is the average of 10,000 experiments and shows the division of outcomes among three mutually exclusive possibilities: no result returned, only correct data releases returned, and correct and incorrect data releases returned (theoretically there is a fourth outcome—only incorrect data releases returned—but this never occurs).

[0136] Here we can clearly see the effect of the confidence level parameter, which limits the rate at which false positives are returned (encounting the token count that modeling shows is necessary to extract the watermark with 95% confidence—1017 tokens—will always return the correct data release, but with an accompanying rate of false positive results that never exceeds the alpha level suggested by this confidence).

[0137] While the previous graph demonstrates the use of a 95% confidence level so that the effect of the parameters can be easily visualized, false positives are an undesirable event in watermark extraction—in some circumstances, they can result in innocent recipients being accused of being the source of data leakage—so a 5% false positive rate would be far too high for practical use. However, the confidence level provides an easily understood mechanism for reducing the frequency of false positives to a desired level—if the user specifies a 99.9% confidence level, false positives are sure to be returned in 0.1% or fewer of the cases. And because the score for true positive data release increases rapidly with the number of tokens, this additional precision comes at a reasonable cost in the number of tokens required to extract the actual watermark. This is shown in Figure 26, where the above experiment is repeated at a 99.9% confidence level.

[0138] Noise immunity The above experiment was repeated, but with increasing levels of random noise added. The graph in Figure 27 shows that the proportion of time obtaining only the correct watermark data release varies with the noise level and the number of tokens from which extraction was performed (again at a 95% confidence level so that the effect is clearly visible). The graph shows that, as expected, the number of tokens required to obtain a correct result at a given confidence level increases.

[0139] Figure 28 shows the rate of false positive occurrence for the same experiment (where a false positive is recorded whenever at least one incorrect data release is returned, regardless of whether a correct data release is also returned).

[0140] Here, it can be clearly seen that the false positive rate is bounded by the confidence level provided (and it does not depend on the level of noise).

[0141] The number of tokens required to achieve authenticity In the previous discussion of watermark extraction algorithms, a method was presented for estimating the number of tokens required to extract a watermark for a given confidence level and the amount of noise added to the input data set. To test its accuracy, the number of tokens required to extract a watermark with 95% confidence was calculated for various noise levels, and then an experiment was performed for each, attempting to extract the watermark 10,000 times on a data set of this size and recording the results. The results of this experiment are shown in Figure 29 (areas representing the proportion of times only correct data releases were returned, and areas representing the proportion of times at least one incorrect data release was returned, regardless of whether a correct data release was also returned—using the right-hand y-axis—the line indicates the number of columns for which extraction was performed for this noise level).

[0142] From this graph, we can see that our estimates of the number of tokens needed are accurate - the extraction success rate follows the expected confidence level (and, as usual, the false positive rate never exceeds the expected 5%).

[0143] Mixed Data Release Watermarks Figures 30 and 31 show the results of experiments in which the input to the extraction function was a data set containing multiple data release watermarks to be mixed, with increasing levels of random noise added. Figure 30 shows an experiment in which two data release watermarks were mixed together, and Figure 31 shows a mixture of three data release watermarks. In all instances, the watermarks were equally shared across the tokens remaining after the random noise was added (e.g., the 40% noise result encompasses 40% noise / 30% data release 1 / 30% data release 2 in Figure 30, and 40% noise / 20% data release 1 / 20% data release 2 / 20% data release 3 in Figure 31). All experiments were performed at a 95% confidence level.

[0144] Here we see that even with the addition of random noise, a large number of watermarks are correctly sorted. As expected, more tokens are needed to extract more watermarks (since from the perspective of one of the watermark bins, data carrying other watermarks appears only as random noise, thus delaying the extraction of this watermark using the techniques presented in the previous section).

[0145] The false positive rate is not shown in the graph above, but is limited to 5%, as expected.

[0146] Alternative Data Release Representations The proposed scheme uses the same bins for each data release in each hash array. However, an alternative scheme uses different bins for each hash function for a data release, representing the data release as a set of hash function+bin pairs (one for each hash function)—the bins of a given hash function are used in many data releases—but the combination of bins in a hash function is unique for that data release. As shown in Figure 32, this setup yields more data releases than the previous case, but more tokens are needed to extract the watermark directly because the hash functions no longer work simultaneously. (In the original setup, a token hash value appearing in a bin for any of the hash functions is sufficient to eliminate this bin from consideration in all hash functions. In this alternative scheme, this is no longer the case, and each hash function works independently.) However, this still requires fewer tokens than a single-hash function structure—each additional token excludes a bin in each of the hash functions, and by summing all values ​​in all hash functions for a data release's bins, a high reliability score can be reached with fewer tokens.

[0147] With this setting, the number of watermarks supported is N=m×b h It can be calculated as follows.

[0148] However, this exponential growth in data releases was ultimately the reason for the rejection of this setup. As can be seen, there is a fundamental relationship between the number of tokens required to extract the watermark and the number of data releases that can be supported for a given token rejection rate - therefore, somewhat paradoxically, it is actually advantageous to have a scheme in which the number of data releases is increased slowly to get as close as possible to the minimum number of tokens required, in order to more accurately match the token rejection rate as close as possible to the allowed 1% limit.

[0149] Appendix A - Deterministic Tokenization of Watermarks This appendix outlines key features AD. Each listed feature may be combined with any other of features AD. Each optional feature defined below may be combined with any other feature or any other feature.

[0150] Key Feature A: A process that incorporates watermarking in addition to deterministic tokenization A computer-implemented process for embedding a watermark in tokenized data comprises: (a) generating a token from a set of input data, the token being generated using a deterministic encryption scheme; (b) embedding a watermark into the generated set of tokens; Includes.

[0151] Key Feature B: A process that incorporates watermarking in addition to deterministic tokenization, where the watermark is probabilistically embedded through the selection of tokens in the generated token set. A computer-implemented process for embedding a watermark in tokenized data comprises: (a) generating a token from a set of input data, the token being generated using a deterministic encryption scheme; (b) embedding a watermark into the set of generated tokens; , where the watermark comprises a pattern that is probabilistically embedded through the selection of tokens in the generated token set.

[0152] Key Feature C: A process that incorporates a watermark in addition to deterministic tokenization, where the watermark is recovered without prior knowledge of the encryption scheme or other processing used on the input data set. A computer-implemented process for embedding a watermark in tokenized data comprises: (a) generating tokens from an input data set, the tokens being generated using a deterministic encryption scheme; (b) embedding a watermark into the set of generated tokens; , so that the watermark can be recovered without prior knowledge of the encryption scheme or other processing used on the input data set.

[0153] Key Feature D: A process that incorporates watermarking on top of deterministic tokenization, where the number of watermark data release candidates can be dynamically scaled. (a) generating tokens from an input data set, the tokens being generated using a deterministic encryption scheme; (b) embedding a watermark into the set of generated tokens; (a) generating a data release and selecting or choosing a watermark based on parameters of the data release; A computer-implemented process for incorporating a digital watermark into tokenized data, comprising:

[0154] Optional Features The process of embedding a digital watermark The watermark is embedded in the set of generated tokens rather than in metadata or redundant data. · Deterministic encryption methods are pseudo-random permutation methods such as pseudo-random permutation methods based on format-preserving ciphertext. The process is (a) scanning or observing an input space corresponding to an input data set; (b) determining or identifying inputs to be mapped to watermark tokens; Includes: The inputs to be mapped to watermark tokens are inferred from knowledge of the input space and what it represents. The inputs to be mapped to watermark tokens are those that are unlikely to occur or be encountered (meaning that unlikely occurrence refers to using the distribution of probabilities of encountering potential input values ​​so that tokens that are undesirable to return are mapped to inputs that are not expected to be encountered). The process involves splitting the input space into two separate or disjoint subspaces, namely a "non-watermark input subspace" and a "watermark input subspace", where inputs in the non-watermark input subspace are mapped to non-watermark tokens and inputs in the watermark input subspace are mapped to watermark tokens. · Encryption of each input subspace is achieved independently. The algorithm includes two deterministic encryption methods such as FPE (Format Preserving Encryption). · Some FPEs are configured to encrypt the non-watermark input subspace, and other FPEs are configured to encrypt the watermark input subspace. ·Each of the two deterministic encryption methods is based on a secret key. A watermark pattern is embedded in the watermark token, and the watermark token is assigned or determined so that the token hashes to a value that falls within a predetermined range in the hash space. The tokens are hashed using a cryptographic hash function based on a secret watermark key, so an attacker cannot know in which subspace the token resides. The process includes the step of attempting to map watermark tokens such that the watermark tokens are never or rarely returned to bins in the hash space that contain no values ​​or values ​​close to zero. Watermark tokens are dynamically assigned or determined at run-time, avoiding the need to brute-force search the entire token space. · Watermark token judgment is (i) segmenting the token space into segments; (j) assigning a watermark token to each segment; Includes: Watermark tokens are assigned by scanning a segment to find the first token in the segment that hashes to a value within a predetermined range. The starting index for scanning each segment is selected using a pseudo-random number generator (PRNG) seeded with the segment index. If the start index is reached without finding a watermark token, the last index of the segment is selected as the watermark token. The size of the segments is chosen to increase the probability that each segment will contain one or fewer tokens that hash into the watermark range (assuming a uniform distribution from the hash function), thus avoiding the need to return these tokens for inputs other than those intended.

[0155] Restoring or extracting digital watermarks from data releases The watermark can be recovered without prior knowledge of the deterministic encryption scheme or other scheme used on the input data set. The watermark is recovered by (a) hashing the tokens of the data release using one or more hash functions, (b) constructing a histogram of hash frequencies in the hash space, and (c) determining the bin that corresponds to the watermark. When multiple hash functions are used, the process includes summing the number of hashes for each bin across all of the hash functions. Each hash function contains a different key. The process includes calculating the probability of obtaining the number or fewer hash values ​​observed in a particular bin when the watermark data release does not contain a watermark corresponding to the particular bin. When the calculated probability is lower than a predefined threshold, the process infers that the data release contains a particular watermark. The process allows for the recovery of the watermark with only a subset of the data release. Processes can be used to handle noise, such as adding or removing columns from a data release. Extracting a watermark is associated with a confidence score related to the likelihood of (a) the presence of the watermark in the tokenized data and (b) that an under-represented bin in the histogram of token hash counts corresponds to a hash bin (how confident can we be that the watermark is actually present in the tokenized data and that the under-represented bin in the histogram of token hash counts is not just a coincidence). The process includes a step of estimating the number of tokens required to recover the watermark. For a given confidence level and amount of noise present in the tokenized data, the number of tokens is estimated.

[0156] Watermarking depends on the parameters of the data release The computer-implemented process includes generating a data release, and selecting or choosing a watermark based on parameters of the data release. · Digital watermarks are selected or chosen based on the data release beneficiary. The watermarks are selected or chosen based on the intended use of the data release. The watermarks are selected or chosen based on the date by which the data release must be deleted. Each data release corresponding to the same tokenized input data set contains a different watermark. · A digital watermark represents a pattern that is a unique ID. Watermarks are selected or chosen based on the type of tokenized input data (i.e. ID, Social Security number, height, salary, etc.). The watermark is selected or chosen based on the deterministic encryption method used. The process includes dynamically scaling the number of watermark data release candidates by updating the hash function. The process includes dynamically scaling the number of watermark data release candidates by hashing the token space with different hash functions.

[0157] Computer device or system 1. A computing device or system adapted to embed a digital watermark in tokenized data, comprising: (a) generating tokens from an input data set, the tokens being generated using a deterministic encryption scheme; (b) embedding a watermark into the set of generated tokens; A device or system comprising a processor configured to:

[0158] Appendix B - Handling variable length input in deterministic tokenization Background to deterministic tokenization In deterministic tokenization, format-preserving cryptograms are used to provide permutations between all of the values ​​in some space and other values ​​in the same space. For example, a space can be declared to be the entire set of valid credit card numbers - a deterministic tokenization system would accept a credit card number as input and return another credit card number as output, to use as a token instead of the actual value. The space of values / tokens is specified using regular expressions.

[0159] How it works: A simple example An example is depicted in the diagram shown in Figure 33, where all input and token spaces are represented using small squares for each possible value - to fit the two-dimensional nature of the example representation (and for spatial reasons) these are arranged in larger rectangles, but are also shown as a long list of values.

[0160] As an example, suppose all values ​​that need to be tokenized consist of two lowercase alphabetic characters. The regular expression [a-z]{2} is used to define this space. Any value that fits this expression can be assigned an ordinal number in the range [1..676]. Imagine that the input jj needs to be tokenized - this is the logic that a deterministic tokenization system would implement. 1. Find the ordinal of the input value in the regular expression. In this example, the input is jj - for simplicity we will assume the order is lexicographical (but in reality it need not be), thus the input ordinal is (9*26)+10=244 (input ordinal 33A is shown in Figure 33). 2. Encrypt this ordinal using a format-preserving ciphertext. For this example, let's imagine this gives us an output of 419. 3. Determine the value from a regular expression containing the ordinal number 419. In this lexicographic order, we thus get the first character of (419 / 26)+1=17 (hence q) and the second character of 419 mod 26=3 (hence c). 4. Hence, input jj is tokenized into token qc (token qc33B is shown in Figure 33).

[0161] Issues with using deterministic tokenization for variable-length values Deterministic tokenization systems work well for values ​​that have a set structure—for example, credit card numbers, social security numbers, and zip codes—but they do not work as well for data that is relatively free-form and can have variable lengths—for example, names, addresses, and email addresses. While it is possible to make this work for variable lengths because regular expressions can be written that accept some characters a variable number of times, the results may not be satisfactory to the user.

[0162] A fairly simple scenario: consider tokenizing surnames. The US census website has a list of the most common surnames from the 1990 census - this covers over 90% of the population, so we can base our thinking on this (in the real world, we would need to worry about outliers such as double surnames, but we will ignore these for now).

[0163] From this list, we can see that the surname lengths range from 2 to 13 characters, with the distribution shown in FIG.

[0164] So we create the expression [AZ][az]{1,12} to cover the entire input range. This works, but what do the resulting tokens look like?

[0165] Because tokenization uses strong cryptography, the selection of tokens from the token space is indistinguishable from random numbers—every candidate token is a possible output value for every other token. This means that the length distribution of a random sample of tokens has the same distribution as the lengths of all candidate tokens. For two-character tokens, there are 676 possible, for three-character tokens, 17,576 possible, and so on—by the time you reach a length of 13 characters, there are over 2 quintillion candidate tokens. The distribution of tokens is shown in Figure 35, plotted on a linear scale axis.

[0166] Adding additional characters results in the number of potential tokens being multiplied by 26, as with the previous token length, so the number of tokens of each length grows exponentially. For the real data, the most common surname length was six characters - however, over 96% of the generated tokens had a maximum length of 13 characters, making the chance of obtaining a six-character token approximately 1 in 84 million.

[0167] Perhaps in the hypothetical example above, this isn't a serious problem - the tokens are intended to be meaningless substitutes for the raw values, and one wouldn't expect to be able to use them to do any analysis on the length of names. But what happens when we make this example more realistic - when we try to extend the representation to cover double surnames? Every time we add a character to the representation, we also extend the length for the majority of the tokens we encounter. What if we wanted to process data such as email addresses? To ensure we covered every possible email address, we would need to allow 64 characters for the local part and 255 characters for the domain part, giving all of our tokens a length of (effectively) 320 characters - a veritable data size explosion that is unacceptable.

[0168] The state-of-the-art in format-preserving masking systems avoids this problem by treating each different input length as a unique ciphertext and generating a token that matches the input length (so in our name example, if we accept a 6-character name, we tokenize it using the representation [AZ][az]{5} to create a 6-character token). However, we believe this is an unacceptable privacy risk, as it does not provide any privacy for values ​​with unique lengths. To illustrate this, we used a list of employees from a specific company provided on a specific date, 06 / 04 / 2021—26 of the 158 employees on this date (a highly significant 16%) were uniquely identifiable using only the lengths of their first and last names, and a length-preserving scheme would provide no protection for all of these people.

[0169] Solution It is desirable to achieve a system where the distribution of token lengths is similar to the distribution of input lengths, but where token lengths are not directly linked to the length of the input. There is an obvious tension between these two requirements, but it is sufficient to break the direct deterministic link between value and token length and instead adopt a scheme where a 6-character value is likely to have a 6-character token, although there is still a significant (and controllable) chance that this will not be the case. And when tokenizing a data set that contains a single length value within it, finding a length token does not guarantee that this is the length input (there is an element of plausible deniability).

[0170] Our proposed solution works by treating each length as a separate format-preserving encryption, but allocating a range of ciphertext to tokens of different lengths. This consists of a minimum and maximum length, and for each input length, a distribution (e.g., a normal distribution) is generated with some standard deviation centered around this length (which can be surfaced to the user as some "spread" parameter), and this is used to derive a weighting for how many tokens of this input length are allocated to tokens of this length. Using [az]{2,13} as an example, Figure 36 shows a graph of the weights that control how many tokens of each length are generated for a first input value of length 2, using a standard deviation of 1 / 3 of the range.

[0171] We then normalize these weights and multiply each by the total number of tokens we have to allocate (676 of these in this case, since we are considering allocations for two-character strings) to see how many of these will be allocated to tokens of each length. A graph of the length tokens generated as a function of the number of tokens of this length allocated is shown in Figure 37 for a two-character input.

[0172] So, for 676 possible two-character input strings, we map 122 of them to two-character tokens, 119 to three-character tokens, etc., so that only 3 of the inputs map to 13-character tokens. When we have a two-character input, we do format-preserving encryption to map ordinals in the input space 1..676 to ciphertext ordinals in the same space. We then interpret this ciphertext ordinal differently depending on which region of the ciphertext space it falls into. This is illustrated in the diagram in Figure 38.

[0173] For example, if we get a two-character input whose ordinal is 1..676 in the space of all two-character inputs and encodes it as ordinal 360, we return a five-character token for this input.

[0174] Move to the next input length - in this example, 3 - and repeat the process.

[0175] Once these weights and ranges are initialized, the cipher can be used for tokenization. In the example above, a two-character input is encoded into ordinal 360, and a five-character token for it is returned. But which five-character token? We first start indexing into the space of five-character tokens, using the chunk resulting from the two-character input, then the chunk for the three-character input, and so on. So in this example, ordinal 360 is the tenth ordinal from the start of the chunk for the two-character input, so we get ordinal 10. But doing this means that the first N ordinal numbers of any token length will always be assigned to two-character inputs, and the next M ordinal numbers to three-character inputs, leading to a bias in which tokens with lower ordinal numbers are assigned to shorter inputs. To correct for this, once we have the ordinal numbers in the five-character space, we perform a further FPE within this very space, which re-randomizes all of the mappings to tokens of this length, so no systematic bias is introduced.

[0176] watermark The watermark also works as described above in relation to the process for embedding a digital watermark in tokenized data. Watermark entries are selected from the longest entry length (as this is assumed to be the least likely to occur in real data), and watermark tokens are assigned to these entries across all lengths. Watermark tokens for each length are found as detailed above—each length is assumed to contain the number of tokens for this length / number of watermark bins of watermark tokens—which are found dynamically using the segmentation and dynamic scanning techniques described previously.

[0177] Therefore, the solution may be combined with any one or more of the features, or any one or more of the features from Appendix A, and the generation of tokens described above.

[0178] Appendix C - Handling Variable Length Input - Summary of Key Features This appendix outlines the main features EF. Each optional feature defined below may be combined with any other feature or with any other feature.

[0179] Key feature E - Token generation from variable length input data sets A computer-implemented process for generating tokens from a set of input data including input values ​​of variable length, the process comprising generating tokens from the input data set, the tokens having lengths that exhibit a distribution substantially similar to the length distribution of the input data set.

[0180] Optional features: The length of a token does not necessarily match the length of the corresponding input value. ·Tokens are generated using encryption methods that treat each subset of the input data corresponding to a particular length of the input value as a separate or distinct format-preserving encryption. · For each subset of input data corresponding to a particular length of the input value, a token is generated with variable length. For each subset of input data corresponding to a particular length of input values, tokens are generated with variable lengths, and the generated tokens have lengths that fit the selected distribution generated for the particular input length. When a token is generated from a subset of the input data corresponding to a particular length of the input value, additional format-preserving encryption is used on the generated token. · Tokens are generated based on one or more potentially user-configurable parameters including minimum token length, maximum token length, or a distribution used to assign token length weighting.

[0181] Key feature F - Embedding digital watermarks into tokens generated from variable-length input data sets 1. A computer-implemented process for embedding a watermark in tokenized data, comprising: (a) generating tokens from an input data set comprising input values ​​of varying lengths, such that the tokens are generated with lengths that exhibit a distribution substantially similar to the distribution of lengths of the input data set; (b) embedding a watermark into the generated set of tokens; A process that encompasses.

[0182] Optional features: Token Generation The length of a token does not necessarily match the length of the corresponding input value. ·Tokens are generated using encryption methods that treat each subset of the input data corresponding to a particular length of the input value as a separate or distinct format-preserving encryption. · For each subset of input data corresponding to a particular length of the input value, a token is generated with variable length. For each subset of input data corresponding to a particular length of input values, tokens are generated with variable lengths, and the generated tokens have lengths that fit the selected distribution generated for this particular input length. When a token is generated from a subset of the input data corresponding to a particular length of the input value, additional format-preserving encryption is used on the generated token. · Tokens are generated based on one or more potentially user-configurable parameters including minimum token length, maximum token length, and the distribution to be used to assign token length weights.

[0183] The process of embedding a digital watermark · The watermark contains a pattern that is probabilistically embedded into the set of generated tokens through the selection of tokens. The watermark can be recovered without prior knowledge of the encryption scheme or other processing used on the input data set. The process includes generating a data release, where watermarks are selected or chosen based on parameters of the data release, and the number of candidate watermark data releases can be dynamically scaled. The watermark is embedded in the generated token set, rather than in any metadata or redundant data. The process is (a) scanning or observing an input space corresponding to an input data set; (b) determining or identifying inputs to be mapped to watermark tokens; Includes: · The inputs to be mapped to watermark tokens are inferred from knowledge of the input space and what it represents. The inputs to be mapped to watermark tokens are unlikely to occur or be encountered (meaning that the probability of encountering potential input values ​​is used to map tokens with undesirable returns to inputs that are not expected to be encountered). The process involves splitting the input space into two separate or disjoint subspaces, a "non-watermark input subspace" and a "watermark input subspace," where entries in the non-watermark input subspace are mapped to non-watermark tokens and entries in the watermark input subspace are mapped to watermark tokens. · Encryption of each input subspace is achieved independently. The watermark pattern is embedded in a watermark token, and the watermark token is assigned and determined so that the watermark token hashes to values ​​that fall within a predetermined range of the hash space. The token is hashed using a cryptographic hash function based on a secret watermark key, so an attacker cannot know in which subspace the token resides. The process involves a step that attempts to map watermark tokens that are never returned or returned infrequently, so that the bins in the hash space contain no values ​​or values ​​close to zero. It is determined whether watermark tokens are dynamically assigned at run time, avoiding the need to brute-force search the entire token space. · Watermark token judgment is (k) segmenting the token space into segments; (l) assigning a watermark token to each segment; Includes: A watermark token is assigned by scanning the segment to find the first token in the segment that hashes to a value within a predetermined range. A pseudo-random number generator (PRNG) seeded with the segment index is used to select the starting index for scanning each segment. If the start index is reached without finding a watermark token, the last index of the segment is selected as the watermark token. The size of the segments is chosen to increase the probability that each segment will contain no more than one token that hashes to the watermark range (assuming a uniform distribution from the hash function), avoiding the need to return tokens for inputs other than those intended.

[0184] Restoring or extracting digital watermarks from data releases The watermark can be recovered without prior knowledge of the encryption scheme or other scheme used on the input data set. The watermark is reproduced by (a) hashing the tokens of the data release using one or more hash functions, (b) constructing a histogram of hash frequencies in the hash space, and (c) determining the bin that corresponds to the watermark. When multiple hash functions are used, the process involves summing the number of hash values ​​for each bin across all hash functions. Each hash function contains a different key. The process includes calculating the probability that a particular bin will have a hash value at or below the number observed when the watermark data release does not contain a watermark corresponding to the particular bin. When the calculated probability is lower than a predefined threshold, the process infers that the data release contains a particular watermark. · The process allows recovery of the watermark on only a subset of the data release. The process can handle noise such as adding or removing columns from the data release. Extracting a watermark is associated with a confidence score related to the likelihood of (a) the presence of the watermark in the tokenized data and (b) that an under-represented bin in the histogram of token hash counts corresponds to a hash bin (how confident can we be that the watermark is actually present in the tokenized data and that an under-represented bin in the histogram of token hash counts is not just a coincidence). The process includes a step of estimating the number of tokens required to recover the watermark. For a given confidence level and the amount of noise present in the tokenized data, the number of tokens is estimated.

[0185] Watermarking depends on the parameters of the data release The computer-implemented process includes generating a data release, and selecting or choosing a watermark based on parameters of the data release. · Digital watermarks are selected or chosen based on the data release beneficiary. · Digital watermarks are selected or chosen based on the intended use of the data release. The watermarks are selected or chosen based on the date by which the data release must be deleted. Each data release corresponding to the same tokenized input data set contains a different watermark. · A digital watermark represents a pattern that is a unique ID. · Watermarks are selected or chosen based on the type of tokenized input data (i.e. ID, Social Security number, height, salary, etc.). The watermark is selected or chosen based on the encryption method used. The process includes dynamically scaling the number of watermark data release candidates by updating the hash function. The process includes dynamically scaling the number of watermark data release candidates by hashing the token space with different hash functions.

[0186] Computer device or system A computing device or system adapted to generate tokens from an input data set including input values ​​of variable length, the device or system comprising a processor configured to generate tokens from the input data set, the tokens having lengths exhibiting a distribution substantially similar to the length distribution of the input data set.

[0187] 1. A computing device or system adapted to embed a digital watermark in tokenized data, the device or system comprising: (c) generating tokens from the input data set, the tokens being generated with lengths that exhibit a length distribution substantially similar to the length distribution of the input data set; (d) Embedding a digital watermark into the generated token set; The method includes a processor configured to:

[0188] Embedding a digital watermark into the output of a generative AI, such as text generated by a D-LLM (large-scale language model) Since the described watermarking scheme is based on rejecting outputs with hash values ​​within the watermark bin, the process can embed a watermark as long as it meets the following two criteria: 1. If the initial selection of an output is a watermark token, then there must be the possibility of substituting one potential output for another, so that a different non-watermark token output may be selected instead. 2. To determine whether a token is a watermark, some system must be in place to interpret the potential output as a "token" that can be hashed. For text generation, this is obvious, but there are many ways to interpret binary output in a similar way—for example, using pixel values ​​in an image or video frame, or the frequency of an audio sample—since it is possible to modify these in subtle ways without affecting the output. When the term "token" is used in this document, it should be understood to refer to the potential output of the generation system interpreted in this way, rather than simply text.

[0189] Generative AI systems that produce text, images, video, or audio meet both criteria, and therefore this method can be used to watermark their output. To see exactly how this works, let's take text generated by an LLM as an example. The LLM generates text by outputting a sequence of generated tokens, where a token can be, for example, a word, part of a word, or punctuation. Each token is generated using a probability distribution of the most likely next token generated in response to a prompt, taking into account previous tokens generated. However, the system does not always simply select the token with the highest probability—there is an element of randomness that can select a token with a low probability. This is where a watermarking method can be inserted. If the system hashes the token it is about to return, it can determine whether the token is a watermark token. If so, this token can be discarded and the distribution resampled to select a different token, and the process can be repeated until a non-watermark token is selected which can then be returned (in the unlikely event that no non-watermark tokens are available for selection, a watermark token must be returned, but as discussed in the "Noise Handling" section on page 25, this will only add a small amount of noise to the resulting watermark).

[0190] The presence of a watermark can be used to determine that the content was created by a generative AI system. However, because each watermark is also unique to the prompting user, extracting the watermark will reveal not only that the content was created by a generative AI, but also for which user. Metadata about the user, the time of use, and the context in which the content was created can also be associated with the watermark.

[0191] Thus, using the described watermarking scheme, the generated AI output may be watermarked using one or a combination of the following: 1. Global - A single watermark for all generated AI output. This could be for example to verify whether the output is machine generated or not. 2. User scope - all output generated for the same user will have the same watermark, different users will get different watermarks. The watermark will tell you that it was machine generated and for which user. 3. Prompt Scope - Every user and / or prompt combination is given a unique watermark so you can know not only which user it is, but also the prompt. Optionally, a timestamp can also be used to disambiguate the same prompt.

[0192] In advantageous instances, associating a watermark with a particular user or a particular prompt or a combination of both allows for tracking how generated content is being used and adds an extra layer of security or customization.

[0193] This is possible because the system is scalable so that individual watermarking of any number of generated AI outputs is possible.

[0194] In an advantageous example, detection or extraction of the watermark may be achieved from the sampled token subset, whether the sampled token subset is obtained continuously or randomly.

[0195] This is possible because the implementation process is such that whether a particular token is a watermark token or not depends only on the token, the watermark to be embedded, and the watermark key. Thus, whether a particular token is a watermark token or not does not depend on one or more previously generated tokens.

[0196] Now, we will list some key features A2-D2. Each feature listed below can be combined with other features A2-D2. Any feature defined below can be combined with any other feature or any other feature.

[0197] Key Feature A1: The process of embedding a digital watermark into "generated AI output" or generated content A computer-implemented process for embedding a watermark in generated content comprises: (a) generating tokens from input data such as user prompts using a generative AI such as an LLM; (b) embedding a digital watermark into the set of generated tokens; Includes.

[0198] Key Feature B1: A process for incorporating watermarks into generated content where the watermark is embedded probabilistically through the selection of tokens in a set of generated tokens. A computer-implemented process for embedding a watermark in generated content comprises: (a) generating tokens from input data such as user prompts using a generative AI such as an LLM; (b) embedding a watermark in the set of generated tokens, the watermark comprising a pattern that is probabilistically embedded through the selection of tokens in the set of generated tokens; Includes.

[0199] Key Feature C1: A process for incorporating a watermark into generated content, such that the watermark can be recovered without prior knowledge of the generation process or other processing used. A computer-implemented process for embedding a watermark in generated content comprises: (a) generating tokens from input data such as user prompts using a generative AI such as an LLM; (b) embedding a watermark in the set of generated tokens so that the watermark can be recovered without prior knowledge of the generation process or other processing used; Includes.

[0200] Key Feature D1: A process for incorporating digital watermarks into generated content, where the count of candidate watermark content can be dynamically scaled. (a) generating tokens from input data such as user prompts using a generative AI such as an LLM; (b) embedding a digital watermark into the set of generated tokens; (a) generating content, wherein a watermark is selected or chosen based on a user prompt or other parameters of the generation; A computer-implemented process for embedding a digital watermark into generated content, comprising: a step of: adding a watermark to a plurality of candidate watermark contents;

[0201] Optional Features Generated content includes text, images, video, and / or audio.

[0202] The process of embedding a digital watermark The watermark is embedded in the generated token set rather than in any metadata or redundant data. A digital watermark pattern is embedded in the watermark token, and it is determined whether the watermark token is assigned so that the token hashes to a value that falls within a predetermined range of the hash space. · The decision as to whether a watermark token should be assigned to a generated token depends only on the generated token itself, the watermark to be embedded, and the secret watermark key. The token is hashed using a cryptographic hash function based on a secret watermark key, so an attacker cannot know in which subspace the token resides. The process includes steps that never or rarely return a watermark token, so that the bins of the hash space contain no values ​​or values ​​close to zero.

[0203] Restoring or extracting digital watermarks from data releases The watermark can be recovered without prior knowledge of the scheme used to generate the token. The watermark is recovered by (a) hashing the content tokens using one or more hash functions, (b) constructing a histogram of hash frequencies in the hash space, and (c) determining the bin that corresponds to the watermark. When multiple hash functions are used, the process includes summing the number of hashes for each bin across all of the hash functions. Each hash function contains a different key. The process includes calculating the probability of obtaining an observed number or fewer hash values ​​in a particular bin when the watermark data release does not contain a watermark corresponding to the particular bin. When the calculated probability is lower than a predefined threshold, the process infers that the data release contains a particular watermark. The process allows recovery of the watermark with only a subset of tokens from the generated content. The process can handle noise such as adding, removing, or modifying tokens in the content. Extracting a watermark is associated with a confidence score related to (a) the presence of the watermark in the content and (b) the likelihood that an under-represented bin in the histogram of token hash counts corresponds to a hash bin (how confident can we be that the watermark is actually present in the content and that an under-represented bin in the histogram of token hash counts is not just a coincidence). The process includes a step of estimating the number of tokens required to recover the watermark. For a given confidence level and amount of noise present in the content tokens, the number of tokens is estimated.

[0204] Watermarking depends on the parameters of the data release The computer-implemented process includes generating content, where a watermark is selected or chosen based on input data parameters, prompts, or user input. The watermark is selected or chosen based on the user issuing the prompt. Each piece of content generated in response to the same input data or user prompt may contain a different watermark. A watermark can represent a pattern that is a unique ID. The process includes dynamically scaling the count of watermark content candidates by updating the hash function. The process includes dynamically scaling the count of potential watermark contents by hashing the token space with a different hash function. · Digital watermarks are associated with timestamps.

[0205] Computer device or system 1. A computing device or system adapted to embed a digital watermark into generated AI output, comprising: (a) Using a generative AI such as LLM to generate tokens from input data such as user prompts; (b) embedding a digital watermark into the generated set of tokens; A device or system comprising a processor configured to:

[0206] User Cases 1. A computing device or system adapted to detect the use of deepfakes, comprising: (a) Accept the Content; (b) hashing the tokens across the entire received content, or across a subset of the received content, using one or more hash functions to construct a hash frequency histogram of the hash space; (c) determining or inferring from the histogram analysis whether the received content contains a watermark and the likelihood that the received content is a deepfake; A device or system comprising a processor configured to:

[0207] Optional Features Deepfake refers to AI-generated content that is watermarked using a computer-implemented process that includes: (a) using a generative AI to generate tokens from input data, such as user prompts; and (b) embedding a watermark in the collection of generated tokens. Detecting the use of deepfakes is linked to a credibility score that relates to the likelihood that the content was created by generative AI. The system is configured to provide a warning or notification when a deepfake is used. The system is configured to provide a warning or notification when a deepfake is not used. The system is configured to present the deepfake along with evaluation criteria related to the deepfake, including a credibility score, input data or user prompts, type of generating AI, and user ID. The system is configured to automatically remove deepfakes from content. The system is configured to detect specific users using deepfakes based on the digital watermark. The system is configured to automatically block certain users. The system is configured to automatically detect the use of deepfakes when content is uploaded to social media, websites, online gaming platforms, or emails. The system is configured to determine or estimate the proportion of media content in social media, or email, or an online gaming system, or a website created by generative AI. The system is configured to detect the use of deepfakes by analyzing only a subset of the generated content.

[0208] A social media system configured to detect the use of deepfakes by detecting digital watermarks embedded in social media content. A system comprising: a processor configured to hash tokens in a subset of social media content using one or more hash functions to construct a histogram of hash frequencies in the hash space; and from analyzing the histogram, determine or infer whether the social media content is watermarked and the likelihood that the social media content contains a deepfake.

[0209] An email system configured to detect the use of deepfakes by detecting digital watermarks embedded in email content. A system comprising: a processor configured to hash tokens in a subset of email content using one or more hash functions to construct a histogram of hash frequencies in the hash space; and from analyzing the histogram, determine or infer whether the email content is watermarked and the likelihood that the email content contains a deepfake.

[0210] An online gaming system configured to detect the use of deepfakes by detecting digital watermarks embedded in online gaming content. A system comprising a processor configured to hash tokens in a subset of online game content using one or more hash functions to construct a histogram of hash frequencies in the hash space, and to determine or estimate from analysis of the histogram whether the online game content is embedded with a digital watermark and the likelihood that the online game content contains a deepfake.

[0211] A web browser system configured to detect the use of deepfakes by detecting digital watermarks embedded in website content. A system comprising: a processor configured to hash tokens in a subset of website content using one or more hash functions to construct a histogram of hash frequencies in the hash space; and, from an analysis of the histogram, determine or infer whether the website content has been watermarked and the likelihood that the website content contains a deepfake.

[0212] Note It is to be understood that the above-referenced arrangements are merely illustrative of the application of the principles of the present invention. Numerous modifications and alternative arrangements can be devised without departing from the spirit and scope of the invention. While the invention has been shown in the drawings and described above in detail with particularity and detail in connection with what are presently considered to be the most practical and preferred embodiments of the invention, it will be apparent to those skilled in the art that numerous modifications can be made thereto without departing from the principles and concepts of the invention presented herein. [Explanation of symbols]

[0213] 33A Input ordinal 33B Token QC 71 Input Space 72 inputs 73 Token Space 74 Light Grayscale Colors 75 inputs 76 inputs 111 Input 112 inputs 141 Empty Bottle 151 Empty Bottle 161 Watermark Bin

Claims

1. (a) generating tokens from input data such as user prompts using a generation AI such as an LLM; (b) embedding a watermark into the set of generated tokens; 1. A computer-implemented process for embedding a digital watermark in generated content, comprising:

2. The computer-implemented process of claim 1 , wherein the watermark comprises a pattern that is probabilistically embedded in the generated set of tokens through token selection.

3. 10. The computer-implemented process of any preceding claim, wherein the watermark can be recovered without prior knowledge of the generation process or any other process used.

4. 10. A computer-implemented process according to any preceding claim, comprising generating watermark content, wherein the watermarks are selected or chosen based on user prompted or other generation parameters, and the count of candidate watermark content may be dynamically scaled.

5. 10. The computer-implemented process of any preceding claim, wherein the generated content comprises text, images, video, and / or audio.

6. 10. The computer-implemented process of any preceding claim, wherein the watermark is embedded in the generated set of tokens rather than in any metadata or redundant data.

7. 10. The computer-implemented process of any preceding claim, wherein a digital watermark pattern is embedded in a watermark token and the token is assigned or determined such that the watermark token has a hash value that falls within a predetermined range of a hash space.

8. The computer-implemented process of any preceding claim, wherein the determination of whether a generated token should be assigned to a watermark token is dependent only on the generated token itself, the watermark to be embedded, and the secret watermark key.

9. 10. The computer-implemented process of any preceding claim, wherein hashing the token based on a secret watermark key is performed using a cryptographic hash function so that an attacker cannot know in which subspace the token resides.

10. 10. The computer-implemented process of any preceding claim, wherein the process includes a step of never returning or rarely returning the watermark token such that a bin of the hash space contains no value or a value close to zero.

11. 10. The computer-implemented process of any preceding claim, wherein the watermark can be recovered without prior knowledge of the scheme used to generate the token.

12. 10. The computer-implemented process of any preceding claim, wherein the watermark is recovered by (a) hashing the tokens of the content using one or more hash functions; (b) constructing a histogram of hash frequencies of the hash space; and (c) determining the bin that corresponds to the watermark.

13. 10. The computer-implemented process of any preceding claim, wherein when multiple hash functions are used, the process includes summing the hash numbers for each bin across all of the hash functions.

14. The computer-implemented process of any preceding claim, wherein each hash function comprises a different key.

15. 10. The computer-implemented process of any preceding claim, a process including calculating the probability of obtaining the number of hashes observed in a particular bin or less when the watermark data release does not contain the digital watermark corresponding to that particular bin.

16. 10. The computer-implemented process of any preceding claim, wherein the process infers that a data release contains a particular watermark when the calculated probability is below a predetermined threshold.

17. 10. A computer-implemented process according to any preceding claim, the process allowing recovery of the watermark on only a subset of tokens from the generated content.

18. 10. A computer-implemented process according to any preceding claim, wherein the process is capable of dealing with noise such as adding, removing or modifying tokens in the content.

19. 10. The computer-implemented process of any preceding claim, wherein the extraction of the digital watermark is associated with a confidence score related to (a) the presence of a watermark in the content and (b) the likelihood that an under-represented bin in a histogram of token hash counts corresponds to the hash bin.

20. 10. The computer-implemented process of any preceding claim, wherein the process includes estimating the number of tokens required to recover the watermark.

21. 10. The computer-implemented process of any preceding claim, wherein the number of tokens is estimated for a given confidence level and amount of noise present in the content tokens.

22. 10. A computer-implemented process according to any preceding claim, comprising generating content, wherein the watermark is selected or chosen based on input data parameters, prompts or a user.

23. 10. The computer-implemented process of any preceding claim, wherein the watermark is selected or chosen based on the prompting user.

24. 10. The computer-implemented process of any preceding claim, wherein each piece of content generated in response to the same input data or user prompt may include a different digital watermark.

25. 10. The computer-implemented process of any preceding claim, wherein the digital watermark represents a pattern that is a unique ID.

26. 10. The computer-implemented process of any preceding claim, the process comprising dynamically scaling a count of candidate watermark contents by updating the hash function.

27. 10. The computer-implemented process of any preceding claim, the process including dynamically scaling the count of candidate watermark contents by hashing the token space with another hash function.

28. 10. The computer-implemented process of any preceding claim, wherein the digital watermark is associated with a timestamp.

29. a computing device or system adapted to embed a watermark in the generated AI output; (a) generating tokens from input data such as user prompts using a generation AI such as an LLM; (b) embedding the digital watermark into the set of generated tokens; A device or system comprising a processor configured to:

30. 1. A computing system adapted to detect the use of deepfakes, comprising: (a) Accept the Content; (b) hashing the tokens in the entire received content or any subset of the received content using one or more hash functions to construct a hash frequency histogram of the hash space; (c) determining or inferring from the analysis of the histogram whether the received content is watermarked and the likelihood that the received content is or contains a deepfake; A device or system comprising a processor configured to:

31. 31. The computing system of claim 30, wherein a deepfake refers to generated AI content that has been watermarked using a computer-implemented process that includes the steps of: (a) using a generating AI to generate tokens from input data such as user prompts; and (b) embedding the watermark in the set of generated tokens.

32. 32. The computing system of any of claims 30-31, wherein detection of the use of deepfakes is associated with a confidence score related to the likelihood that the content was created by generative AI.

33. 33. The computing system of any of claims 30 to 32, wherein the system is configured to provide a warning or notification when a deepfake is used or detected.

34. 34. The computing system of any of claims 30 to 33, wherein the system is configured to provide a warning or notification when a deepfake has not been used.

35. 35. The computing system of any of claims 30 to 34, wherein the system is configured to present a deepfake along with evaluation criteria associated with the deepfake, including a confidence score, input data or user prompts, type of generating AI, and user ID.

36. 36. The computing system of any of claims 30 to 35, wherein the system is configured to automatically remove deepfakes from the content.

37. 37. A computing system according to any one of claims 30 to 36, configured to detect a particular user who has used a deepfake based on the digital watermark.

38. 38. A computing system according to any one of claims 30 to 37, wherein the system is configured to automatically block the particular user.

39. 39. The computing system of any of claims 30 to 38, wherein the system is configured to automatically detect the use of deepfakes when content is uploaded to social media, a website, an online gaming platform, or an email.

40. 40. The computing system of any of claims 30 to 39, wherein the computing system is configured to determine or estimate the proportion of media content created by generative AI in the social media or email or online gaming system or website.

41. 41. The computing system of any of claims 30 to 40, wherein the system is configured to detect the use of deepfakes by analyzing only a subset of the generated content.

42. 1. A social media system configured to detect the use of deepfakes by detecting digital watermarks embedded in social media content, comprising: (a) hashing tokens in any subset of the social media content using one or more hash functions to construct a hash frequency histogram of the hash space; (b) determining or inferring from the analysis of the histogram whether the social media content is watermarked and the likelihood that the social media content contains a deepfake; A system comprising a processor configured to:

43. 1. An email system configured to detect the use of deepfakes by detecting digital watermarks embedded in email content, comprising: (a) hashing tokens of any subset of said email content using one or more hash functions to construct a hash frequency histogram of said hash space; (b) determining or inferring from the analysis of the histogram whether the email content is watermarked and the likelihood that the email content contains a deepfake; A system comprising a processor configured to:

44. 1. An online gaming system configured to detect deepfakes by detecting digital watermarks embedded in online gaming content, comprising: (a) hashing tokens of any subset of the online gaming content using one or more hash functions to construct a hash frequency histogram of the hash space; (b) determining or estimating, from the analysis of the histogram, whether the online game content is embedded with a digital watermark and the likelihood that the online game content contains a deepfake; A system comprising a processor configured to:

45. 1. A web browser system configured to detect the use of deepfakes by detecting digital watermarks embedded in website content, comprising: (a) hashing tokens of any subset of the website content using one or more hash functions to construct a hash frequency histogram of the hash space; (b) determining or inferring from the analysis of the histogram whether the website content is watermarked and the likelihood that the website content contains a deepfake; A system comprising a processor configured to:

Images