Tokenization of arbitrary data types

Abstract

The computing device includes a processor and a machine-readable memory storing instructions. The instructions are executable by the processor to: receive a bit vector representing input data to be tokenized; partition the bit vector into two vector portions; and perform a plurality of rounds of a Feistel network on the two vector portions, each round comprising transforming one vector portion using a table-based hash function that combines a plurality of tokens retrieved from at least one token table.

Description

Background Art

[0001] Businesses (e.g., retailers, professional firms, financial institutions, etc.) may collect, process, and / or store data representing sensitive or confidential information about individuals or businesses. For example, a business website may use a customer's bank account number to conduct a sales transaction. This sensitive data can be protected from unauthorized access using technologies such as encryption and tokenization. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Some embodiments are described with reference to the following figures.

[0003] Figure 1 is a schematic diagram of an example system according to some implementations.

[0004] Figure 2 is a flowchart of an example process according to some implementations.

[0005] Figure 3 is an illustration of example operations according to some implementations.

[0006] Figure 4 is a flowchart of an example process according to some implementations.

[0007] Figures 5A to 5C is an illustration of example operations according to some implementations.

[0008] Figure 6 is an illustration of example operations according to some implementations.

[0009] Figure 7 is a schematic diagram of an example computing device in accordance with some embodiments.

[0010] Figure 8 is an illustration of an example machine-readable medium storing instructions in accordance with some implementations. DETAILED DESCRIPTION

[0011] To protect sensitive or confidential data from inappropriate access, some systems may convert or "tokenize" the sensitive data into tokens (e.g., strings of characters). Some tokenization techniques may include using a token table to map input data elements to tokens and replacing the input data elements with corresponding tokens. However, traditional tokenization techniques may be limited to input data having a defined format. For example, in a traditional tokenization system, a token table may be predefined to convert numeric data elements having a fixed number of digits (e.g., a credit card number). Accordingly, such a traditional tokenization system may not be usable for input data having arbitrary data size and / or format.

[0012] As referenced below Figures 1 to 8As further described, some embodiments can provide improved tokenization for input data of arbitrary size and / or format. In some embodiments, the input data can be processed through multiple rounds of a Feistel network, where each round includes executing a table-based hash function. As used herein, the term "table-based hash function" refers to a function that combines multiple values ​​retrieved from one or more token tables to generate an output value. In some embodiments, the table-based hash function can be applied to uniformly sized portions of the input data, regardless of the specific format of the input data. Accordingly, embodiments can provide an improved tokenization system that can tokenize arbitrary data types in a secure manner.

[0013] Figure 1 A schematic diagram of an example computing device 110 according to some embodiments is shown. Computing device 110 can be, for example, a computer, a portable device, a server, a network device, an appliance, a communication device, etc. In other examples, computing device 110 can be a server rack system including multiple computing modules (e.g., blade servers), network devices, storage devices, power supply components, etc. In other examples, computing device 110 can be a computing cluster, a data center, a distributed system, etc.

[0014] In some embodiments, computing device 110 may include processor 115, memory 120, and machine-readable memory 130. Processor 115 may include a microprocessor, a microcontroller, a processor module or subsystem, a programmable integrated circuit, a programmable gate array, multiple processors, a microprocessor including multiple processing cores, or another control or computing device. Memory 120 may be any type of computer memory (e.g., dynamic random access memory (DRAM), static random access memory (SRAM), etc.).

[0015] In some embodiments, the machine-readable memory 130 may include a non-transitory storage medium such as a hard drive, flash memory, an optical disk, etc. As shown, the machine-readable memory 130 may store the tokenization engine 140 and the token table 150. In some examples, the tokenization engine 140 may be implemented as executable instructions stored in the machine-readable memory 130 (e.g., software and / or firmware). However, the tokenization engine 140 may be implemented in any suitable manner. For example, some or all of the tokenization engine 140 may be hard-coded into circuitry included in the processor 115 and / or the computing device 110. In other examples, some or all of the tokenization engine 140 may be implemented as a web service, etc., on a remote computer (not shown). In another example, the tokenization engine 140 may be implemented in one or more controllers of the computing device 110.

[0016] In one or more embodiments, the tokenization engine 140 may receive input data to be tokenized. For example, the input data may include sensitive or confidential information about an individual or business organization (e.g., name, financial information, medical history, salary, etc.). In some embodiments, the tokenization engine 140 may process the input data through multiple rounds of the Feistel network. Further, in each round of the Feistel network, the tokenization engine 140 may combine multiple values ​​retrieved from the token table 150 by performing a table-based hash function to generate an output value. The functionality of the tokenization engine 140 is described below with reference to examples according to various embodiments. Figures 2 to 8 Further description.

[0017] Now refer to Figure 2 , shows an example process 200 for tokenizing data according to some embodiments. In some examples, process 200 can be performed by Figure 1 The process 200 may be performed by part or all of the tokenization engine 140 shown. The process 200 may be implemented as hardware and / or machine-readable instructions (e.g., software and / or firmware). The machine-readable instructions are stored in a non-transitory computer-readable medium such as an optical, semiconductor, or magnetic storage device. For illustrative purposes, details of the process 200 may be described below with reference to an example operation 300 of a Feistel network according to some embodiments. Figure 3 However, other implementations are possible.

[0018] Block 210 may include receiving a bit vector representing input data to be tokenized. Block 220 may include dividing the bit vector into two vector portions. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may receive an input vector 305 including sensitive data to be protected (eg, a credit card number, a password, etc.). The tokenization engine 140 may divide the input vector 305 into an input portion A 310 and an input portion B 315.

[0019] Block 230 may include performing multiple rounds of the Feistel network on the two vector portions, each round including transforming one vector portion using a table-based hash function that combines multiple tokens retrieved from at least one token table. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may execute multiple rounds of Feistel networks 320A-320N (also collectively referred to as "rounds 320"). Figure 3 As shown in FIG, the first round 320A may include applying a table-based hash function to input portion B 315, and then performing an exclusive-OR operation ("XOR") of the table-based hash function output with input portion A 310 (in FIG. Figure 3 denoted by a plus sign in a circle). Further, the next round 320B includes applying a table-based hash function to the output of the XOR of the first round 320A, and then performing an XOR of the table-based hash function output with input portion B 315. As shown, the Feistel network can repeat a particular number of rounds, and the output value of each round 320 can serve as the input value of the next round 320. After completing rounds 320 of the Feistel network, output portion A 340 and output portion B 345 can be concatenated or otherwise combined to generate output 350. After block 230, process 200 can be completed. Example implementations of the table-based hash function applied in block 230 are described below with reference to Figures 4 to 6 discuss.

[0020] Now refer to Figure 4 , shows an example process 400 for a table-based hash function according to some embodiments. In some examples, process 400 can be performed by Figure 1 140. The process 400 may be implemented as hardware and / or machine-readable instructions (e.g., software and / or firmware). The machine-readable instructions are stored in a non-transitory computer-readable medium such as an optical, semiconductor, or magnetic storage device. For illustrative purposes, the details of the process 400 may be referred to below in the accompanying drawings. Figures 5A to 5C The example operations 500 shown in FIG. Figure 6 The example operation 600 shown in FIG. 6 is described. However, other implementations are possible.

[0021] Block 410 may include dividing the input portion into a plurality of input elements. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may apply a table-based hash function to the input portion B 315 (e.g., in round 320A), which may include dividing the input portion B 315 into elements of a specified size and / or format (e.g., "bytes" comprising 32 bits of binary data). For example, referring to Figure 5A , an input portion (eg, input portion B 315 during round 320A) is illustrated as being divided into bytes of data corresponding to digital values ​​"6," "0," "3," and so on.

[0022] Block 420 may include, for each of the plurality of input elements, retrieving a set of tokens from at least one token table based on the input element. Figure 1 and Figure 5A , the tokenization engine 140 can use the first byte value "6" as an index into two different token tables 510A, 510B. Figure 5AIn the example of , the byte order is indexed into rows, and thus the first byte value is indexed into the first row in the tables 510A, 510B. Further, in Figure 5A In the example of , the numeric value of the byte is indexed into the column identifier, and thus the byte value "6" is indexed into column "6" of tables 510A, 510B. In some embodiments, the retrieved token set includes a specific number (e.g., three) of sequential tokens starting at the index token. Thus, as in Figure 5A As shown in FIG, a first byte value of "6" results in the selection of a set of three sequential tokens (i.e., A 0，6 , A 0，7 and A 0，8 ), and also results in the selection of a set of three sequential tokens from table 510B (i.e., B 0，6 , B 0，7 and B 0，8 ).

[0023] In another example, refer to Figure 5B , the second byte value "0" causes a set of three sequential tokens to be selected from the second row of table 510A (i.e., A 1，0 , A 1，1 and A 1，2 ), and also results in the selection of a second set of three sequential tokens from the second row of table 510B (i.e., B 1，0 , B 1，1 and B 1，2 In yet another example, reference Figure 5C , the third byte value "3" results in the selection of a set of three sequential tokens from the third row of table 510A (i.e., A 2，3 , A 2，4 and A 2，5 ), and also results in the selection of a second set of three sequential tokens from the third row of table 510B (i.e., B 2，3 , B 2，4 and B 2，5 ). In some embodiments, the set of tokens may continue to be selected from the token tables 510A, 510B based on each byte value in the input portion currently being processed by the table-based hash function. Further, in some embodiments, each token retrieved from the tables 510A, 510B may be a byte value (e.g., 32 bits of data). Note that reference Figures 5A to 5C , retrieving three tokens per input element is merely one example embodiment, and other embodiments are not limited thereto. For example, other embodiments may retrieve four tokens per input element, five tokens per input element, etc. In some examples, the number of tokens retrieved may be a fixed number selected based on the expected length of the output of the table-based hash function.

[0024] Block 430 may include combining the sets of tokens retrieved based on multiple input elements. Figure 1 and Figure 6 , block 610 may represent the set of tokens retrieved by the tokenization engine 140 (at block 420). In the example operation 600, the first row 615A in block 610 represents the six tokens retrieved based on the first byte value "6" (i.e., A 0，6 , A 0，7 , A 0，8 , B 0，6 , B 0，7 , B 0，8 ) in series, as in Figure 5A Further, the second row 615B represents the concatenation of the six tokens retrieved based on the second byte value "0" (as in Figure 5B ), and the third row 615C represents the concatenation of the six tokens retrieved based on the third byte value "3" (as shown in Figure 5C ). Additionally, the remaining rows 615D can represent additional sets of tokens retrieved from the token tables 510A, 510B based on other byte values ​​included in the input portion currently being processed by the table-based hash function. In some embodiments, the remaining rows 615D can also represent sets of tokens retrieved from token tables other than the token tables 510A, 510B. For example, the token tables 510A, 510B can have fewer rows than the number of byte values ​​in the input portion, and thus, indexing of byte values ​​can continue at rows of additional token table pairs once the available rows of the token tables 510A, 510B have been used during indexing.

[0025] As in Figure 6 As shown in , in some embodiments, the example operation 600 can include performing an XOR operation 620 to combine the tokens in each row of block 610. For example, the XOR operation 620 can combine the first tokens in each row to generate a first output token, combine the second tokens in each row to generate a second output token, and so on. Accordingly, the output of the XOR operation 620 can be a set of six concatenated tokens (e.g., six concatenated byte values). Note that the output of six tokens from the XOR operation 620 is merely an example embodiment, and other implementations are not limited thereto. For example, other implementations can generate an XOR output of eight tokens, ten tokens, etc.

[0026] Reference again Figure 4 , block 440 may include applying a secure message digest function to the set of combined tokens to obtain a table-based hash value. For example, referring to Figure 6, a secure message digest function (SMDF) 630 can be applied to the output of the XOR operation 620. In some examples, the SMDF 630 can include a hash-based message authentication code (HMAC) using a secure hash algorithm (SHA), such as HMAC-SHA256, HMAC-SHA3, etc. In some embodiments, the SMDF 630 can be applied in multiple iterations, with the first iteration comprising applying the SMDF 630 to the concatenation of the prefix value and the output of the XOR operation 620, and subsequent iterations comprising applying the SMDF 630 to the concatenation of the output of the first iteration with a different integer value. XOR operation 620. In such embodiments, the final output of operation 600 can include the concatenation of the outputs of multiple iterations of the SMDF 630. Further, in some examples, the number of iterations of the concatenation can be selected to obtain a desired data length for the final output of operation 600. Referring again to Figure 4 , after block 440, process 400 may be complete. In some examples, the output of process 400 and / or operation 600 may be generally consistent with the output of Figure 3 3 corresponds to the output of the table-based hash function applied in each round 320 as illustrated in FIG.

[0027] Now refer to Figure 7 , shows a schematic diagram of an example computing device 700. In some examples, the computing device 700 can generally be used with Figure 1 As shown, computing device 700 may include a hardware processor 702 and a machine-readable storage medium 705. Machine-readable storage medium 705 may be a non-transitory medium and may store instructions 710-730. Instructions 710-730 may be executed by hardware processor 702.

[0028] Instruction 710 may be executed to receive a bit vector representing input data to be tokenized. Instruction 720 may be executed to divide the bit vector into two vector parts. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may receive an input vector 305 and may divide the input vector 305 into an input portion A 310 and an input portion B 315 .

[0029] Instructions 730 may be executed to perform multiple rounds of a Feistel network on two vector portions, each round comprising transforming one vector portion using a table-based hash function that combines multiple tokens retrieved from at least one token table. For example, referring to Figure 1 and Figure 3, the tokenization engine 140 may execute multiple rounds of the Feistel network 320, wherein each round 320 includes applying a table-based hash function to an input portion and performing an XOR of the table-based hash function output with another input portion. Figure 4 ) and / or operation 600 (shown in Figure 6 ) can usually be combined with Figure 3 Corresponding to the example implementation of applying a table-based hash function in FIG.

[0030] Now refer to Figure 8 , shows a machine-readable medium 800 storing instructions 810-830 according to some embodiments. The instructions 810-830 may be executed by any number of processors (e.g., Figure 1 The machine-readable medium 800 may be a non-transitory storage medium such as an optical, semiconductor, or magnetic storage medium.

[0031] Instruction 810 may be executed to receive a bit vector representing input data to be tokenized. Instruction 820 may be executed to divide the bit vector into two vector parts. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may receive an input vector 305 and may divide the input vector 305 into an input portion A 310 and an input portion B 315 .

[0032] Instructions 830 may be executed to perform multiple rounds of a Feistel network on two vector portions, each round comprising transforming one vector portion using a table-based hash function that combines multiple tokens retrieved from at least one token table. For example, referring to Figure 1 and Figure 3 , the tokenization engine 140 may perform multiple rounds of the Feistel network 320 , where each round 320 includes applying a table-based hash function to an input portion and performing an XOR of the table-based hash function output with another input portion.

[0033] As above reference Figures 1 to 8 As described above, some embodiments can provide improved tokenization for input data of arbitrary size and / or format. In some embodiments, the input data can be processed through multiple rounds of a Feistel network, where each round includes executing a table-based hash function. The table-based hash function can be applied to uniformly sized portions of the input data, regardless of the specific format of the input data. Accordingly, embodiments can provide an improved tokenization system that can tokenize arbitrary data types in a secure manner.

[0034] Note that although the above reference Figures 1 to 8Various example embodiments have been described, but the embodiments are not limited to these examples. Figures 2 to 6 Some of the operations shown in can be excluded and / or performed in a different order. In another example, Figures 2 to 6 The process shown in may include additional operations (eg, data formatting, length adjustment and / or padding, format conversion, etc.).

[0035] Data and instructions are stored in corresponding storage devices, which are implemented as one or more computer-readable or machine-readable storage media. Storage media include various forms of non-transitory memory, including semiconductor memory devices such as dynamic or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory; magnetic disks such as fixed disks, floppy disks, and removable disks; other magnetic media including magnetic tape; optical media such as compact disks (CDs) or digital video disks (DVDs), or other types of storage devices.

[0036] Note that the instructions discussed above may be provided on one computer-readable or machine-readable storage medium, or alternatively, may be provided on multiple computer-readable or machine-readable storage media distributed in a large system that may have multiple nodes. Such computer-readable or machine-readable storage media are considered part of an article of manufacture (or article of manufacture). An article of manufacture or article of manufacture may refer to any manufactured single component or multiple components. The storage medium or media may be located either in the machine that runs the machine-readable instructions or at a remote site from which the machine-readable instructions may be downloaded over a network for execution.

[0037] In the foregoing description, numerous details are set forth to provide an understanding of the subject matter disclosed herein. However, embodiments may be implemented without some of these details. Other embodiments may include modifications and variations to the above details.

Claims

1. A computing device comprising: Hardware processor; as well as a machine-readable memory storing instructions executable by the processor to: receiving a bit vector representing input data to be tokenized; dividing the bit vector into two vector parts; as well as performing a plurality of rounds of a Feistel network on the two vector portions, each round comprising transforming one of the vector portions using a table-based hash function that combines a plurality of tokens retrieved from at least one token table, wherein performing the plurality of rounds of the Feistel network comprises repeatedly performing a specific number of rounds of the Feistel network with the output value of each round serving as the input value of the next round, and wherein the instructions for performing the plurality of rounds of the Feistel network comprise instructions for performing the following operations for each round: dividing the one vector portion into a plurality of input elements; for each of the plurality of input elements, retrieving a plurality of tokens from the at least one token table based on the input element; and The plurality of tokens retrieved based on the plurality of input elements are combined.

2. The computing device according to claim 1, wherein: The instructions for executing the multiple rounds of the Feistel network include instructions for performing the following operations for each round: A secure message digest function is applied to the combined multiple tokens.

3. The computing device according to claim 1, wherein: The instructions for combining the plurality of tokens include instructions for performing the following operations: An exclusive OR (XOR) operation is performed to combine the plurality of tokens.

4. The computing device according to claim 1, wherein: The instructions for retrieving a plurality of tokens from the at least one token table based on the input element include instructions for: retrieving a first set of tokens from a first token table based on the numeric value of the input element; as well as A second set of tokens is retrieved from a second token table based on the numeric value of the input element.

5. The computing device according to claim 1, wherein: Each input element of the plurality of input elements comprises a byte of binary data.

6. The computing device of claim 1 , the instructions being executable by the processor to: The two output parts of the multi-round Feistel network are connected in series to generate a combined output of the multi-round Feistel network.

7. A computer-implemented method comprising: receiving, by a tokenization engine, a bit vector representing input data to be tokenized; dividing, by the tokenization engine, the bit vector into two vector portions; as well as performing, by the tokenization engine, a plurality of rounds of a Feistel network on the two vector portions, each round comprising transforming one vector portion using a table-based hash function that combines a plurality of tokens retrieved from at least one token table, wherein performing the plurality of rounds of the Feistel network comprises repeatedly performing a specific number of rounds of the Feistel network, with the output value of each round serving as the input value of the next round, and Among them, for each round: dividing the one vector portion into a plurality of input elements; for each of the plurality of input elements, retrieving a plurality of tokens from the at least one token table based on the input element; and The plurality of tokens retrieved based on the plurality of input elements are combined.

8. The computer-implemented method of claim 7 , comprising, for each round: A secure message digest function is applied to the combined multiple tokens.

9. The computer-implemented method of claim 7, wherein: Combining the plurality of tokens includes performing an exclusive OR (XOR) operation on the plurality of tokens.

10. The computer-implemented method of claim 7, wherein: Retrieving the plurality of tokens comprises: retrieving a first set of tokens from a first token table based on the numeric value of the input element; and A second set of tokens is retrieved from a second token table based on the numeric value of the input element.

11. The computer-implemented method of claim 7, wherein: Each input element of the plurality of input elements comprises a byte of binary data.

12. The computer-implemented method of claim 7, comprising: Two output parts of the multi-round Feistel network are connected in series to obtain a combined output of the multi-round Feistel network.

13. A non-transitory machine-readable storage medium storing instructions that, when executed, cause a processor to: receiving a bit vector representing input data to be tokenized; dividing the bit vector into two vector parts; and performing a plurality of rounds of a Feistel network on the two vector portions, each round comprising transforming one vector portion using a table-based hash function that combines a plurality of tokens retrieved from at least one token table, wherein The performing of multiple rounds of the Feistel network includes repeatedly performing a specific number of rounds on the Feistel network, and the output value of each round is used as the input value of the next round, and The instructions further cause the processor to, for each round: dividing the one vector portion into a plurality of input elements; for each of the plurality of input elements, retrieving a plurality of tokens from the at least one token table based on the input element; and The plurality of tokens retrieved based on the plurality of input elements are combined.

14. The non-transitory machine-readable storage medium of claim 13, wherein: The instructions for executing the multiple rounds of the Feistel network include instructions for performing the following operations for each round: A secure message digest function is applied to the combined multiple tokens.

15. The non-transitory machine-readable storage medium of claim 13, wherein: The instructions for combining the plurality of tokens include instructions for performing the following operations: An exclusive OR (XOR) operation is performed to combine the plurality of tokens.

16. The non-transitory machine-readable storage medium of claim 13, wherein: The instructions to retrieve a plurality of tokens from the at least one token table based on the input element include instructions to: retrieving a first set of tokens from a first token table based on the numeric value of the input element; as well as A second set of tokens is retrieved from a second token table based on the numeric value of the input element.

17. The non-transitory machine-readable storage medium of claim 13, the instructions being executable by the processor to: The two output parts of the multi-round Feistel network are connected in series to generate a combined output of the multi-round Feistel network.

Images