Personal Data Protection
A three-zone data storage system with UUID-based personal data keys addresses the challenge of maintaining data security and flexibility for future processing, preventing unauthorized disclosure and ensuring privacy in data handling.
Patent Information
- Application Number
- JP2024523637
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-10-21
- Filing Date
- 2022-10-18
- Publication Date
- 2024-11-06
AI Technical Summary
Data processors face challenges in maintaining personal data security and flexibility to support future pseudonymization methods while ensuring privacy, as existing normalization, encryption, and hashing techniques are not future-proof and can lead to unauthorized disclosure.
Implementing a system with three data storage zones - general, private, and transformation zones - and using personal data keys generated through UUIDs with cryptographic salts to ensure secure data processing and future compatibility.
The system effectively prevents unauthorized disclosure and supports future data processing by using unique, randomly generated personal data keys, ensuring privacy and flexibility in data handling.
Smart Images

Figure 2024540941000001_ABST
Abstract
Description
[Technical field]
[0001] In a data privacy environment, a data controller is a party that controls the purposes and means for which personal data are processed in the digital environment. A data processor is a party that processes personal data on behalf of a data controller. Personal data means any information relating to an identified or identifiable natural person. An identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier, such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person. Data controllers can use personal data to customize website activities, tailor content to a given individual, select advertising based on personal preferences, and perform many other useful functions. However, in order to protect the privacy of the individuals to whom this data relates, all personal data must be kept secure and released only to authorized recipients. [Background technology]
[0002] In almost all cases, applicable privacy laws and regulations require that original (i.e. raw) personal data must be kept only by the original data controller or data processor and only for valid business purposes in order to protect privacy. Therefore, in order to use personal data for customization purposes, the raw personal data must be transformed to generate pseudonymous data that cannot be traced back to the original data subject (i.e. person) associated with the personal data. Pseudonymous data is defined as data that no longer allows for the identification of the data subject associated with the personal data, without additional information that is kept separate from the pseudonymous data. This differs from anonymous data, which is data that is processed in such a way as to remove all possibility of reversibly identifying the data subject, regardless of the presence of any additional information that may be combined with the data.
[0003] Various means are used to process personal data, including but not limited to canonicalization, encryption and cryptographic hashing. These processes have numerous variations and generate pseudonymous data that is exchanged between data controllers, data processors, websites, advertisers and other entities. As an illustrative example of canonicalization, consider the email address "John.O'Shea@mail.com". Depending on the data processing, the canonicalized email address might be "johnoshea@mail.com", "john.oshea@mail.com", "john.o'shea@mail.com" or other variations. The canonicalized email address is then further processed, for example, by encryption or cryptographic hashing. Encryption and hashing also have variants, for example, using different encryption keys, hashing / encryption techniques and encryption salts. The final value of this type of processing is considered pseudonymous and is exchangeable for customization purposes without loss of privacy.
[0004] As can be seen from this example, different pseudonymization methods are possible. If data is normalized by one method to create pseudonymized data and the original data is destroyed, the data processor cannot process requests using data that would have been normalized differently during pseudonymization. Data processors, whether it is existing technology or one that may be introduced in the future, may find it advantageous to retain the data in a form that makes it usable regardless of the normalization method applied. This increases the data processor's processing flexibility while still protecting the privacy of the associated data. Thus, data processors may be faced with the following simultaneous problems: (1) how to retain personal data in a manner that minimizes the possibility of unauthorized disclosure, (2) how to support future pseudonymization methods that differ from current methods, and (3) how to generate files that can be processed under normal computing environments and that support the principle of "privacy by design". What is then required is a way to protect the privacy of personal data while also allowing future processing of the data by methods that are not known when the personal data are received.
[0005] Nothing mentioned in this background section is admitted to be prior art with respect to the present invention. Summary of the Invention [Means for solving the problem]
[0006] The present invention relates to a system and method that protects the privacy of personal data while allowing future processing of the data in a manner that is not known when the personal data is received. The personal data can be securely stored, preventing unauthorized disclosure of the data. This is accomplished by creating a reference to each individual data item, which can be used to identify a combination of processing and representation, using the original personal data as input to the specified processing. In various embodiments, the system and method may include a process for extracting raw personal data and inserting the data into a secure database, a process for assigning a primary key to the personal data for retrieval, a personal data repository maintained in a high security zone and with management policies and personnel access restrictions to prevent visibility of the personal data, a service for retrieving the processed (e.g., normalized, encrypted, and / or hashed) personal data, and a parsed record file containing attributes and the personal data key.
[0007] These and other features, objects and advantages of the present invention will be better understood from consideration of the following detailed description in conjunction with the drawings described below. [Brief description of the drawings]
[0008] [Figure 1] FIG. 2 is a diagram of a personal data workflow according to one embodiment of the present invention. [Diagram 2] FIG. 2 is an architectural diagram of hardware implementing personal data workflow according to one embodiment of the present invention. [Diagram 3] FIG. 13 is a detailed diagram of data flow for a conversion zone according to one embodiment of the present invention. [Figure 4] FIG. 1 is an architectural diagram of hardware implementing data flow for a conversion zone according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Before the present invention is described in greater detail, it is to be understood that the scope of the present invention is limited only by the claims, and therefore is not limited to the specific embodiments described, and that the terminology used in describing the specific embodiments is for the purpose of describing those specific embodiments only and is not intended to be limiting.
[0010] The processes and methods for certain embodiments of the invention described herein utilize three data storage zones. These zones may be implemented on individual local storage media, with physical or logical separation, or across one or more cloud storage media. The zones are referred to herein as the general zone, the private zone, and the transformation zone. The general zone applies general security and is private within the company, but is generally visible to the provider and developers. Personal data is not allowed in the general zone. The private zone also applies general security and is private within the company, but access is limited to the company's technical services and a subset of developers with a need to know. The private zone is used to accept and process customer files, including personal data. Files are temporary in this zone, and all personal data is deleted as soon as the files have completed ingest processing. Finally, the transformation zone is the highest security zone, restricted to a small set of company personnel trained by policy. The network providing access to the transformation zone is restricted to authorized traffic only, with no external access allowed. The transformation zone contains highly sensitive data such as encryption keys.
[0011] Corporate management policies for data security may include minimizing access to data. In an embodiment of the present invention, specific organizations within the provider company are designated to maintain systems housing personal data such that access is denied to people and organizations that do not need access to the systems. Additionally, people within the designated organizations are specially trained in data security, and these people must legally protect the confidentiality of the data under their control.
[0012] With reference to the Zone description provided below, a general personal data workflow overview according to one embodiment of the invention may be described with reference to Figure 1. In step 1, an external data controller creates files containing records with raw personal data. These files are made available within the Private Zone.
[0013] Ingestion Calculation Engine – At step 26, the ingestion process monitors the arrival of these raw personal data files 1 and processes them. The ingestion engine 26 examines each record of raw personal data input, identifies specific personal data items and passes them to the personal data transformation service 3.
[0014] Next, the Personal Data Transformation Service 3 receives the raw personal data as input. It then creates a personal data key for each item using the Key Management Service 9. Once the personal data keys are created, the Personal Data Transformation Service 3 then stores the key, the personal data and associated metadata in a Personal Data Key Map 10. The Personal Data Key Map 10 is an encrypted database. The personal data keys are returned to the Ingest Engine 26.
[0015] The ingestion engine 26 then receives the personal data key and replaces the original raw items with the personal data key. It then creates a parsed record 4. Once ingestion is complete, all personal data has been removed from the original entries. The parsed record 4 is stored for further processing.
[0016] During the next processing, starting from Record Processing step 5, the Record Processing module reads the parsed record containing the Personal Data Key. It contacts the Personal Data Transformation Service 3, retrieves the encrypted, hashed and normalized Personal Data, and replaces the Personal Data Key in the parsed record 4 with these new values.
[0017] The distributed processing 6 then performs one final transformation on the parsed records 4. Each item of encrypted and hashed personal data is encrypted with a key specific to the destination platform. Therefore, the "same" data being sent to different destinations will be different, making it impossible for the two destinations to correlate the encrypted and hashed personal data items.
[0018] Returning to the Personal Data Transformation Service - Step 3, the service performs the encryption referenced above. It first decrypts the encrypted and hashed personal data item using a standard key. It then uses a key management service 9 to derive an encryption key specific to the destination platform. It then encrypts the data item using this destination-specific key.
[0019] Finally, in step 8, the updated parsed records are sent to the destination platform.
[0020] The Personal Data Key is implemented in a particular embodiment with the following characteristics, which are assigned letters for further reference below:
[0021] Property A of personal data keys is that each personal data key should allow for a unique association to an item of personal data. A personal data transformation service does not expose the raw personal data, but provides access to encrypted, hashed, and normalized personal data. The personal data key is a value that the client submits for a request. Thus, the personal data key should uniquely map to an individual piece of personal data.
[0022] Property B of the Personal Data Key is that the Personal Data Key should be implemented so that the analyzed records protect personal identities. The analyzed records are kept in a general zone of relatively low security. They are protected from public exposure but are visible to people within the provider company, which makes the files vulnerable to theft or exposure by bad actors within the company. Therefore, if the analyzed record files are sent outside the company (against company policy), the Personal Data Key should not make it possible to associate specific records with individual people.
[0023] Property C of private data keys is that they should be implemented in a way that prevents systematic probing of the private data key map. This protection addresses two problems. First, it prevents retrieving the map data for a known individual, given that individual. Second, it prevents identifying the individual associated with the data, given the data that is retrieved from the map. Access to the private data key map is limited to authorized users, and the map is protected within a relatively secure transformation zone. However, if a bad actor wishes to exploit that access, the private data key should thwart efforts to associate the data with an individual.
[0024] Property D of personal data keys is that they should support any new canonicalization, hashing and encryption methods and allow existing parsed records to be processed by the new methods.
[0025] A personal data key is generated in a high security transformation zone. The transformation zone components also manage the encrypted personal data and provide normalization and cryptographic hashing. In overview, the creation of a personal data key may proceed with the following steps: First, the raw personal data is presented for encoding, such as "A.Smith". A new universally unique identifier (UUID) is generated for the data. A UUID is a 128-bit label typically used for information in computer systems. UUIDs may be created in multiple ways, including but not limited to concatenation of a MAC address with a timestamp, hashing a namespace identifier, and creation by random number generation. For a large number of potential UUIDs in any given identity space, they are practically unique. An example UUID for "A.Smith" could be "8399d898-b826-11eb-8529-0242ac130003". Once the UUID is created, it is then cryptographically salted. As is well known in cryptography, a salt is random or pseudorandom data that is used as an additional input to a one-way function, such as one that hashes data. In this case, the salted UUID might be "8399d898-b826-11eb-8529-0242ac130003+334573c16f9c2006". The salted UUID is then hashed to produce a personal key. In one specific, non-limiting example, using a SHA-256 hashing technique, the personal key in this case might be "ad3aa2cdabe4781e44950d7077c1f006828167b84e9f8a08a1705749437138ad". As described above, the personal data key replaces the original personal data, transforming the raw personal data file into a parsed record.
[0026] A personal key generated in this way satisfies each of the above mentioned properties. As already mentioned above, the process starts with a personal data item such as the name "A. Smith". Leaving this value in the parsed record would allow the data of the record to be directly associated with the individual, thereby violating privacy. Using raw personal data as a key would not pass property B mentioned above.
[0027] Each UUID is a 128-bit value, approximately 3.4 × 10 38 represents the possibility of about 7.9 billion (7.9 x 10 9 Given a world population of 10,000 (2021), UUIDs offer a lot of capacity to uniquely identify personal data. Since there is no explicit link between UUIDs and personal data, UUIDs pass properties A and B above. On the other hand, UUIDs can be generated in a predictable way, for example, in the following sequence: 3c153032-b8c9-11eb-8529-0242ac130003 3c1532e4-b8c9-11eb-8529-0242ac130003 3c153654-b8c9-11eb-8529-0242ac130003 3c153730-b8c9-11eb-8529-0242ac130003 When UUIDs are used as keys, the similarity of these values may allow an adversary to predict the range of UUIDs. This may in turn allow systematic probing of the personal data-key map with the predicted key values. Bad actors could potentially use this kind of probing to violate privacy, and therefore UUIDs of this nature do not pass property C.
[0028] Applying a cryptographic salt adds randomness to the UUID; specifically, 64 bits are added in the example above. If both the UUID and the salt appear in the actual key, then the salted UUID has the same properties as a raw UUID; that is, if a bad actor can determine the salt, the salt does not add security. Thus, a raw salted UUID satisfies properties A and B, but fails property C.
[0029] Cryptographically hashing salted UUIDs distributes their values across a larger key space. In the example above, the salted UUID is a unique random 192-bit value (roughly 6.3×10 57 With SHA-256, for example, a raw salted UUID creates a 256-bit digest space (roughly 1.2×10 77 , . Assuming that any personal data input is mapped to a salted UUID, the SHA-256 digest generates a unique key for the data. Thus, property A is met. The parsed record contains the SHA-256 digest instead of the personal data. Even if the SHA-256 digest could be parsed (which is not possible with current technology), the root value is a salted UUID, disconnected from the original personal data. Thus, the parsed record itself cannot be used to derive personal data, satisfying property B.
[0030] Testing whether a private data key satisfies property C may be divided into two parts. First, property “Ca” may be considered: given the private data for a known individual, can it be used to find associated data? For example, assume a bad actor owns the name of A. Smith and wants to find other associated information. That requires deriving the private data key (ad3a...38ad) from the well-known data (“A. Smith”). The process of creating the original key generated a UUID and a random salt. Both of these values are extremely difficult to duplicate. Even if the creation time of a UUID can be reduced to one second in time, the bad actor has 163 billion possibilities to consider. Determining a 64-bit random salt is even more difficult. With a secure random number generation, a 64-bit value has 1.8×10 19 Taken together, this gives the bad guys a chance of 2.8×10 30 To put this number into context, the age of the Earth is 4.5 billion years or 1.4×10 17 Therefore, the evildoer would have to evaluate approximately 10 trillion items per second since the creation of the earth in order to check all of the possibilities.
[0031] Alternatively, a bad actor could try to probe the SHA-256 digest value directly, such as through a rainbow table. This approach is also not possible, since salt is an effective countermeasure against rainbow table attacks. The raw UUID + salt is 6.3 x 10 57 These values are 1.2×10 77 The SHA-256 digest is distributed across 1.9 x 10 19 gives roughly one valid digest for every 10 57 Maybe not 10 12 Therefore, the actual private data key is limited to 10 65Therefore, the personal data key satisfies property Ca.
[0032] We can also approach the problem from the opposite direction, with what is referred to herein as property Cb. That is, given that the encrypted, hashed, and normalized personal data is retrieved from the personal data key map, the goal is to prevent that data from being associated with a known individual. This can be done using two approaches. One is to directly attack the encrypted, hashed, and normalized personal data. With current technology, it is not possible to decrypt and analyze the hash of the encoded data. The other approach is to associate the encrypted, hashed, and normalized personal data with a known person, using the personal data key as the vulnerability. This problem solves the same problem as Ca described above. Thus, the personal data key satisfies property Cb.
[0033] With respect to property D, the Personal Data Transformation Service provides all normalization, hashing, and encryption to its clients. Importantly, the creation of the Personal Data Key can be done completely independent of the normalization, hashing, and encryption. Thus, first a parsed record file can be created, and then the Personal Data Transformation Service can be augmented with a new normalization, hashing, or encryption method. The Record Processing Module or Distribution Module can use the old parsed record as input and request a new normalization, hashing, or encryption method. Thus, the Personal Data Key satisfies property D.
[0034] With reference now to FIG. 2, a hardware architecture for an embodiment of a system that performs these processes may be described. An external data controller 20 transfers personal files to a secure private zone 22. The data controllers 22 are operated by external entities (e.g., companies) that run their own computing environments. They are given specific networking permissions and pass data to the private zone 22 through a network firewall 36. These permissions take the form of a login and password for account access plus a network configuration. The network configuration controls electronic access, allowing traffic from the data controller 20 and blocking network traffic from unknown sources. The login and password further vets the data controller 20 and provides a second level of protection from unauthorized access. Only authorized users have a valid login and password combination. Each data controller 20 places personal data files 24 with raw personal data into the physical storage of the private zone 22. These are presented to the ingestion computation engine 26.
[0035] The ingestion computation engine 26 runs on computation engine hardware in the private zone 22. This ingestion computation engine 26 reads the raw personal data files 24, processes each record, replaces the raw personal data items with personal data keys, and transfers these parsed records to physical storage in the parsed records storage system 28 in the public zone 30. When the ingestion computation engine 26 has finished processing the personal data files and the parsed records have been generated, the personal data files 24 are deleted. At that point, all instances of the raw personal data are removed within the domain of the data processor 20.
[0036] FIG. 3 provides a further illustration of steps in a process using the system described above, focusing on the transformation zone 32. In this case, two records containing personal information associated with people “A. Smith” and “B. Jones” are received from a data controller. This information is shown at the data controller input 34. The information in the records further includes an email address and a postal address for each person. At the ingestion calculation engine 26, the output parsed record 28 contains a personal data key for each of these six individual data points. At the transformation zone 32, the personal data transformation service 3 associates the keys stored in the output parsed record with the encrypted data and metadata in the personal data key map 10.
[0037] With reference to FIG. 4, a hardware architecture implementing the transformation zone 32 may be described. Ingestion 26, Recording 5, and Distribution 6 rely on the high security transformation zone 32 for their functions. They are given specific networking permissions and pass data to the transformation zone 32 through a network firewall 36. In certain embodiments, they do not have direct access to the hardware resources of the transformation zone 32. That is, they have no direct access to the personal data key map 10, the key management 9, or the random number generator 34. Furthermore, the transformation zone 32 is protected by multiple levels of security. Physical security controls access to the real world facility. In other words, unauthorized persons are denied physical entry to the data center facility. Network configuration controls electronic access, allowing traffic from known clients and blocking network traffic from unknown sources. In this example, all three clients (Ingestion 26, Recording 5, and Distribution 6) are controlled by the owner of the transformation zone 32. Although they operate outside of the transformation zone 32, unknown clients are denied access to the personal data transformation service 3. Special security training is provided to all personnel with access to the conversion zone 32. Management policies and practices limit access to the conversion zone 32.
[0038] The Personal Data Key Map 10 and Key Management System 9 secure the data stored in the specialized encrypted storage device, so that even if the physical device is stolen (or accessed outside of the security policy) by a malicious actor, the raw personal data and raw encryption keys are not exposed to the malicious actor.
[0039] The Personal Data Transformation Service 3 runs on the computation engine hardware in the transformation zone 32. This component prevents access to raw personal data and provides certain services to clients. First, given personal data, it provides the service of creating a personal data key. This includes generating a personal data key, encrypting the raw personal data, and adding the new key and encrypted data to the personal data key map 10. Second, given a personal data key, it provides the service of computing an encrypted, hashed and normalized value for the associated personal data. Third, given data encrypted with one key, it provides the service of returning the same data encrypted with another key, i.e., converting the encrypted, hashed and normalized data from one key space to another.
[0040] The random salt value plays a crucial role in the generation of the private data key. A specialized hardware random number generator 34 provides the random data used for the cryptographic salt. As with other components of the transformation zone, the random number generator 34 is not accessible from outside the transformation zone 32.
[0041] In certain data privacy implementations, there may be a goal of "data minimization" and data retention limits (explicit or implicit). In other words, a data controller or data processor may want to collect personal data, keep it for a certain period of time based on business needs, and then automatically destroy the data. FIG. 3 shows metadata columns 40 associated with each record in the Personal Data Key Map 10. In a particular, non-limiting example, the metadata may include an expiration value. Depending on the details of the storage hardware and the Personal Data Key Map 10, this expiration may be enforced through hardware or software. Either way, entries in the Personal Data Key Map 10 may be deleted without human intervention, ensuring that data retention limits are respected.
[0042] An alternative embodiment of the invention described above is a private envelope of the analyzed record. This alternative places encrypted personal data in the analyzed record file. The private envelope can have the following structure: [protocol_id][initialization_vector][encrypted_payload] The ciphertext "encrypted_payload" can be passed to a service in transformation zone 32, decrypted, and then processed into encrypted, hashed, and normalized personal data. The underlying data can be interpreted as raw personal data or possibly an anonymous identifier. Either way, both types of raw payload are generally considered personal data for this purpose.
[0043] In the private envelope approach, given a parsed record to process, the workflow invokes the Personal Data Transformation Service 3 in the secure transformation zone 32. The service receives the encrypted personal data, decrypts it, applies the selected operation, generates hashed personal data for downstream platforms, and returns the hashed personal data to the caller. All encryption and decryption operations occur in the secure transformation zone 32. The encryption keys reside only in the transformation zone 32 and are managed by the secure key management service 9, possibly with key rotation. The new parsed record exposes the encrypted personal data alone (optionally, this can be doubly encrypted to further improve security). The new record allows for flexible normalization and recovering the raw personal data for the next hash / encryption.
[0044] The advantage of the private envelope approach is that new normalization, hashing and encryption requirements can be supported by updating the Personal Data Transformation Service 3. Since the Personal Data Transformation Service 3 can recover the original, raw personal data, all files previously ingested under this alternative are available for the new destination. The customer can request delivery to a new platform without having to resend previously ingested files. Furthermore, the customer does not need to pre-declare the destination platform the customer was planning to use.
[0045] Another advantage of the private envelope approach is that it uses a single representation of each personal data item in the analyzed record file. Duplication of the "same" data with different encodings does not occur.
[0046] A potential drawback of the private envelope approach is that it inserts encrypted personal data directly into the analyzed records. According to certain embodiments of the present invention, those analyzed record files are stored in the general zone 30. Despite the difficulty of breaking modern encryption, making data directly accessible is a potential concern.
[0047] Another potential drawback of the private envelope approach is that encrypted personal data is still considered personal data under certain standards, meaning that in this alternative, analyzed records may be geographically restricted by applicable laws and regulations. This means that all processing of analyzed records must occur in a specific geographic location; i.e., onboarding, workflow, and distribution must occur in the same geographic area.
[0048] Yet another potential drawback of the private envelope approach is that putting encrypted personal data directly into analysed records may also be inconsistent with customer expectations: the principles of "privacy by design" and "data minimisation" influence the commercial decisions of data providers, and the direct use of encrypted personal data may have negative consequences as a result.
[0049] Another alternative embodiment of the present invention uses pre-computed encrypted hashed personal data in the parsed record. This alternative reuses the basic parsed record framework, but it inserts duplicate values for the hashed personal data fields. Given a set of normalization rules and hashing techniques, the ingest applies all normalizations and all hashes, generating multiple hash values for each personal data input value. All "duplicate" values are added to the parsed record file.
[0050] For normalization, there are multiple rules to apply, with different rules for different types of personal data. For example, an email address might undergo the following hypothetical normalization: J.O'Shea@mail.com → joshea@mail.com (lowercase, no dots, no punctuation) J.O'Shea@mail.com → j.oshea@mail.com (lowercase, no punctuation) J.O'Shea@mail.com → JOShea@mail.com (case preserved, no dots, no punctuation) J.O'Shea@mail.com → j.o'shea@mail.com (lowercase only) Hashing options may include, by way of non-limiting example, SHA-1, SHA-256, and MD5 techniques. Thus, in this example, a single email address in the input file may produce 4 x 3 = 12 (i.e., 4 normalizations, 3 methods of each hash) hashed email values in the parsed records.
[0051] The advantage of this pre-computation approach is that it uses existing security and data ethics frameworks. It also extends the current parsed record format with minimal disruption to existing onboarding components. Finally, the resulting hashed personal data in the parsed record can be directly used to prepare the distribution file 6. However, it is still desirable for the personal data transformation service 3 to manage the normalization, hashing and encryption operations in the transformation zone 32.
[0052] A potential drawback of the pre-computation approach is that when a new operation is added, it cannot use old parsed records; i.e., if a new destination requires a new canonicalization operation, the data in the existing parsed records cannot be used. The original raw personal data is not available, and new canonicalized hash values cannot be created from the existing data. The same problem applies when dealing with integration problems: if an exchange with a distributed platform does not work and the provider needs to revise its canonicalization and hashes, the input data must be redelivered and reprocessed.
[0053] Another potential drawback of the pre-computation approach is that the storage of analyzed records is expanded and duplicate hashed personal data values are added, however this can be reduced at the cost of limiting future use for additional downstream platforms.
[0054] Yet another potential drawback of the pre-computation approach is that the audience store will grow with duplicate hashed personal data values (the audience store is managed by Data Access). Currently hashed personal data is treated as an anonymous identifier, and the anonymous identifiers of the parsed record files are kept in the audience store. This is potentially more significant than parsed record storage, since the audience store is kept indefinitely. Furthermore, increasing the size of the audience store adds read / write overhead for the associated processing that occurs with each use of the store.
[0055] Yet another potential drawback of the pre-computing approach is that it does not support processing in certain jurisdictions that have stricter privacy laws and regulations.
[0056] Yet another potential drawback of the pre-computation approach is that when a provider prepares parsed records with duplicate hashed values for distribution, it needs to omit irrelevant values from the distribution file for a particular platform. For example, when sending data to company A, it needs to omit all non-company A hashed personal data.
[0057] Yet another potential drawback of the pre-computation approach is that hashed personal data objects must characterize their internal values, i.e., the type of personal data, the normalization applied, and the hashing technique used. Objects according to certain embodiments of the present invention have implicit normalization rules; object definitions must explicitly specify the normalization behavior.
[0058] The systems and methods described herein may be implemented by any combination of hardware and software in various embodiments. For example, in one embodiment, the systems and methods may be implemented by a computer system or collection of computer systems, each including one or more processors executing program instructions stored on a computer-readable storage medium coupled to the processor. The program instructions may perform the functions described herein. The various systems and methods illustrated in the drawings and described herein represent example implementations. The order of steps in the methods may be changed, and various elements may be added, modified, or omitted from the systems.
[0059] The computing system or computing device described herein may be implemented using hardware portions of a cloud computing system or a non-cloud computing system. The computing system may be any of a variety of types of devices, including, but not limited to, a commodity server, a personal computer system, a desktop computer, a laptop or notebook computer, a mainframe computer system, a handheld computer, a workstation, a network computer, a consumer device, an application server, a storage device, a mobile phone, or generally any type of computing node or device. The computing system includes one or more processors (any of which may include multiple processing cores, and the processing cores may be single-threaded or multi-threaded) coupled to a system memory via an input / output (I / O) interface. The computing system may further include a network interface coupled to the I / O interface.
[0060] In various embodiments, the computer system may be a single processor system including one processor or a multiprocessor system including multiple processors. The processor may be any suitable processor capable of executing computing instructions. For example, in various embodiments, the processor may be a general purpose or embedded processor implementing any of a variety of instruction set architectures. In a multiprocessor system, each processor may typically, but not necessarily, implement the same instruction set. The computer system also includes one or more network communication devices (e.g., network interfaces) for communicating with other systems and / or components over a communications network, such as a local area network, a wide area network, or the Internet. For example, a client application executing on the computing device may use the network interface to communicate with a server application executing on a single server or a cluster of servers implementing one or more of the components of the system described herein in a cloud computing or non-cloud computing environment as implemented in the various subsystems. In other instances, an instance of a server application executing on a computer system may use the network interface to communicate with other instances of the application that may be implemented on other computer systems.
[0061] A computing device also includes one or more persistent storage devices and / or one or more I / O devices. In various embodiments, the persistent storage devices may correspond to disk drives, tape drives, solid state memory, other mass storage devices, or any other persistent storage device. A computer system (or a distributed application or operating system running thereon) may store instructions and / or data in the persistent storage devices as desired and may retrieve the stored instructions and / or data as needed. For example, in some embodiments, the persistent storage devices may include solid state drives attached to server nodes. Multiple computer systems may share the same persistent storage device or may share a pool of persistent storage devices, where the devices in the pool represent the same or different storage technologies.
[0062] A computer system includes one or more system memories that may store code / instructions and data accessible by a processor. System memories may include, for example, multiple levels of memory and memory caches in systems designed to swap information in memory based on access speed. Interleaving and swapping may be extended to persistent storage in virtual memory implementations. Technologies used to implement memory may include, for example, static random-access memory (RAM), dynamic RAM, read-only memory (ROM), non-volatile memory, or flash type memory. With respect to persistent storage, multiple computer systems may share the same system memory or may share a pool of system memory. One or more system memories may include program instructions executable by a processor to implement the routines described herein. In various embodiments, the program instructions may be coded in binary, assembly language, any interpreted language such as Java, compiled language such as C / C++, or any combination thereof. The specific languages shown herein are merely illustrative. In some embodiments, the program instructions may implement multiple separate client, server nodes and / or other components.
[0063] In some embodiments, the program instructions may include instructions executable to implement an operating system, which may be any of a variety of operating systems, such as UNIX, LINUX, MacOS, or Microsoft Windows. Any or all of the program instructions may be provided as a computer program product or software, which may include a non-transitory computer-readable storage medium having instructions stored thereon, which may be used to program a computer system (or other electronic device) to perform a process according to various embodiments. A non-transitory computer-readable storage medium may include any mechanism for storing information in a form (e.g., software) readable by a machine (e.g., a computer). Generally speaking, a non-transitory computer-accessible medium may include computer-readable storage media or memory media, such as magnetic or optical media, e.g., disks or DVDs / CD-ROMs coupled to a computer system via an I / O interface. A non-transitory computer-readable storage medium may include any volatile or non-volatile medium, such as RAM or ROM, that may be included in some embodiments of a computer system as system memory or other type of memory. In other embodiments, the program instructions may be implemented via a network interface, such as by communication using optical, acoustic or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.) carried over a communication medium, such as a network and / or wired or wireless link. The network interface may be used to interface with other devices, which may include other computer systems or any type of external electronic device.In general, system memory, persistent storage, and / or remote storage accessible in other devices over a network may store data blocks, copies of data blocks, metadata associated with the data blocks and / or their state, database configuration information, and / or any other information usable in performing the routines described herein.
[0064] In certain embodiments, the I / O interface may coordinate I / O traffic between the processor, system memory, and any peripheral devices in the system, including via a network interface or other peripheral interface. In some embodiments, the I / O interface may perform any necessary protocol, timing, or other data conversion to convert data signals from one component (e.g., system memory) into a format suitable for use by another component (e.g., processor). In some embodiments, the I / O interface may include support for devices attached via various types of peripheral buses, such as variants of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard. Also, in some embodiments, some or all of the functionality of the I / O interface, such as the interface to system memory, may be incorporated directly into the processor.
[0065] The network interface may enable data to be exchanged between the computer system and other devices attached to the network, such as, for example, other computer systems (which may implement one or more storage system server nodes, main nodes, read-only nodes, and / or clients of the database system described herein). In addition, the I / O interface may enable communication between the computer system and various I / O devices and / or remote storage devices. The input / output devices may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other device suitable for inputting or retrieving data by one or more computer systems. These may be directly connected to a particular computer system or generally connected to multiple computer systems in a cloud computing environment or other system including multiple computer systems. The multiple input / output devices may be in communication with the computer system or may be distributed among various nodes of a distributed system including the computer system. The user interfaces described herein may be visible to users using various types of display screen technologies. In some embodiments, input may be received through a display using touch screen technology, while in other embodiments, input may be received through a keyboard, mouse, touchpad, or other input technology, or any combination of these technologies.
[0066] In some embodiments, the similar input / output devices may be separate from the computer system and may interact with one or more nodes of a distributed system including the computer system via a wired or wireless connection, such as over a network interface. The network interface may generally support one or more wireless networking protocols (e.g., Wi-Fi / IEEE 802.11 or other wireless networking standards). The network interface may support communication over any suitable wired or wireless general data network, such as, for example, other types of Ethernet networks. In addition, the network interface may support communication over a telecommunications / telephony network, such as an analog voice network or a digital fiber communications network, over a storage area network, such as a Fibre Channel Storage Area Network (SAN), or over any other suitable type of network and / or protocol.
[0067] Any of the embodiments of the distributed system described herein or any of their components may be implemented as one or more network-based services in a cloud computing environment. For example, read-write and / or read-only nodes in a database tier of a database system may provide database services and / or other types of data storage services to clients as network-based services using the distributed storage system described herein. In some embodiments, the network-based services may be implemented by software and / or hardware systems designed to support interoperable machine-to-machine interaction over a network. A web service may have an interface described in a machine-processable format, such as Web Services Description Language (WSDL). Other systems may interact with the network-based service in a manner specified by the description of the network-based service's interface. For example, the network-based service may define various operations that other systems may invoke and may define specific application programming interfaces (APIs) that other systems may be expected to follow when requesting various operations.
[0068] In various embodiments, a network-based service may be requested or invoked through the use of a message that includes parameters and / or data associated with a network-based service request. Such a message may be formatted according to a particular markup language, such as Extensible Markup Language (XML), and / or may be encapsulated using a protocol, such as Simple Object Access Protocol (SOAP). To make a network-based service request, a network-based service client may assemble a message that includes the request and convey the message to an addressable endpoint (e.g., a Uniform Resource Locator (URL)) corresponding to the web service using an Internet-based application layer transport protocol, such as Hypertext Transfer Protocol (HTTP). In some embodiments, a network-based service may be implemented using Representational State Transfer (REST) technology rather than a message-based technology. For example, a network-based service implemented according to REST technology may be invoked through parameters contained within an HTTP method, such as PUT, GET, or DELETE.
[0069] Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Although any methods and materials similar or equivalent to those described herein can also be used in the practice or testing of the present invention, a limited number of exemplary methods and materials are described herein. It will be apparent to those skilled in the art that many further modifications are possible without departing from the inventive concepts herein.
[0070] All terms used herein should be interpreted in the broadest possible manner consistent with the context. In particular, the terms "comprises" and "comprising" should be interpreted as referring to an element, component, or step in a non-exclusive manner, indicating that the referenced element, component, or step may be present or utilized, or may be combined with other elements, components, or steps not expressly referenced. When groupings are used herein, all individual members of the group and all possible combinations and subcombinations of the group are intended to be individually included in the disclosure. When ranges are stated herein, all subranges within the ranges and all distinct points within the ranges are intended to be individually included in the disclosure. All references cited herein are incorporated herein by reference to the extent that they are not inconsistent with the disclosure herein.
[0071] The present invention has been described with reference to preferred and alternative embodiments which are intended to be illustrative only and not limiting of the full scope of the invention as set forth in the appended claims.
Claims
1. 1. A system for protecting privacy of personal data, the system comprising: a private zone implemented on one or more first storage devices in communication with the one or more processors; a transformation zone implemented on one or more second storage devices in communication with said one or more processors; a general zone implemented on one or more third storage devices; a private data key creation subroutine; Equipped with the one or more storage devices storing a plurality of personal data files comprising records of raw data and a set of instructions that, when executed by the one or more processors, cause the one or more processors to identify particular data items within the set of raw data; the transformation zone comprising a personal data key map comprising a mapping between each of the specific data items and a corresponding personal data key for each of the specific data items, and an instruction set which, when executed by the one or more processors, causes the one or more processors to replace each of the specific data items with the corresponding personal data key; the general zone comprises a set of analyzed records, and the specific data items within the records of raw data are replaced by the corresponding personal data keys; The personal data key creation subroutine executes on the one or more processors and provides the one or more processors with: To receive a specific data item, generating a unique identifier corresponding to said particular data item; Generate a random or pseudorandom salt; concatenating the unique identifier with the salt to generate a salted unique identifier; configured to hash the salted unique identifier to generate the personal data key corresponding to the original particular data item. system.
2. the private zone comprises a private zone security protocol configured to provide access only on a need-to-know basis; The system of claim 1 .
3. the transformation zone comprises a transformation zone security protocol configured to provide access only to designated persons; The system of claim 2.
4. the general zone comprising a general zone security protocol configured to enable access to a developer; The system of claim 3.
5. 1. A system for personal data protection, comprising: one or more processors; and one or more storage devices in communication with the one or more processors, the storage devices comprising an instruction set that, when executed by the one or more processors, causes the one or more processors to: receiving a plurality of data records from at least one data controller; Identifying a particular data item within a set of data records; creating a personal data key for each of said specific data items by generating a unique identifier corresponding to each of said specific data items, generating a random or pseudo-random salt for each unique identifier, concatenating each unique identifier to the salt, generating a salted unique identifier, hashing each salted unique identifier, and generating a personal data key corresponding to each of said specific data items; storing said particular data items, a corresponding set of personal data keys and associated sets of metadata in a personal data key map within a restricted access zone on said one or more storage devices; replacing said particular data item with said corresponding personal data key and creating a set of parsed records comprising said personal data key; system.
6. 1. A method of personal data workflow, the method comprising: receiving a set of raw personal data in a private zone from at least one data controller; taking said set of raw personal data and identifying specific data items within said set of raw personal data; creating a personal data key for each of the specific data items in the set of raw personal data by generating a unique identifier for each of the specific data items, generating a random or pseudo-random salt for each unique identifier, concatenating each unique identifier with the salt, generating a salted unique identifier, hashing each salted unique identifier, and generating a personal data key corresponding to each of the specific data items; storing, for each of said particular data items in said set of raw personal data, said particular data item, its corresponding personal data key and a set of associated metadata in a personal data key map in a transformation zone; replacing the particular data item with the corresponding private data key and transmitting the private data key back to the private zone; creating a set of parsed records comprising said personal data keys corresponding to said particular data items; The method includes:
7. The method further includes retrieving the set of analyzed records and transferring at least one of the analyzed records in the set of analyzed records to a personal data transformation service. The method according to claim 6.
8. the personal data transformation service replaces personal data in the at least one analyzed record; The method of claim 7.
9. The method further includes performing a second hash using a second hash key specific to the destination platform. The method of claim 7.
10. The method further includes transmitting the parsed record to the destination platform corresponding to the second hash key.
10. The method of claim 9.
11. The method further includes storing the second hash key corresponding to each of a plurality of destination platforms. The method of claim 10.
12. 1. A method for generating a personal data key corresponding to a particular data item, the method comprising: receiving one of said particular data items at a processor; generating a unique identifier corresponding to said particular data item; generating a random or pseudorandom salt in said processor; concatenating, at the processor, the unique identifier with the salt to generate a salted unique identifier; hashing the salted unique identifier by the processor to generate the personal data key corresponding to the particular data item; The method includes:
13. the unique identifier comprises a universally unique identifier (UUID); The method of claim 12.
14. The hashing step is performed using a SHA-256 hash algorithm. The method of claim 13.
15. 1. A personal data key engine, comprising: one or more processors; and a non-transitory computer-readable medium storing instructions that, when executed by the one or more processors, cause the one or more processors to: To receive a specific data item, generating a unique identifier corresponding to said particular data item; Generate a pseudorandom salt, concatenating the unique identifier with the salt to generate a salted unique identifier; hashing the salted unique identifier to generate a personal data key corresponding to the original particular data item; Personal Data Key Engine.
16. the unique identifier comprises a universally unique identifier (UUID); 16. The personal data key engine of claim 15.
17. The instructions further comprise a SHA-256 hash for hashing the salted unique identifier.
17. The personal data key engine of claim 16.
Citation Information
Patent Citations
Device and method for making information unindividualized
JP2000324094A
System, method and program distributed information access
JP2010186249A
Data processing program, data processing device, and data processing system
JP2014066831A
Secret information analysis system, and secret information analysis method
JP2017223784A
Using machine learning to predict psychometric profiles from behavioral data while preserving user anonymity
JP2019527874A
Cited By
Systems and methods for de-identifying and transforming data representations for data queries while maintaining data privacy
US12783728B2
