Data privacy system

Abstract

Data privacy systems are provided. A backend computer and a method of using the backend computer are described. The method can include receiving, at a first backend computer, sensor data associated with a vehicle; determining a labeling of the sensor data including determining personal data and determining non-personal data separate from the personal data, wherein each of the personal data and the non-personal data includes the labeled data, wherein the personal data includes information related to at least one identified or identifiable natural person; and performing, at the first backend computer, via the personal data and the non-personal data separate from the personal data, data processing associated with collecting the sensor data associated with the vehicle.

Description

Technical Field

[0001] This disclosure generally concerns data security and data privacy. Background Technology

[0002] Private and / or public (e.g., government) entities may expect to use data collected from cameras, etc., for a variety of purposes. In some cases, this data may contain personally identifiable information (PII). Improper handling of this data may violate local, regional, or global privacy laws—such as the General Data Protection Regulation (GDPR) or the California Consumer Privacy Act (CCPA). Summary of the Invention

[0003] According to one embodiment, a method for managing vehicle-related personal data is disclosed. The method may include: receiving vehicle-related sensor data at a first back-end computer; determining annotations for the sensor data, including: identifying personal data and identifying non-personal data separate from the personal data, wherein each of the personal data and non-personal data includes annotation data, wherein the personal data includes information related to at least one identified or identifiable natural person; and performing data processing associated with collecting the vehicle-related sensor data at the first back-end computer, via the personal data and the non-personal data separate from the personal data.

[0004] According to another embodiment, a first back-end computer is disclosed, which may include: one or more processors; and a memory storing a plurality of instructions executable by the one or more processors, wherein the plurality of instructions include: receiving sensor data associated with a vehicle at the first back-end computer; determining annotations for the sensor data, including: determining personal data and determining non-personal data separate from the personal data, wherein each of the personal data and the non-personal data includes annotation data, wherein the personal data includes information related to at least one identified or identifiable natural person; and performing data processing associated with collecting sensor data associated with the vehicle at the first back-end computer, via the personal data and the non-personal data separate from the personal data.

[0005] According to another embodiment, a non-transitory computer-readable medium is disclosed. The medium may include a plurality of instructions stored thereon, wherein the plurality of instructions are executable by one or more processors of a first back-end computer, wherein the plurality of instructions include: receiving sensor data associated with a vehicle at the first back-end computer; determining annotations for the sensor data, including: determining personal data and determining non-personal data separate from the personal data, wherein each of the personal data and the non-personal data includes annotation data, wherein the personal data includes information related to at least one identified or identifiable natural person; and performing data processing associated with collecting sensor data associated with the vehicle at the first back-end computer, via the personal data and the non-personal data separate from the personal data. Attached Figure Description

[0006] Figure 1 This is a schematic diagram illustrating an example of a data privacy system that includes a data collection system and multiple data protection systems.

[0007] Figure 2A , 2B 2C is a flowchart illustrating the process of using a data privacy system.

[0008] Figure 3 This is a flowchart illustrating another process using a data privacy system.

[0009] Figure 4A , 4B 4C is a flowchart illustrating another process using a data privacy system.

[0010] Figure 5A , 5B 5C is a flowchart illustrating another process using a data privacy system.

[0011] Figure 6A , 6B This is a flowchart illustrating another process using a data privacy system.

[0012] Figure 7 This is a flowchart illustrating another process using a data privacy system.

[0013] Figure 8 Another embodiment of the data collection system is illustrated. Detailed Implementation

[0014] Embodiments of this disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The figures are not necessarily to scale; some features may be enlarged or reduced to show details of particular components. Therefore, the specific structural and functional details disclosed herein should not be construed as limiting, but merely as a representative basis for teaching those skilled in the art to employ the embodiments in various ways. As will be understood by those skilled in the art, the various features illustrated and described with reference to any figure may be combined with features illustrated in one or more other figures to produce embodiments not explicitly illustrated or described. The combinations of illustrated features provide representative embodiments of typical applications. However, for a particular application or implementation, various combinations and modifications of features consistent with the teachings of this disclosure may be desired.

[0015] Turning now to the figures, where the same reference numerals indicate the same or similar functions or features, a data privacy system 10 is shown, which may include a data collection system 12 (e.g., embodied here within vehicle 14) and one or more data protection systems 16, 18, 20 (also referred to as “back-end computers 16, 18, 20”) (e.g., three back-end computers are shown here; however, more or fewer may be used instead). Modern computing systems collect vast amounts of data on objects—including humans (e.g., natural persons)—during their operation. This data can be used for a variety of reasons—for example, in some cases, engineers can use the data to improve vehicle computing systems at back-end facilities (e.g., such as enabling partially or fully autonomous driving modes—e.g., advanced driving systems according to levels 1, 2, 3, 4, and 5 as defined by the Society of Automotive Engineers (SAE)). For example, simulations and training for developing software can be better implemented when using real-life scenarios as input. However, current data privacy laws may prevent the use of some of this data—e.g., if the data includes personal data (e.g., such as personally identifiable information (PII)). System 10 enables the collection and protection of both personal and non-personal data—for example, in compliance with evolving privacy laws such as the General Data Protection Regulation (GDPR) and the California Consumer Privacy Act (CCPA). More specifically, among other things, System 10 utilizes a multi-party computation (MPC) framework, a trusted execution environment (TEE), or both to facilitate the protection of personal data. It should be understood that although the following disclosure uses vehicle 14 (which can collect data while operating in at least one autonomous driving mode) to illustrate data collection system 12, other data collection systems are also possible—for example, other uses such as cameras or other sensors mounted to infrastructure (e.g., regardless of whether the sensors are being used in conjunction with autonomous driving).

[0016] Before describing the accompanying drawing (Figure) 1, personal data, non-personal data, multi-party computation (MPC) framework, and trusted execution environment (TEE) are described, as these terms may be used in the written specification and claims.

[0017] Personal data can refer to one or more of the following: any information relating to an identified or identifiable natural person; an identifiable natural person is a person who can be identified directly or indirectly—particularly by reference to an identifier such as a name, identification number, location data, online identifier, or by reference to one or more factors distinctive to that natural person’s physical, physiological, genetic, mental, economic, cultural, or social identity. Personally identifiable information (PII) is a non-limiting example of personal data. A natural person can refer to an individual who has his or her own legal personality (however, for example, a legal person in this document can refer to an individual, a private organization (e.g., a commercial entity or a non-governmental organization), or a public organization (e.g., a government entity)). Therefore, for example, personal data can refer to address information associated with a specific identified or identifiable natural person, neighbor or location information associated with a specific identified or identifiable natural person, address number associated with at least one identified or identifiable natural person, biometric information associated with a specific identified or identifiable natural person, physical characteristics of at least one identified or identifiable natural person, vehicle information (e.g., license plate information) associated with a specific identified or identifiable natural person, image data or video data associated with a specific identified or identifiable natural person (e.g., where video data includes image sequences), and so on.

[0018] Non-personal data can refer to data that is not personal data. Continuing with the example of vehicle 14, the sensors of vehicle 14 can receive a combination of personal and non-personal data (e.g., referred to herein as unseparated data). For example, the camera sensors of vehicle 14 may not filter out all personal data from the image; instead, personal and non-personal elements may often be captured together—for example, when imaging a leading vehicle (in front of vehicle 14), the license plate identifier of the leading vehicle is often captured simultaneously; the leading vehicle may not be personal data, while the license plate identifier may be personal data.

[0019] A multi-party computation (MPC) framework can refer to masking computation of personal data or unseparated data, wherein at least a first input (e.g., one or more random masks) is received from a first party (one of data protection systems 16, 18, 20), and at least a second input (e.g., one or more random masks) is received from a second (different) party (e.g., another of data protection systems 16, 18, 20). The masking computation uses the first and second inputs to determine an output (e.g., a share of the masked data), and each of the first and second parties receives the output (e.g., the first party receives a first portion of the set of shares of the masked data, and the second party receives a different second portion of the set of shares of the masked data, wherein the shares of the first portion may not include the shares of the second portion). According to this framework, the first party cannot decipher the original personal data or unseparated data without the shares of the second party (where the first party does not have said shares), and vice versa. Therefore, any data breach (e.g., due to a malicious attack) cannot decipher the first party's personal data (even if the data breach includes access to the first party's shares). If a data breach occurs in the second party, the data is similarly protected in advance. It should be understood that the parties to an MPC framework cannot access data without the consent of all or a quorum of the parties. Therefore, the use of an MPC framework may comply with GDPR or CCPA.

[0020] A Trusted Execution Environment (TEE) can refer to an isolated computing environment of a computer, implemented in both hardware and software. A TEE may include an isolated (e.g., partitioned) processor portion with an independent operating system (OS) (e.g., called a Trusted OS), which executes software applications on an isolated (e.g., partitioned) memory portion—for example, enabling only predetermined software applications (e.g., those typically developed by the TEE developer) to be executed. The TEE memory may store (cryptographic) private keys (e.g., public-private key pairs based on Rivest–Shamir–Adleman (RSA) keys, Elliptic Curve Diffie-Hellman Key Exchange (ECDHE) keys, etc.); in some cases, this private key may be used with the (cryptographic) public key when input data is received from outside the TEE. In this way, the provider of the input data can verify that the TEE (and only the TEE) has performed the predetermined computation using the input data. For example, in the context of this disclosure, the TEE may receive input data from a first party, perform a cryptographic computation (hash function), and sign the output with the private key (e.g., generate a hash). Subsequently, the TEE can provide the hash and corresponding public key to the first party. Similarly, the TEE can transact with a second (or other) party. In this paper, the cryptographic function can utilize cryptographic keys, where a cryptographic key can refer to a public key, private key, symmetric key, etc.—for example, depending on any suitable public-key-private-key infrastructure, symmetric key infrastructure, etc.

[0021] Now go to Figure 1 Among other things, the data collection system 12 may also include a computer 30, a communication system 32, and one or more sensors 34. The computer 30 facilitates the collection of unseparated data, some processing of the data, and communication of the data to at least one of the data protection systems 16-22. The computer 30 may include one or more processors 36 and memory 38.

[0022] One or more processors 36 may be any suitable device for controlling one or more sensors 34 and / or communication system 32. One or more processors 36 may be programmed to process and / or execute digital instructions to perform at least some of the tasks described herein. Non-limiting examples of one or more processors 36 include one or more of the following: microprocessors, microcontrollers or controllers, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), one or more circuits including discrete digital and / or analog electronic components arranged to perform predetermined tasks or instructions, etc.—to name just a few. In at least one example, one or more processors 36 read from and / or execute multiple sets of instructions from memory 38, which may be embodied as a computer program product stored on a non-transitory computer-readable storage medium (e.g., such as memory 38). Some non-limiting examples of instructions are described in the following processes and illustrated in the accompanying drawings. Unless otherwise stated, these and other instructions may be executed in any suitable order. The instructions and example processes described below are merely embodiments and are not intended to be limiting.

[0023] Memory 38 may include volatile and / or non-volatile memory devices. Non-volatile memory devices may include any non-transitory computer-usable or computer-readable medium, storage device, storage article, etc., including persistent memory (e.g., non-volatile). Non-limiting examples of non-volatile memory devices include: read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), optical disk, magnetic disk (e.g., such as hard disk drive, floppy disk, magnetic tape, etc.), solid-state memory (e.g., floating-gate metal-oxide-semiconductor field-effect transistor (MOSFET), flash memory (e.g., NAND flash memory, solid-state drive, etc.), and even some types of random access memory (RAM) (e.g., such as ferroelectric RAM). According to one example, a non-volatile memory device may store one or more instruction sets, which may be embodied as software, firmware, or other suitable programmable instructions executable by (one or more) processors 36—including, but not limited to, the instruction examples set forth herein.

[0024] Volatile memory devices can include any non-transitory computer-usable or computer-readable medium, storage device, storage item, etc., including non-persistent memory (e.g., which may require power to maintain the stored information). Non-limiting examples of volatile memory include: general-purpose random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), etc.

[0025] Communication system 32 may include electronic circuitry (and / or programmable software) to facilitate wired, wireless, or both communication. For example, communication system 32 may include a wireless chip for short-range (e.g., Wi-Fi, Bluetooth, etc.) or long-range (e.g., cellular, satellite, etc.) wireless communication. Furthermore, communication system 32 may include a wired interface with a port, allowing a trained technician to physically connect a service computer to the port and download protected personal and / or non-personal data from memory 38. Other aspects of communication system 32 are also contemplated herein.

[0026] One or more sensors 34 may include any suitable electronic hardware capable of collecting sensor data of their surrounding environment. Non-limiting examples of sensors 34 include: light detection and ranging (LiDAR) sensors; digital camera sensors (e.g., detecting light in and around the visible spectrum); infrared cameras; short, medium, or long-range thermal imaging sensors; millimeter-scale radar sensors; sonar sensors (e.g., ultrasonic sensors), etc. As shown, sensors 34 may transmit unseparated data to computer 30, which in turn may provide the unseparated data to communication system 32. As further described below, computer 30 may modify the unseparated data before providing it to communication system 32—for example, computer 30 may mask the data, separate personal data from non-personal data, encrypt the data, perform combinations of these tasks, etc.

[0027] Sensor data can refer to any suitable image data, multiple data points from a lidar sensor, multiple data points from a millimeter-scale radar sensor, multiple data points from a sonar sensor, etc. Image data can refer to digital images from a digital camera sensor, elements of a digital image (e.g., pixels or groups of pixels), video frames, etc. Non-personal data can be embodied in sensor data, and personal data can be embodied in image data and some other forms of sensor data.

[0028] The data collection system 12 can communicate with one or more of the back-end computers 16-20 via wired and / or wireless system 40. Similarly, any one of the back-end computers 16-22 can communicate with each other via system 40. System 40 may include public telephone infrastructure, cable communication infrastructure, cellular tower and base station infrastructure, satellite and satellite base station infrastructure, etc., all of which are known in the art. Therefore, wired and / or wireless system 40 should be interpreted broadly. At least in this implementation, system 40 may include any suitable hardware and / or software for implementing vehicle-to-vehicle (V2V) communication, vehicle-to-infrastructure (V2I) communication, and / or vehicle-to-everything (V2X) communication.

[0029] exist Figure 1 An example of a back-end computer 16 is shown. It should be understood that some of the illustrated aspects of back-end computer 16 are optional and are not used in all embodiments. Furthermore, at least some hardware and software aspects of back-end computer 16 are similar to those of back-end computer 18 and / or back-end computer 20; however, the data stored and / or processed in each of back-end computers 16, 18, and / or 20 may be different.

[0030] According to one example, the back-end computer 16 may include one or more processors 42 (only one is shown) and memories 44, 46. According to one example, the hardware of the processor(s) 42 may be similar to that of the processor 36 described above; therefore, for the sake of brevity, the hardware will not be described in detail here. At least some instructions executed by the processor(s) 42 may differ from those executed by the processor(s) 36, as will be illustrated in the following flowcharts.

[0031] According to at least one non-limiting example, processor(s)42 may include a Trusted Execution Environment (TEE)48, and TEE48 may be optional. Figure 1 The illustration shows an example of how TEE 48 and processor 42 can interact. For example, processor 42 can generally be embodied as a rich execution environment having an open software application 50 stored in memory 44 and an embedded operating system (OS) 52 stored in memory 44 and executable by processor 42, while TEE 48 can include a trusted software application 54, a trusted operating system (OS) 56, and a trusted memory 58 (e.g., the memory can be partitioned in both hardware and software). Trusted software application 54 can be stored in trusted memory 58 and can be specifically executed by trusted OS 56. Trusted software application 54 can include a data privacy system using private-public key pairs, where memory 58 securely stores one or more (cryptographic) private keys and their corresponding public keys. As described in more detail below, TEE 48—via processor 42—can provide the public key to vehicle 14 to encrypt its sensor data; then, upon receiving the sensor data (or a portion thereof) at backend computer 16, TEE 48—using the corresponding private key—can decrypt the sensor data within TEE 48. Another such private key, stored within and used by the TEE 48, can be referred to as a sealing key. This sealing key can be used by the TEE 48 to encrypt personal data (e.g., a portion of sensor data), and the personal data can then be stored in memory 46 or elsewhere. In either case, the private key is not shared with the embedded OS 52, other parts of the processor 42, or other devices.

[0032] According to one example, the hardware of memories 44 and 46 can be similar to memory 38 described above; therefore, for the sake of brevity, these will not be described in detail here. According to one example, memory 44 can store at least some instructions executable by processor 42 (e.g., embodied in open software application 50 and embedded OS 52), and memory 46 can be embodied as a database of non-volatile memory. Thus, continuing with one of the examples above, personal data encrypted using a sealed key can be stored in memory 46. Furthermore, memory 58 can include volatile and / or non-volatile memory (e.g., partitioned memory) accessible only by TEE 48.

[0033] According to one embodiment (described more below), TEE 48 operates as a primary enclave. A primary enclave can refer to a TEE that has subordinate enclaves (e.g., also embodied as TEEs). In this way, data processed by one TEE can be at least partially accessed by another TEE. For example, as explained below, when the primary enclave signs data using a sealing key, one or more subordinate enclaves can decrypt the data, provided they use both the sealing key and a unique signature that identifies them as enclaves subordinate to the primary enclave.

[0034] In at least one example, the architecture of backend computer 18 can be similar to that of backend computer 16, except that the TEE of backend computer 18 can be a subordinate TEE. For example, as Figure 1 As shown, the backend computer 18 may include one or more processors 62 and memories 64, 66. The processor(s) 62 may include a Trusted Execution Environment (TEE) 68. The TEE 68 may also be optional. The TEE 68 and processor 62 may interact similarly as described above. For example, processor 62 may generally be embodied as a rich execution environment with an open software application 70 and an embedded operating system (OS) 72, while the TEE 68 may include a trusted software application 74, a trusted operating system (OS) 76, and a trusted memory 78 (e.g., memory that may be partitioned in both hardware and software). As will be described in at least one flowchart, a subordinate TEE (e.g., TEE 68) may use the same sealing key used by TEE 48 plus its own unique signature to access data stored in memory 46 (e.g., a database).

[0035] The back-end computer 20 may include one or more processors 82 and memories 84, 86, and may or may not include a TEE (subordinate or otherwise). Again, for the sake of brevity, the hardware of the processors 82 and memories 84, 86 may be similar to that of the processors 42 and memories 44, 46—for example, again, the processors 82 may execute instructions that are at least partially different from those of the processors 42 and 62, and store data that is at least partially different from the data stored in the memories 44, 46, 64, 66.

[0036] According to one example, the hardware of backend computer 22 may be similar to or equivalent to backend computer 16 or 18—for example, it may include TEE 24, which may include a subordinate enclave (e.g., similar to the operation of optional TEE 68). According to one example, the subordinate enclave is subordinate to the primary enclave associated with TEE 48.

[0037] It should be understood that in the process example described below, backend computers 16, 18, 20, and 22 can each represent different parties that do not collude with each other. For example, they are unrelated entities—for instance, they may be owned by different organizations that do not share or exchange confidential or other data information with each other under any contractual or organizational relationship or obligation. This non-collusion of sensor data content promotes compliance with data privacy regulations.

[0038] Figure 1The diagram also illustrates a third-party entity 88, which may (or may not) include a third-party server 90. In some cases, the third-party entity 88 includes, for example, an organization that securely analyzes personal data and may comply with local, regional, and / or global data privacy laws. According to a non-limiting example, the third-party entity 88 may receive a share of masked data (and a corresponding mask used to mask the data) – for example, according to the MPC framework – and use shares from at least two different parties to unmask the masked (personal) data. In this way, experienced humans can analyze and annotate the personal data therein. The annotation of the data can refer to any suitable classification technique that categorizes objects for computer analysis. For example, in the context of autonomous driving mode, determining the annotation may include associating identifiers with vehicles, lane signs, pedestrians, etc., and annotating personal data and portions of sensor data associated with the personal data. To illustrate the latter example, consider image data of the environment surrounding vehicle 14. The image data may include license plate numbers of other vehicles on the road; vehicles, roads, and license plate numbers can all be associated with annotations; furthermore, pixel data associated with each of the vehicles, roads, and license plate numbers may also be identified. Continuing the example above, once the vehicle is labeled at third-party entity 88, the fact that it has a license plate can be stored (i.e., based on its label); however, characters identifying the license plate and / or its owner can be hidden (e.g., to facilitate compliance with privacy laws). Third-party entity 88 can re-mask the labeled (personal) data and re-share it (i.e., send it back to a computer such as backend computer 16 or 18, as described more further below)—thereby facilitating compliance with privacy laws. Sensor data including masked (e.g., hidden) personal data can be useful for engineering design software models that use real-world scenario simulations and train autonomous driving computers. Furthermore, by securely hiding personal data, engineering designs can comply with local, regional, and / or global data privacy laws.

[0039] In an instance where third-party entity 88 includes server 90, server 90 may include one or more processors and memory, such as those described above (not shown). Server 90 may be configured to execute a software application that extracts or identifies (at least partially) personal data and performs annotation functions on the personal data.

[0040] Now go to Figure 2A , 2BFigure 2C illustrates process 200, which describes the use of an MPC framework to collect sensor data and protect personal data therein, wherein computer 30 of vehicle 14 separates personal data from non-personal data. Separating data can refer to isolating one portion of sensor data from another in an attempt to minimize the risk of data privacy breaches. This separation can occur in a hardware context (e.g., in a Trusted Execution Environment (TEE) and / or using the TEE's cryptographic keys for signing). In another context, separation can occur in a software context, where the disclosure of data held by one entity (e.g., backend computer 16) is useless without the disclosure of data from a second facility (e.g., such as backend computer 18). Of course, separation can also occur in both hardware and software contexts.

[0041] In block 205 of the flowchart, the computer 30 (e.g., processor 36) of vehicle 14 can receive vehicle sensor data. As discussed above, according to at least one example, vehicle 14 may be able to operate in one or more autonomous driving modes. While doing so, sensors (one or more) 34 can collect sensor data—e.g., lidar sensor data, camera sensor data, ultrasonic sensor data, radar sensor data, etc.

[0042] In a subsequent block 210, computer 30 may request one or more random masks from backend computer 16. In response (in block 215), backend computer 16 may generate and / or send random masks. A mask may refer to any suitable data used to hide at least a portion of the sensor data. In this way, if the sensor data (e.g., personal data within the sensor data) is obtained by a malicious attacker or unauthorized party, the personal data will be hidden and unviewable / accessible, provided the attacker / party is unable to remove the mask. According to a non-limiting example, the mask may be random noise, and the mask may be combined with the sensor data such that the data is secure (e.g., unidentifiable) without removing the mask or rendering the MPC algorithm capable of processing the data despite the masking ineffective. According to one example, when protecting image data, computer 30 may request multiple masks; for example, different random masks may be applied to each pixel of the personal data in the image data (or, for example, different random masks may be applied to a relatively small set of pixels of the personal data in the image data). This is merely an example, and other embodiments are contemplated herein.

[0043] In block 220, computer 30 may also request one or more random masks from backend computer 18. And in response, in block 225, backend computer 18 may generate and / or send one or more random masks to computer 30 (e.g., similar to the random masks in block 215).

[0044] In block 230, computer 30 can separate (e.g., delimit) the sensor data into two categories: personal data and non-personal data. For example, computer 20 can execute a set of computer instructions that parses and identifies the sensor data for personal data (as described above). For example, in the context that the sensor data is an image, computer 30 can identify specific pixels in an image that include personal data (e.g., a natural person's face, a natural person's address number, a natural person's license plate, etc.). A non-limiting example of an algorithm that computer 30 can execute to separate personal data from non-personal data can be designed using Haar cascades for face detection. Other examples may also exist.

[0045] In block 235, given that personal data within the sensor data set has been identified, computer 30 can perform masking of that personal data. Masking may include determining a so-called share of the masked data by applying one or more masks to the personal data. In at least one example, these shares may be (at least temporarily) stored in memory 38 of computer 30.

[0046] Performing masking can include using one or more masks provided by backend computer 16 and one or more masks provided by backend computer 18. Continuing with the example illustrated above, both masks can be used to mask sensor data associated with personal data. For example, according to a non-limiting example, random noise (from a random mask from computer 16) and random noise (from a different random mask from computer 18) can be applied to common pixels containing or associated with personal data (and this can also be repeated for other pixels using masks from computers 16, 18). In this way, personal data can only be deciphered by an unintended recipient of the masked data if the unintended recipient has both masks—a highly unlikely scenario. Such masking techniques can appropriately comply with global and regional data privacy regulations (e.g., GDPR and CCPA as discussed above). In this example, two backend computers 16, 18 provide random masks to vehicle 14; however, it should be understood that in other examples, three or more backend computers can provide random masks (e.g., such that three or more corresponding masks are applied to personal data).

[0047] In block 240, computer 30 may also store at least one file containing non-personal data, including a set of sensor data, in memory 38. According to at least one example, the non-personal data is stored as a single file, while the masking data is stored in multiple files.

[0048] In block 245, a first portion of the masked personal data can be provided to backend computer 16. This can occur in any suitable manner. For example, in some cases, computer 30 can wirelessly transmit the masked portion to backend computer 16 via communication system 32—e.g., via a security technology (e.g., according to Transport Layer Security (TLS) protocol, etc.). According to another example, vehicle 14 can be serviced by an authorized service technician who manually downloads the first portion of the masked data (e.g., at an authorized facility)—e.g., using a physical port of communication system 32. Other technologies may also be used.

[0049] Similarly, in block 250, one or more files containing non-personal data are provided to backend computer 16. This can happen in any suitable manner (e.g., and can be similar to block 245).

[0050] In block 255, a second portion of the masked personal data is securely provided to backend computer 18. According to one example, the first portion may not include the second portion. This can also occur in any suitable manner (e.g., similar to block 245).

[0051] Now go to Figure 2B Process 200 continues. In block 260, backend computer 16 can determine labeled data associated with non-personal data. Those skilled in the art will understand that labeling can refer to the use of classification algorithms to identify objects within computer data, whereby, once identified, the objects are tagged with labels (e.g., metadata). Automotive and other engineers use such labels to leverage sensor data collected by vehicle 14. For example, when sensor data includes labels such as “vehicle,” “pedestrian,” “lane sign,” this can be used during computer simulations of autonomous driving, training of autonomous driving modules, etc. A non-limiting example of a labeling algorithm is the YOLO (You Only Look Once) convolutional neural network used for object classification algorithms; however, other algorithms can be used instead or in combination with it.

[0052] In block 265 (which may include blocks 265a-265d), backend computers 16 and 18 may determine the labeling of the first and second portions of the masked personal data shares according to the MPC framework, for example, by utilizing an MPC algorithm that separates the shares of personal data among two or more non-colluding computing systems. For example, in block 265a, backend computer 16 may perform a local MPC computation and provide the output of those computations(s) to backend computer 18; similarly, in block 265d, backend computer 18 may perform a local MPC computation and provide the output of those computations(s) to backend computer 16. In each of blocks 265b and 265c, backend computers 16 and 18 may respectively execute local computation segments of the MPC computation to facilitate labeling using a classification algorithm—for example, using the information provided in blocks 265a and 265d. According to the example embodiments, the local computations of blocks 265a and 265d may include addition computations (e.g., scalar addition of random numbers (e.g., masked)), and the local operation segments of the MPC computations of blocks 265b and 265c may include multiplication computations (e.g., scalar multiplication). A non-limiting implementation of block 265 is referred to as the beaver triple; however, other techniques may be employed instead. Furthermore, it should be understood that the computations and operation segments for labeling personal data described in blocks 265a-265d may be used for other data processing procedures (e.g., simulation, model training, etc.) according to the MPC framework or according to the MPC-TEE (hybrid) environment, as described below. Using the MPC framework to protect personal data can comply with GDPR, CCPA, and other government regulations because sensor data containing personal data is separated into two distinct locations.

[0053] According to one example, the annotation of personal data occurs at third-party entity 88—for example, instead of backend computer 16. For example, block 270 illustrates an illustrative embodiment that can be used in place of blocks 265a-265d.

[0054] Block 270 may include 270a-270h. In block 270a, backend computer 16 may permit third-party entity 88 to access labeled non-personal data (or block 270a may include providing non-personal data to third-party entity 88 to perform labeling of non-personal data). In any case, in block 270b, backend computer 16 may provide third-party entity 88 with a first portion of its personal data masking share. Similarly, in block 270c, backend computer 18 may provide third-party entity 88 with a second portion of its personal data masking share.

[0055] Once third-party entity 88 receives the first and second masking shares from backend computers 16 and 18, in block 270d, third-party entity 88 can identify the personal data and determine the labeled data associated with the personal data. According to the MPC framework, personal data is exposed when using the masking shares of both computer 16 (in block 270b) and computer 18 (in block 270c). Therefore, third-party entity 88 can be a trusted, secure environment—for example, an organization practicing global and regional data privacy regulations. Typically, in block 270d, employees of such an organization can analyze and manually label personal data; however, such a third-party entity can alternatively execute one or more labeling algorithms (e.g., using server 90).

[0056] Once the third-party entity 88 has labeled the personal data, in blocks 270e and 270f, the third-party entity 88 can receive new random masks from each of the back-end computers 16 and 18, respectively (e.g., entity 88 can request these new random masks, and computers 16 and 18 can provide them via system 40). Thereafter, the third-party entity 88 can perform masking of the now-labeled personal data and return the first and second masked portions of the re-masked personal data to each of the back-end computers 16 and 18, respectively (e.g., the first re-masked portion returns to back-end computer 16, and the second re-masked portion returns to back-end computer 18).

[0057] Now go to Figure 2C In blocks 280 (including blocks 280a-280d), the backend computer 16 can perform data processing using separately labeled personal data and labeled non-personal data—for example, this could include vehicle simulation, vehicle model training, vehicle model testing, etc. Furthermore, other embodiments may focus less on autonomous driving modes and more on other features captured by sensor data from vehicle 14. And furthermore, as discussed above, any personal data acquired can be secure if personal data is corrupted at the backend computer 16 (e.g., if there is a data breach in memory 44 or memory 46) because it is masked and undecipherable for unauthorized recipients.

[0058] Blocks 280a, 280b, 280c, and 280d can correspond to blocks 265a, 265b, 265c, and 265d, respectively, as a technique to facilitate the processing of data securely stored at computer 16 and data securely separated from data stored at computer 18. In block 265 (blocks 265a-265d), processing refers to annotated personal data; here, in block 280 (blocks 280a-280d), processing can be data processing, such as performing computer simulations, model training, model testing, etc. (the above are just examples). After block 280, process 200 can end.

[0059] Now go to Figure 3 The diagram illustrates process 300, which illustrates the use of an MPC framework to collect sensor data and protect personal data therein, wherein a back-end computer 16 (or alternatively a third-party entity 88) separates personal data from non-personal data.

[0060] Process 300 may begin with block 305. In block 305, computer 30 may receive vehicle sensor data. This may be similar to block 205 described above; therefore, it will not be described in detail here.

[0061] Blocks 310, 315, 320, and 325 may correspond to blocks 210, 215, 220, and 225 (of process 200), respectively; therefore, these will not be described in detail here. In short, in blocks 310-325, the computer 30 of vehicle 14 may request and receive one or more random masks generated by backend computers 16 and 18.

[0062] Blocks 345 and 355 may correspond to blocks 245 and 255, respectively—for example, the shares of masked data are not limited to personal data. For example, according to process 300, computer 30 may determine the masked shares of sensor data from vehicle 14 and provide them to backend computers 16 and 18, respectively; however, here, computer 30 of vehicle 14 may not separate personal data from non-personal data, but may perform masking. For example, the masked shares of sensor data may include unseparated personal and non-personal data. More specifically, the masked shares of sensor data may include a first portion of the masked shares (e.g., sent to backend computer 16) and a second portion of the masked shares (e.g., sent to backend computer 18). Providing the masked shares in blocks 345 and 355 may be based on any suitable technology; for example, using communication system 32 and system 40 and / or a physical connection (via a port of communication system 32), as described above. According to embodiments of process 300, computer 30 may not be equipped to parse and / or identify personal data from sensor data and to separate personal data from non-personal data.

[0063] In block 365, backend computer 16 can separate personal data from non-personal data and use an MPC framework to label the personal and non-personal data. According to at least one example, backend computers 16 and 18 use blocks 365a, 365b, 365c, and 365d corresponding to blocks 265a, 265b, 265c, and 265d, using shares of masked sensor data provided to them in blocks 345 and 355 respectively, to separate personal data from non-personal data. According to at least one example, backend computers 16 and 18 also label data during blocks 365a-365d. According to another example, backend computers 16 and 18 first execute blocks 365a-365d to separate personal and non-personal data, and then re-execute blocks 365a-356d to label personal (and / or non-personal) data. In at least one example, labeling can be determined by executing a process similar to block 270 (… Figure 2B The instructions are executed according to the instructions of the MPC framework, for example, still using the MPC framework to maintain the separation of personal data.

[0064] In subsequent block 380, backend computers 16 and 18 can execute data processing instructions (e.g., computer simulation, model training, model testing, etc.). According to at least one example, block 380 may include blocks 380a, 380b, 380c, and 380d, which may correspond to blocks 280a, 280b, 280c, and 280d. Since blocks 280a-280d have already been described, they will not be described again here.

[0065] Now go to Figure 4A , 4B 4C illustrates process 400, which illustrates the use of a Trusted Execution Environment (TEE) to collect sensor data and protect personal data therein, wherein the computer 30 of vehicle 14 (or back-end computer 16 or third-party entity 88) separates personal data from non-personal data.

[0066] Process 400 can begin in a manner similar to processes 200 and 300. For example, in block 405, the computer 30 of vehicle 14 can receive vehicle sensor data. As described above, this block will not be repeated here.

[0067] According to an embodiment using TEE 48, in block 410, the computer 30 of vehicle 14 can request a public key from TEE 48. Although not mandatory, according to at least one embodiment, TEE 48 can be used as a primary enclave—with subordinate enclaves, as described in more detail below. The request can be passed from computer 30 through system 40 to backend computer 16, where processor 42 can provide the request to TEE 48.

[0068] In block 415, TEE 48 can provide a public key corresponding to the private key stored in the secret repository of TEE 48. This can be transmitted from TEE 48 to processor 42 and computer 30 via communication system 32 in system 40 and vehicle 14.

[0069] After block 415, process 400 can proceed by executing either block 420 or block 425. Each will be discussed in turn.

[0070] Block 420 may include blocks 420a, 420b, and 420c. According to the embodiment of block 420a, computer 30 can separate personal data from non-personal data—for example, as described in block 230 above. In block 420b, computer 30 can encrypt personal data using a public key provided by TEE 48. And in block 420c, computer 30 can provide encrypted data (personal data) to TEE 48. Furthermore, in block 420c, computer 30 can provide unencrypted data to backend computer 16. The provision of encrypted or unencrypted data can be based on any suitable technology (wireless transmission, direct / manual download, etc., as described in block 245 above).

[0071] In block 425, the processor 36 of computer 30 can encrypt the sensor data set using the public key provided by TEE 48 in block 415. Computer 30 can then provide the encrypted sensor data set to TEE 48 (as described above with respect to block 245). Therefore, block 420 can be utilized when computer 30 is equipped to and / or capable of separating personal data from non-personal data, while block 425 can be executed when computer 30 is not so equipped or capable.

[0072] In block 430, which may follow block 420 or 425, TEE 48 (within the main enclave) can decrypt encrypted data—whether it includes personal data or a collection of sensor data (i.e., both personal and non-personal data).

[0073] In block 435, if this has not been done previously (in block 420), TEE 48 can separate personal data from non-personal data. Since this may have already occurred previously, block 435 is optional.

[0074] Now go to Figure 4B Process 400 can continue with block 440. In block 440, labeled data associated with non-personal data can be identified. This can occur within TEE 48. Alternatively, the server 90 of third-party entity 88 can identify the labeled data. Or, the natural person of third-party entity 88 can examine and identify it. The use of third-party entity 88 has been described above and does not need to be described again here.

[0075] Within block 445—and within TEE 48—TEE 48 can identify labeled data associated with personal data. Evaluating personal data within TEE 48 can comply with global and regional laws regarding data privacy compliance because the Trusted OS 56 and Trusted Application 54 can perform labeling. For example, when TEE 48 separates personal data from non-personal data, labeling algorithms (e.g., YOLO (You Only See Once) convolutional neural networks used for object classification) can be stored as trusted applications within TEE 48.

[0076] In block 450, the primary enclave of TEE 48 can use the sealing key known within TEE 48 to encrypt labeled personal data. This allows personal data to be stored in a lower-cost (or more readily available) storage environment (e.g., a general-purpose database).

[0077] For example, in a subsequent block 455, both non-personal data and personal data (encrypted with a sealing key) can be stored in a database such as memory 46. By using a database, large amounts of personal data can be securely stored using cryptographic keys known to TEE 48.

[0078] In block 460, TEE 48 can perform processing using labeled data (i.e., both personal and non-personal data). The nature of the data processing can be similar to the data processing described above in block 280 (of process 200) (e.g., computer simulation, model training, model testing, etc.); therefore, these aspects will not be described again here. That is to say, it should be understood that block 280 occurs within the MPC framework, while block 460 occurs within the context of a Trusted Execution Environment.

[0079] Now go to Figure 4C Process 400 can then continue. According to a non-limiting example, the backend computer 18 may also include a Trusted Execution Environment (TEE 68) within its processor (processor 62). Furthermore, TEE 68 may be a subordinate enclave of the primary enclave of TEE 48. Blocks 465, 470, 475, and 480 are optional and associated with the backend computer 18 having subordinate enclaves.

[0080] In block 465, remote proof can occur between the primary enclave of TEE 48 and the secondary enclave of TEE 68, allowing the secondary enclave to retrieve personal data using a copy of the sealed key stored within its TEE combined with the secondary enclave's unique signature. Proving the secondary enclave's existence is a known process and will not be described in further detail here.

[0081] In block 470, backend computer 18 may be permitted to access the database of memory 46, such that non-personal data stored in memory 46 may be copied or otherwise stored and used by backend computer 18 (e.g., stored on memory 66 of backend computer 18). Furthermore, block 470 may include retrieving personal data stored on memory 46, which was previously encrypted with a sealing key.

[0082] In block 475, TEE 68 can decrypt personal data using the sealing key (the same private key used in block 450) plus a signature unique to the dependent enclave. The ability of a dependent enclave to decrypt data using the sealing key and its unique signature is known and will not be described in detail here.

[0083] In block 480, the processing of labeled personal and non-personal data can also occur at backend computer 18. In at least some examples, this can be similar to that described in block 460 above.

[0084] Now go to Figure 5A , 5B Figure 5C illustrates process 500, which illustrates the use of an MPC framework and a Trusted Execution Environment (TEE) (e.g., a hybrid architecture) to collect sensor data and protect personal data therein, wherein the vehicle 14's computer 30 (or back-end computer 16 or even back-end computer 18) separates personal data from non-personal data.

[0085] Process 500 may begin at block 505, where the computer 30 of vehicle 14 receives vehicle sensor data. As described above, this may be similar to block 205.

[0086] Blocks 510 and 515 may be similar to blocks 410 and 415 previously described above. These blocks will not be described in detail again. In short, in block 510, computer 30 can request a public key from TEE 48, and in block 515, TEE 48 can provide the public key. In at least one embodiment of process 500, TEE 48 is a master enclave securely storing the private key corresponding to the public key.

[0087] Process 500 can continue by executing either block 520 or block 525. Each will be discussed in turn.

[0088] In block 520 (which may include blocks 520a, 520b, and 520c), computer 30 can separate personal data from non-personal data. As mentioned above, blocks 520a, 520b, and 520c may correspond to blocks 420a, 420b, and 420c, respectively. Therefore, these will not be described again here.

[0089] In block 525 (which includes blocks 525a and 525b), computer 30 can use the public key provided in block 515 to encrypt sensor data. As mentioned above, blocks 525a and 525b can correspond to blocks 425a and 425b, respectively. Therefore, this will not be described again here.

[0090] Block 530 and optional block 535 can be similar to blocks 430 and 435, respectively—for example, where TEE 48 decrypts encrypted data and separates personal data from non-personal data if it had not been previously separated. Since these blocks can be similar to the corresponding blocks 430 and 435, they will not be described again here.

[0091] Now go to Figure 5B Blocks 540 and 545 may follow. These blocks may be similar to or equivalent to blocks 440 and 445, respectively, in which the annotation data for non-personal data is determined (block 540), and the annotation data for personal data is determined within TEE 48. Since blocks 540 and 545 are similar to blocks 440 and 445, respectively, they will not be described in detail again.

[0092] In block 550, the processor 42 of backend computer 16 may request one or more random masks from backend computer 18. And in block 560, in response, backend computer 18 may generate and / or send the requested random mask.

[0093] Similarly, in block 565, backend computer 16 can request one or more random masks from backend computer 20. And in block 570, backend computer 20 can generate and / or send the requested random mask.

[0094] In block 580, TEE 48 can perform masking of personal data using the random mask received in blocks 560 and 570. The resulting masked shares of personal data (e.g., a first masked share and a second masked share) can be (at least temporarily) stored in memory 44 or 46.

[0095] In a subsequent block 585, backend computer 16 may provide labeled non-personal data to backend computer 18 (e.g., or provide access to it). Additionally, block 585 may include a first masked share of the labeled personal data provided to backend computer 18.

[0096] Similarly, in block 590, backend computer 16 may provide backend computer 20 with labeled non-personal data (e.g., or provide access to it), and block 590 may further include a second masking share of the labeled personal data provided by backend computer 16 to backend computer 20.

[0097] Now go to Figure 5C Process 500 may include execution block 595 or block 597, wherein in block 595, the MPC framework is used for data processing, and in block 597, a subordinate TEE is used for data processing. In block 595 (which may include blocks 595a, 595b, 595c, and 595d), backend computers 18 and 20 may use labeled personal data (and also labeled non-personal data) to perform data processing. Blocks 595a, 595b, 595c, and 595d may be similar to the previously described blocks 380a, 380b, 380c, and 380d; therefore, these blocks will not be described in detail here. After block 595, process 500 may terminate.

[0098] In block 597 (which may include blocks 597a, 597b, 597c, 597d, 597e, 597f, 597g, 597h), the subordinate TEE 24 of backend computer 22 can be used for data processing of personal data. For example, in blocks 597a and 597b, backend computer 18 and backend computer 20 can respectively provide a first portion of the masking share (e.g., labeled personal data) to TEE 24 and a second portion of the masking share (e.g., labeled personal data) to TEE 24. In block 597c, TEE 24 uses both the first and second portions to determine the original masked data and performs data processing using the personal data therein. In blocks 597d and 597e, TEE 24 can request (and receive) new masks from backend computers 18 and 20, respectively. Subsequently, in block 597f, using the new mask, TEE 24 can generate masking shares (e.g., a new first portion and a new second portion). Furthermore, in blocks 597g and 597h respectively, the first portion of the masking share can be provided back to the backend computer 18, and the second portion of the masking share can be provided back to the backend computer 20. After this, process 500 can end.

[0099] Now go to Figures 6A-6B The diagram illustrates another hybrid architecture process 600 (e.g., using both an MPC framework and a trusted execution environment). Process 600 may include blocks 605, 610, 615, 620, 625, 630, 635, 640, 645, 650, and 655, where these blocks may be similar to or equivalent to blocks 205, 210, 215, 220, 225, 230, 235, 240, 245, 250, and 255 (of process 200 in Figure 2). Therefore, these blocks will not be described again here.

[0100] In subsequent blocks 660 and 665, backend computers 16 and 18 may (respectively) provide first and second masking shares to TEE 24 (e.g., which may include subordinate enclaves). Within TEE 24, TEE 24 may perform annotation of personal (and non-personal) data. Furthermore, in block 670, TEE 24 may use the masking shares to perform data processing (e.g., similar to block 597c).

[0101] In the subsequent blocks 675, 680, 685, 690, and 695, these blocks can be similar to the previously described blocks 597d, 597e, 597f, 597d, 597g, and 597h. Therefore, these will not be described again here.

[0102] Now go to Figure 7 The diagram illustrates process 700, which can be applied to any of processes 200, 300, 400, 500, or 600. Process 700 may begin with block 705. In block 705, the computer 30 of vehicle 14 may receive sensor data—for example, while operating in autonomous driving mode; this may be similar to block 205 described above.

[0103] In block 710, backend computer 16 can determine whether vehicle 14 (more specifically, computer 30) is capable of separating personal data from the remainder of sensor data collected by sensor(s)(one or more) sensors 34. This determination can occur in several ways. For example, backend computer 16 can simply receive data from computer 30 and determine that the data has not been separated. Thus, backend computer 16 can conclude that computer 30 is unable or unsuitable to separate personal data from the sensor data. Or, for example, computer 30 can explicitly send a message to backend computer 16 informing it that it is not capable (at least for now) of performing such data separation, or that it is not capable of transmitting such data via system 40 (at least for now). These are merely examples; other examples of how backend computer 16 can determine the capabilities of computer 30 exist. When backend computer 16 determines that computer 30 possesses such capabilities, process 700 proceeds to block 715. And when backend computer 16 determines that computer 30 does not possess such capabilities, process 700 proceeds to block 720.

[0104] In block 715, the sensor data received by backend computer 16 will include personal data separated from non-personal data. Block 725 can follow.

[0105] In block 720, the sensor data received by backend computer 16 will include personal data that is not separated from non-personal data. And block 725 may follow.

[0106] In block 725, backend computer 16 (alone or in cooperation with backend computer 18) can separate personal data from sensor data—for example, identify personal data and identify non-personal data.

[0107] In block 730, if the MPC framework is used, process 700 can proceed to block 735; if a TEE (e.g., such as TEE 48) is used, process 700 can proceed to block 740; and if both are used, process 700 can proceed to block 745. In block 735, backend computers 16 and 18 can determine the labeling of personal data and perform data processing using a masking share to maintain the security of the personal data. In block 740, backend computer 16 can determine the labeling of personal data and perform data processing using the cryptographic key of TEE 48 to maintain the security of the personal data. And in block 745, one or more backend computers (e.g., such as computer 16) can use a trusted execution environment to determine the labeling, while two different backend computers (e.g., such as computers 18 and 20) can use a masking share to perform data processing. In this latter example, the MPC framework and TEE can be used to carry out the separation of personal data and various aspects of data processing. Furthermore, in blocks 740 or 745, in some examples, a primary enclave at a single backend computer can be used, and subordinate enclaves at different backend computers can also be used. Procedure 700 may terminate after any of blocks 735, 740, or 745.

[0108] Other embodiments of System 10 may also be used. For example, memories 44, 46 (or memories 64, 66) are described as suitable for storing masked or encrypted data (e.g., encrypted with a sealing key). According to at least one example, memories 44 and / or 46 may include a data lake. A data lake can refer to a system or repository of data stored in its natural / raw format, typically a file or binary large object (BLOB), where a BLOB can refer to a collection of binary data stored as a single entity in a database management system (e.g., a BLOB can be an image, audio, or other multimedia object, although sometimes binary executable code is also stored as a BLOB). In at least some examples, a data lake is a single storage of all enterprise data, including raw copies of source system data and data transformed (e.g., masked or encrypted) for tasks such as reporting, visualization, advanced analytics, and machine learning, where a data lake may include structured data (rows and columns), semi-structured data (CSV, logs, XML, JSON), unstructured data (emails, documents, PDFs), and binary data (images, audio, video) from relational databases.

[0109] Other examples also exist. For example, in the preceding description, the data collection system 12 is embodied in vehicle 14. As previously stated, other examples also exist. For example, turning to... Figure 8 The data collection system 12' may be at least partially embodied in the infrastructure 800 (e.g., a streetlight with a camera sensor and corresponding computer and / or communication system). Here, the infrastructure 800 may collect data related to the vehicle 14, and this data may also include personal data. Other examples (not shown) also exist—for example, the data collection system may be embodied (additionally or alternatively) as security camera infrastructure, satellite cameras and / or GPS systems, point-of-sale equipment, etc.

[0110] It should be understood that, in some cases, data protection systems 12, 12' can increase the computational efficiency of system 10. For example, when systems 12, 12' can mask or encrypt personal data, system efficiency is improved—for example, because sending the entire set of sensor data could impose a computational burden on both ends (at systems 12, 12' and system 16).

[0111] It should be understood that aspects of any of Processes 200, 300, 400, 500, 600, or 700 can be used together to promote data privacy and compliance with data privacy regulations.

[0112] Therefore, a data privacy system that allows the collection of large amounts of data has been described, which, among other things, can also be used to improve autonomous driving systems while promoting the privacy of information considered personal. The data privacy system may include a data collector, a data protector, and data users, wherein the data users process the collected data without compromising the security of the personal data contained therein. Furthermore, in the event of a data breach, any data stolen from the data protector or used in connection with the data will not disclose the personal data of one or more natural persons.

[0113] The processes, methods, or algorithms disclosed herein may be deliverable to / implemented by a processing device, controller, or computer, which may include any existing programmable electronic control unit or dedicated electronic control unit. Similarly, processes, methods, or algorithms may be stored in various forms as data and instructions executable by a controller or computer, including but not limited to information permanently stored on non-writable storage media such as ROM devices and information changeably stored on writable storage media such as floppy disks, magnetic tapes, CDs, RAM devices, and other magnetic and optical media. Processes, methods, or algorithms may also be implemented in a software executable object. Alternatively, suitable hardware components (such as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), state machines, controllers) or other hardware components or devices, or a combination of hardware, software, and firmware components, may be used to embody processes, methods, or algorithms, wholly or partially.

[0114] While exemplary embodiments have been described above, they are not intended to describe all possible forms covered by the claims. The language used in this specification is descriptive rather than limiting, and it should be understood that various changes may be made without departing from the spirit and scope of this disclosure. As previously stated, features of various embodiments may be combined to form other embodiments of the invention that may not be explicitly described or illustrated. While various embodiments may have been described as providing advantages over or preferred over other embodiments or prior art implementations in one or more desired characteristics, those skilled in the art will recognize that one or more features or characteristics may be compromised to achieve desired overall system properties depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, lifecycle cost, merchantability, appearance, packaging, size, suitability, weight, manufacturability, ease of assembly, etc. Accordingly, any embodiment described as less desirable than other embodiments or prior art implementations in one or more characteristics is not outside the scope of this disclosure and may be desirable for a particular application.

Claims

1. A method for managing personal data associated with a vehicle, comprising: Provide the vehicle with one or more random masks; In response to providing the one or more random masks, sensor data associated with the vehicle is received at a first back-end computer, wherein the receiving includes receiving a first portion of the masking share and a second portion of the masking share, wherein the first portion of the masking share and the second portion of the masking share each include personal data; Determining the labeling of sensor data includes: determining personal data and determining non-personal data separate from the personal data, wherein each of the personal data and non-personal data includes labeled data, wherein the personal data includes information relating to at least one identified or identifiable natural person; as well as Data processing associated with collecting sensor data related to the vehicle is performed via personal data and non-personal data separate from personal data, wherein the data processing is performed by communicating with a second back-end computer according to a multi-party computation (MPC) framework, such that neither the first portion of the masking share nor the second portion of the masking share is shared between the first and second back-end computers.

2. The method according to claim 1, wherein, The sensor data is collected by the vehicle while it is operating in autonomous driving mode, or by the infrastructure associated with the vehicle operating in autonomous driving mode.

3. The method of claim 1, wherein the personal data comprises image data of the at least one identified or identifiable natural person, wherein the image data is captured by at least one sensor in the vehicle, and wherein the image data comprises one or more of the following: human biometric information of the at least one identified or identifiable natural person, physical characteristics of the at least one identified or identifiable natural person, address number associated with the at least one identified or identifiable natural person, license plate number or other vehicle information associated with the at least one identified or identifiable natural person, or neighbor information associated with the at least one identified or identifiable natural person.

4. The method of claim 1, wherein the sensor data includes image data, and the image data includes personally identifiable information (PII) of the at least one identified or identifiable natural person.

5. The method of claim 1, wherein receiving sensor data associated with the vehicle further comprises receiving masked sensor data, wherein the masked sensor data includes both personal data and non-personal data.

6. The method of claim 1, wherein determining the labeling of the sensor data further comprises providing at least a portion of the sensor data to a third-party server; and receiving the labeled sensor data as a return in response to providing the at least a portion of the sensor data to the third-party server.

7. The method of claim 1, further comprising: Before receiving sensor data at the first back-end computer, the vehicle is provided with a cryptographic key from the Trusted Execution Environment (TEE); In response to providing the cryptographic key to the vehicle, at least a portion of sensor data encrypted with the cryptographic key is received; and then, within the TEE, the decrypted sensor data is determined.

8. The method of claim 7, further comprising: Before determining the labeling of sensor data, personal data is separated from non-personal data within the TEE.

9. The method of claim 7, further comprising: Non-personal data is stored in a database; personal data is encrypted with a sealing key after the sensor data to be decrypted is determined. And then the personal data, encrypted with a sealing key, is stored in the database.

10. The method of claim 9, further comprising: Proof of a dependent enclave allows the enclave to retrieve personal data using a copy of the sealed key stored within its TEE, combined with the enclave's unique signature.

11. The method of claim 7, further comprising: Request one or more random masks from the second backend computer; request one or more random masks from the third backend computer; perform a first masking of the decrypted sensor data; Perform a second masking of the decrypted sensor data; and provide the first masking of the decrypted sensor data to a second backend computer and the second masking of the decrypted sensor data to a third backend computer, such that the second and third backend computers can process the sensor data according to a multi-party computation (MPC) framework, thereby maintaining the separation between the sensor data associated with the first masking and the sensor data associated with the second masking.

12. The method of claim 7, wherein the labeling of the sensor data occurs within the TEE.

13. The method according to claim 1, wherein, At least a portion of the sensor data received at the first back-end computer is encrypted with a cryptographic key of a Trusted Execution Environment (TEE) within the first back-end computer; after receiving the at least a portion of the sensor data, a masking share is generated for the first portion of the sensor data within the TEE, and a masking share is generated for the second portion of the sensor data within the TEE; the masking share for the first portion is provided to the second back-end computer; and the masking share for the second portion is provided to the third back-end computer, such that at least one of the second or third back-end computers performs data processing.

14. The method according to claim 1, wherein, At least a portion of the sensor data received at the first back-end computer includes a first masking share, and further includes: providing the first masking share to a trusted execution environment (TEE) within another computer, such that the TEE can perform annotation or data processing, or both, wherein the TEE receives the first masking share from the first back-end computer and receives a second masking share associated with the sensor data from a second back-end computer, wherein the first and second back-end computers participate according to a multi-party computation (MPC) framework.

15. The method of claim 1, wherein determining the separation of personal data from non-personal data, determining labeling, or performing data processing occurs within a Trusted Execution Environment (TEE) associated with the primary or secondary enclave.

16. The first back-end computer includes: One or more processors; and A memory storing a plurality of instructions executable by the one or more processors, wherein the plurality of instructions includes: Provide the vehicle with one or more random masks; In response to providing the one or more random masks, sensor data associated with the vehicle is received at a first back-end computer, wherein the receiving includes receiving a first portion of the masking share and a second portion of the masking share, wherein the first portion of the masking share and the second portion of the masking share each include personal data; Determining the labeling of sensor data includes: identifying personal data and identifying non-personal data separate from the personal data, wherein each of the personal data and non-personal data includes labeled data, wherein the personal data includes information relating to at least one identifiable or identifiable natural person; and Data processing associated with collecting sensor data related to the vehicle is performed via personal data and non-personal data separate from personal data, wherein the data processing is performed by communicating with a second back-end computer according to a multi-party computation (MPC) framework, such that neither the first portion of the masking share nor the second portion of the masking share is shared between the first and second back-end computers.

17. The first back-end computer according to claim 16, wherein, The plurality of instructions further include: providing a cryptographic key from a Trusted Execution Environment (TEE) to the vehicle before receiving sensor data at a first back-end computer; receiving at least a portion of sensor data encrypted with the cryptographic key in response to providing the cryptographic key to the vehicle; and then, within the TEE, determining the decrypted sensor data.

18. The first back-end computer of claim 17, wherein the plurality of instructions further comprises: Request one or more random masks from the second backend computer; request one or more random masks from the third backend computer; perform a first masking of the decrypted sensor data; Perform a second masking of the decrypted sensor data; and provide the first masking of the decrypted sensor data to a second backend computer, and provide the second masking of the decrypted sensor data to a third backend computer.

19. A non-transitory computer-readable medium comprising a plurality of instructions stored thereon, wherein the plurality of instructions are executable by one or more processors of a first back-end computer, wherein the plurality of instructions includes: Provide the vehicle with one or more random masks; In response to providing the one or more random masks, sensor data associated with the vehicle is received at a first back-end computer, wherein the receiving includes receiving a first portion of the masking share and a second portion of the masking share, wherein the first portion of the masking share and the second portion of the masking share each include personal data; Determining the labeling of sensor data includes: determining personal data and determining non-personal data separate from the personal data, wherein each of the personal data and non-personal data includes labeled data, wherein the personal data includes information relating to at least one identified or identifiable natural person; and Data processing associated with collecting sensor data related to the vehicle is performed via personal data and non-personal data separate from personal data, wherein the data processing is performed by communicating with a second back-end computer according to a multi-party computation (MPC) framework, such that neither the first portion of the masking share nor the second portion of the masking share is shared between the first and second back-end computers.

Images