Optical Encryption Camera

Abstract

The optical encryption camera includes a sensor array (404) and a filter (402) positioned above the sensor array to receive light before the sensor array. The filter, in turn, includes a multiplexing mask (106) and a scaling mask (108). The multiplexing mask and the scaling mask combine to provide an encryption key for encrypting image data prior to capture.

Description

[Technical Field]

[0001] Related Application Information This application claims priority to Provisional Patent Application No. 63 / 421,674, filed November 2, 2022, Provisional Patent Application No. 63 / 423,076, filed November 7, 2022, Provisional Patent Application No. 63 / 423,077, filed November 7, 2022, Provisional Patent Application No. 63 / 460,056, filed April 18, 2023, and U.S. Patent Application No. 18 / 498,677, filed October 31, 2023, all of which are incorporated by reference in their entireties.

[0002] The present invention relates to optical encryption, and more particularly to a system, apparatus and method for encrypting image information prior to image capture. [Background technology]

[0003] As camera-based technology becomes increasingly integrated, the risk of sensitive visual data being compromised also increases. A standard approach to mitigating this risk is to apply software or hardware-based encryption methods to sensitive visual data after image capture. However, such methods are susceptible to data sniffing attacks that access sensitive data before it is encrypted. Summary of the Invention

[0004] According to one aspect of the present invention, an optical encryption camera includes a sensor array and a filter disposed over the sensor array and receiving light before the sensor array, the filter in turn including a multiplexing mask and a scaling mask, the multiplexing mask and the scaling mask combined to provide an encryption key for encrypting image data prior to capture.

[0005] According to another aspect of the invention, an optical encryption method includes encrypting an image by applying a multiplexing mask to the image and applying a scaling mask to the image with an optical encryption camera to provide an encrypted image. The encrypted image is received on a sensor array, and a combination of the multiplexing mask and the scaling mask generates an encryption key for encrypting the image data prior to capture. The encrypted image is stored.

[0006] These and other features and advantages will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings. [Brief explanation of the drawings]

[0007] The present disclosure provides details in the following description of preferred embodiments with reference to the following figures.

[0008] [Figure 1] FIG. 1 is a block / flow diagram illustrating an optical encryption camera coupled to a mask generation module and a training module for a mask generator, according to one embodiment of the present invention.

[0009] [Figure 2] FIG. 1 is a block diagram illustrating an optical encryption camera with a connectable decryption module connected thereto, according to one embodiment of the present invention.

[0010] [Figure 3] FIG. 10 illustrates a multiplexing mask and a scaling mask combined to provide a composite mask image, according to one embodiment of the present invention.

[0011] [Figure 4] 1 is a block diagram illustrating a surveillance system using optical encryption cameras, according to one embodiment of the present invention.

[0012] [Figure 5] 1 is a block diagram illustrating a medical imaging system that uses optical encryption cameras to maintain confidential patient data, according to one embodiment of the present invention.

[0013] [Figure 6] FIG. 1 is a block diagram illustrating a facial recognition system that uses an optical encryption camera to maintain sensitive bio-data, according to one embodiment of the present invention.

[0014] [Figure 7] FIG. 1 is a flow diagram illustrating a method for encrypting images to protect them from cyber attacks, according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] According to embodiments of the present invention, systems, devices, and methods are provided that prevent access to sensitive visual data by encrypting it prior to image capture via optical encoding of the incident light field (incoherent light). In one embodiment, image capture also allows for high-quality software-based keyed decryption.

[0016] In a useful embodiment, an optical encryption camera can include a camera with the computational power to encrypt image data prior to image capture. This can be done by optically encoding the incident light field. Recovering unencrypted image data is only possible with access to the camera's encryption key, which is defined by each camera's unique optical elements. In a useful embodiment of the invention, the camera includes a bare sensor array with an optical scaling mask coplanar with the sensor pixels and an optical multiplexing mask positioned slightly (e.g., a few millimeters) above the scaling mask. Combining the camera with an algorithm that generates the unique optical mask produces images with desirable security and image properties, such as high-quality, calibration-free keyed decryption and robustness against various attacks, including blind decryption attacks and point spread function estimation attacks that exploit bright illumination sources in the scene.

[0017] The embodiments described herein may be entirely hardware, entirely software, or contain both hardware and software elements. In a preferred embodiment, the invention is implemented in software, which includes but is not limited to firmware, resident software, microcode, etc.

[0018] Embodiments may include a computer program product accessible from a computer-usable or computer-readable medium that provides program code for use by or in connection with a computer or any instruction execution system. A computer-usable or computer-readable medium may include any apparatus that stores, communicates, propagates, or transfers a program for use by or in connection with an instruction execution system, apparatus, or device. The medium may be a magnetic, optical, electronic, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium. The medium may include a computer-readable storage medium such as a semiconductor or solid-state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, an optical disk, or the like.

[0019] Each computer program may be tangibly stored on a machine-readable storage medium or device (e.g., program memory or magnetic disk) readable by a general-purpose or special-purpose programmable computer, and may be read by the computer to configure and control the operation of the computer when the storage medium or device executes the procedures described herein. The system of the present invention may also be considered to be embodied in a computer-readable storage medium configured with a computer program, the storage medium so configured causing the computer to operate in a specific, predefined manner to perform the functions described herein.

[0020] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements via a system bus. The memory elements may include local memory used during actual execution of the program code, mass storage, and cache memory for temporarily storing at least some program code to reduce the number of times code must be retrieved from mass storage during execution. Input / output (I / O) devices (including, but not limited to, keyboards, displays, pointing devices, etc.) may be coupled to the system either directly or through intervening I / O controllers.

[0021] Network adapters, attached to a system, enable the data processing system to become connected to other data processing systems, remote printers, or storage devices through intervening private or public networks. Modems, cable modems, and Ethernet cards are just a few examples of currently available network adapters.

[0022] Referring now in detail to the drawings, like reference numerals in the figures represent like or similar elements. Referring initially to FIG. 1, an exemplary high-level system / method for optical encryption according to one embodiment of the present invention is shown. An optical encryption camera 102 according to one embodiment is shown. The optical encryption camera 102 is coupled to a mask generation module 202 and a mask generator training module 300. In use, after the optical encryption camera 102 has been trained, it can be detached from the modules 202 and 300. The mask generation module 202 and the mask generator training module 300 can be stored in the memory of a computing device (such as a portable computer, mobile phone, laptop, etc.) or can be implemented as dedicated computing hardware devices.

[0023] The optical encryption camera 102 includes two modules: a sensing module 120 and a decryption module 130. The sensing module 120 directly captures an encrypted image 110 of the incident light field 104 via optical multiplexing 106 and optical scaling 108. The decryption module 130 accepts the encrypted image 110 as input and returns a decrypted image (predicted input image) 112.

[0024] The sensing module 120 converts the incident light field 104 into encrypted measurements by optical computation. The module 120 contains a bare sensor array with an optical scaling mask 108 located coplanar with the sensor array and an optical multiplexing mask 106 located slightly (1-5 mm) above the scaling mask 108. These two masks constitute the "encryption key." Decryption is impossible without knowledge of the characteristics of these masks.

[0025] The decryption module 130 accepts the encrypted image 110 as input and returns the decrypted image 112. Decryption is achieved by applying the inverse of the calculations applied by the two optical masks 106 and 108. Therefore, decryption requires knowledge of the mask design.

[0026] The mask generation module 202 is connected between the mask generator training algorithm 300 and the optical encryption camera 102. The mask generation module 202 includes a mask generator that generates a multiplexing mask and a scaling mask. The multiplexing mask generator 206 receives a random seed 204 as input and outputs a corresponding multiplexing mask to the optical multiplexing mask 106. Similarly, the scaling mask generator 208 receives a random seed 210 as input and outputs a scaling mask corresponding to the optical scaling mask 108. The mask generators 206 and 208 can use hand-designed and / or trained algorithms for mask generation. Without training, uniform noise can be used as the generating function for the scaling mask and colored noise can be used as the generating function for the multiplexing mask. Colored noise can be used to thwart attacks based on autocorrelation.

[0027] The mask generator training algorithm 300 is connected to the mask generation module 202. The mask generator training includes a differentiable mask module 338. The mask generator training includes instantiating a differentiable multiplex generator 322 and a differentiable scaling mask generator 324 with learnable parameters. This includes providing a multiplex mask seed 320 and a scaling mask seed 326. The learnable parameters include pixel values ​​for the scaling mask and the multiplex mask. These masks can be amplitude masks or phase masks. For amplitude masks, the pixel values ​​represent opacity. For phase masks, the pixel values ​​represent phase shifts.

[0028] Next, a simulated optical encryption camera module 336 is generated, which includes generating an optical multiplexing mask 314 for the simulated camera and generating an optical scaling mask 316. The input image 312 is masked (simulated multiplexing and scaling) to simulate an encrypted digital image 318.

[0029] The keyed decryption module 334 provides keyed decryption of the input image 312. The blind decryption module 332 provides blind decryption of the input image 312. The keyed separation 308 is performed after descaling 310 of the encrypted image 318. A blind decryption 304 is also performed on the encrypted image 318 to recover an estimate of the input image. The estimates of the input image are a predicted input image (blind) 302 and a predicted input image (keyed) 306.

[0030] The parameters of the mask generators 322 and 324 are updated to minimize and maximize the reconstruction error, as determined by the adversarial loss function 330, for the keyed decryption from module 334 and the blind decryption from module 332, respectively. The approach is deemed successful if the keyed decryption is successful and the blind decryption is unsuccessful. The output of the training algorithm 300 are the learned parameters of the mask generators 322 and 324. These learned parameters are used by the mask generator module 202 and subsequently by the optical encryption camera 102.

[0031] The differentiable mask module 338 includes learnable mask generators 322 and 324. The generators 322 and 324 can be modeled as deep learning neural networks for optimizing Kamala parameters. The parameters of the neural networks are updated to minimize and maximize the reconstruction error for the keyed and blind decoding processes, respectively.

[0032] The simulated optical encryption camera module 336 can use Fourier optics to simulate the optical multiplexing (314) and scaling (316) with respective optical masks. The blind decoding in module 332 can be modeled as a deep learning neural network and trained in conjunction with trainable mask generators 322 and 324, or can employ a traditional blind deconvolution algorithm.

[0033] Our two-mask design makes blind decryption more difficult by expanding the key space and thwarts mask estimation attacks that exploit bright illumination sources in the scene. While a single multiplexed mask would reveal the mask due to a bright illumination source, a two-mask design does not reveal either mask.

[0034] The trained, hand-designed optical encryption masks generated in accordance with embodiments of the present invention improve image security compared to existing approaches. Using colored noise to generate multiplexed masks thwarts autocorrelation attacks. By employing learning in the mask generator, the present embodiments learn to generate masks that thwart a wider range of attacks.

[0035] Referring to FIG. 2, a block diagram illustrating in greater detail a lensless computational camera 102 that encrypts image data before or during image capture is depicted, according to one embodiment. The camera 102 receives incident light-field light through an optical filter 402. The optical filter 402 includes two masks that filter the light according to an optical multiplexing mask 106 and an optical scaling mask 108 before the incident light strikes a sensor array 404. The sensor array 404 can include any suitable image capture device. In one embodiment, the sensor array 404 includes an integrated circuit chip or multiple integrated circuit chips configured with optical sensors for capturing light output from the optical filter 402. The mask of the optical filter 402 is programmable and can include features and designs of the mask itself or stored in a memory 406. A hardware processor 408 interacts with the memory 406, the sensor array 404, and the masks 106 and 108 to control the operation of the camera 102. The camera 102 can include a power supply 412 for powering the functions of the camera 102. The power source 412 can be a fixed or portable power source, as desired. A control panel 414 can be provided to allow interaction with the camera 102, including programming, image capture, storage commands and guidance, etc.

[0036] The camera 102 also includes an interface 416, which may include a wired or wireless connection port. The interface 416 may allow for uploading and downloading of commands and data to and from the camera 102, either wired or wirelessly. In one embodiment, the masks 106, 108 may be updated with each image capture process so that each image contains a unique encryption. In another embodiment, the encryption remains constant until an update of the masks 106, 108 is requested.

[0037] In one embodiment, incoherent optical encryption including a randomly generated amplitude mask can be used for the scaling mask 108, and a randomly generated phase mask can be used for the multiplexing mask 106. The characteristics of the masks 106 and 108 are modified according to training parameters. The training parameters can be learned using a training mask generator 300 that is trained to be resistant to specific attacks. The learned parameters can be used to bias the mask design to be particularly resistant to a range of attacks, resulting in optical encryption masks that are more robust against the range of attacks.

[0038] Image reconstruction in this embodiment employs both the regularized least squares method and deep learning-based reconstruction methods. Because optical elements operate as linear operators, optically encrypted cameras are always susceptible to chosen-plaintext attacks, in which an attacker uses a large set of ciphertext-plaintext pairs to estimate the encryption function. For this reason, the practical application of optically encrypted cameras is limited to situations in which an attacker does not have physical access to the camera's location. Therefore, an attacker typically does not have physical access to the camera's location, but can access the camera's stream. In other words, an attacker can access the ciphertext (measurements) from the camera, but cannot access the corresponding plaintext (raw image), except in three special cases. This is a reasonable assumption in many situations, such as home or office security cameras.

[0039] For example, four types of attacks are considered: ciphertext-only attacks (COA) and three special cases of known-plaintext attacks: impulse known-plaintext attacks (I-KPA), uniform known-plaintext attacks (U-KPA), and uniform impulse known-plaintext attacks (UI-KPA). Bright impulse illumination sources and uniform backgrounds often appear naturally in a scene and may be recognizable in the corresponding encrypted sensor measurements. Thus, an attacker may be able to access an approximate impulse or uniform response of a sensor without physical access to the sensor.

[0040] C:

number

number

number

number

number

number

number

number

number

number

[0041] To train D, a random key is generated for each ciphertext sample in each batch, and a loss consisting of a combination of an L1 pixel loss and an L2 perceptual loss is used on the output of rectified linear unit (relu) layers (relu1_1, relu2_2, relu3_3) pre-trained on data for image classification on the ImageNet dataset. Specifically, the loss is given by:

[0042]

number

[0043] where w1=0.5 and w2=1.2.

number

number

number

[0044] The dual mask sensor design of the optical encryption camera 102 has two masks arranged coaxially. The optical scaling mask 108

number

[0045] The point spread function (PSF) generated by the multiplexing mask 106 is

number

number

number

number

number

[0046] Compared with the single-mask lensless design, the double-mask design according to the present embodiment has a larger key space, which makes decryption attacks more difficult and also suppresses impulse known-plaintext attacks and uniform known-plaintext attacks. In the single-mask design, the encryption key (PSF P) is equal to the camera's impulse response, so IKPA completely compromises the system. This is different from the double-mask design, where the scaling mask makes the system shift-variable, so the impulse response only reveals the "valid" encryption key for a single scene pixel.

[0047] Due to the structure of the forward imaging model, as in this embodiment, it is possible to employ imaging a uniform scene or a uniform scene response (USR). However, in a scene such as a plain wall, the scaling mask S may be revealed by scaling up. To reveal the scaling mask S, P*X in a uniform scene, it must also be uniform. However, due to full-size linear convolution with zero-padded boundary conditions, P*X is not uniform even in a plain wall scene. Therefore, S cannot be revealed from the USR.

[0048] In optical encryption cameras, the mask design is an important consideration for achieving good performance, i.e., enabling high-quality keyed decryption while simultaneously thwarting decryption attacks. It is desirable for the mask to have the following properties: 1) To enable high-quality lensless reconstruction, the PSF should include directional filters for all angles. 2) The autocorrelation of the PSF should not be impulsive. PSFs with impulsive autocorrelation are susceptible to autocorrelation-based correlation analysis (COA). These attacks exploit the fact that if the PSF's autocorrelation is impulsive, taking the autocorrelation of the encoded measurements removes the mask component. This reduces the COA problem to recovering the image from the autocorrelation. 3) The PSF generated by the multiplexed mask should not be binary. A binary PSF could be revealed to an attacker when a point source appears in the scene. While the double-mask design according to this embodiment provides defense against such attacks, a compromised PSF is still undesirable because the key size is significantly reduced.

[0049] Consider the one-dimensional case of a binary PSF p and a positive scaling mask s. The measurements y(n) recorded on the sensor for a single point source are given by

number

number

[0050] The PSF by the multiplexing mask 106 is P=αP colr +(1-α)P cont where P cont is the binary contour PSF obtained from Perlin noise, and P colris a colored noise with a constant roll-off. Perlin contours, known for high-quality lensless imaging, have impulse-like autocorrelation. On the other hand, P colr has a smoother, non-impulsive autocorrelation with varying roll-off. The Perlin feature size is fixed, and the permutation vector of the Perlin noise is randomized, so the contours of the PSF P cont The length of the permutation vector is the same as the height or width of the PSF. P cont To generate, for example,

number

[0051] Corresponding Colored Noise P colr (β) is

number

number

number

number

[0052] For the scaling mask 108 (S), colored noise is again used without Perlin contours. For a given (S, P) pair, the noise color (β) is determined by the sum of S and P. aoir is the same for both, avoiding attacks by filtering either component. Note that S must be positive, as we do not want to throw away information from any pixel of the sensor.

[0053] The performance of two different multiplexed mask designs can be compared under an autocorrelation-based ciphertext-only attack. The mask designs compared are:

number

[0054] P W The autocorrelation of is close to an impulse (P white and P cont Since the autocorrelation of is impulsive, P W It is possible to reconstruct the underlying scene from the autocorrelation of measurements of P O This does not apply in the case of

[0055] The decryption module 130 accepts three inputs: the encrypted image Y, the scaling mask S, and the PSF P with the multiplexing mask M, and outputs a decrypted image

number

number

[0056] where Y N is the measurement value after scaling normalization, i.e.

number

[0057] In the second step, the initial estimates XT, the measurements Y, and the multiplexed mask PSF P are used to compute a dense predictive transformer (DPT), i.e.

number

number

number

[0058] In one embodiment, the mask generation module 202 (FIG. 1) can be stored in the memory 406 of the camera 102. The mask generation module 202 can generate the scaling and multiplexing masks for the camera 102 in real time, or can generate and store the masks until they are updated. In one embodiment, mask generation is performed using a mask generation function (MGF) or multiple mask generation functions. An MGF is a cryptographic primitive similar to a cryptographic hash function. The mask is defined over an array of pixels tiled or replicated across the image.

[0059] The encrypted digital image 110 can be output directly in digital form or can be input to a decryption module 130, which can be part of the camera 102 or stored in a separate memory. In one mode, the camera 102 can be used in conjunction with the decryption module 130. The decryption module 130 can include a display 410. Alternatively, the decryption module 130 can be used in a separate computing / processing device with its own display. The camera 102 can capture an image that can be displayed on the display 410 only if proper authorization is given. In other words, the decryption steps, including descaling 116 and separation 114, can be performed. To perform the descaling 116 and separation 114, the encryption key provided according to the optical multiplexing mask 106 and the optical scaling mask 108 must be accessible / known.

[0060] In another mode, the camera 102 can be used to output and store an encrypted image 110. The encrypted image 110 can be transmitted in its encrypted form for storage and use independent of the camera 102. In one example, a sensitive photograph, such as one containing classified information, can remain protected from the time the image is captured until it is properly decrypted and is never stored in an unencrypted state.

[0061] Because the filter 402 includes masks 106 and 108 that are optimized (e.g., trained using artificial intelligence) to be particularly resistant to various potential cyberattacks, the encrypted image 110 remains protected from cyberattacks and other security risks. The decryption module 130 can include a display 410. Alternatively, the decryption module 130 can be used in a separate computing / processing device with its own display. The camera 102 can capture an image (light 104) that can be displayed on the display 410 only with appropriate authorization. In other words, the decryption steps, including descaling 116 and separation 114, can be performed. To perform the descaling 116 and separation 114, the encryption key provided in response to the optical multiplexing mask 106 and the optical scaling mask 108 must be accessible / known.

[0062] Referring to FIG. 3, an exemplary multiplexing mask 506 and scaling mask 108 are depicted. When a frame 502 is captured and filtered through the multiplexing mask 506 and scaling mask 108, a composite mask image 504 is obtained. The multiplexing mask 506 includes a randomization / multiplexing pattern for pixel masking, and the scaling mask 108 blocks out the randomized portions of the image. The types and characteristics of the multiplexing mask 506 and scaling mask 108 can be trained and optimized for a specific cyber attack or various cyber attacks. The optimization involves applying the masks 106 and 108 according to a loss function using a deep neural network. The deep neural network uses training data from images filtered in different ways to determine parameters to be updated to minimize and maximize the reconstruction error, as determined by the adversarial loss function 330 (FIG. 1), for keyed decoding and blind decoding, respectively. The neural network parameters are updated to minimize and maximize the reconstruction error, as determined by the adversarial loss function 330 (FIG. 1), for keyed decoding and blind decoding, respectively. If the keyed decryption is successful and the blind decryption is unsuccessful, the approach is considered successful.

[0063] Our two-mask design makes blind decryption more difficult by expanding the key space and thwarts mask estimation attacks that exploit bright illumination sources in the scene. While a single multiplexed mask would reveal one mask due to a bright illumination source, a two-mask design does not reveal either mask.

[0064] Referring to Figure 4, a camera system 600 according to one embodiment of the present invention is shown. The camera system 600 includes one or more optical encryption cameras 102. The cameras 102 may be mounted on towers, buildings, or other stable locations. The cameras 102 may also be mounted on mobile platforms such as trucks, airplanes, drones, or automobiles. The cameras 102 may use wired or wireless connections to collect images captured by the cameras 102.

[0065] The camera 102 may be utilized for surveillance functions, to monitor a location or object 606, or to capture images of a particular environment or situation. Images captured by the camera 102 may contain sensitive information. Therefore, the image data may be encrypted at the time the image is captured. There may be several reasons for encrypting this data. For example, the images may need to be protected from viewing by unauthorized persons or others who should not be able to view images captured by the camera 102. Furthermore, the image data may be of a sensitive nature, for example, if the system 600 is located in or around a home. In addition to being encrypted, the image data may also desirably be more resistant to cyber-attacks if the computer 602 or device storing the image data in memory 604 is compromised by a cyber-attack.

[0066] Referring to Figure 5, a camera system 700 for use in a medical environment protects sensitive image data in accordance with an embodiment of the present invention. In a medical environment, medical professionals often need to photograph patient injuries for record or examination purposes. In other situations, photographs are needed to plan surgery or other treatment programs. These images are sensitive due to their personal nature and the doctor-patient relationship, especially if they depict the patient's face or other sensitive body parts.

[0067] The camera system 700 comprises one or more optical encryption cameras 102. The cameras 102 may be installed on fixed or mobile platforms within a medical facility or may be operated by medical personnel. Wired or wireless connections may be employed to collect the images captured by the cameras 102.

[0068] The camera 102 can be used to capture images of a patient 702 in a medical environment or situation (such as an accident scene or other medical emergency). Images captured by the camera 102 may contain sensitive information; therefore, the image data is encrypted at the time the image is captured. The images need to be protected from viewing by non-medical personnel, unauthorized persons, or others who should not be able to view images captured by the camera 102. Due to the sensitive nature of the image data, e.g., medical privacy, the image data should not only be encrypted but also be further resistant to cyber-attacks if a computer 706 or device storing the image data in memory 708, such as a medical database or other memory, is compromised by a cyber-attack. Medical images can be securely protected and optimized for easy access by appropriate personnel.

[0069] 6, in another embodiment, a facial recognition application is used in conjunction with the optical encryption camera 102 according to the present invention. An encrypted query image 802 of a human face is captured using the optical encryption camera 102. The encrypted query image 810 is sent to a computer 806 to perform facial recognition. The computer 806 includes a processor 804 and a memory 808. The memory 808 may include an application or program for facial recognition.

[0070] The computer 806 can perform facial recognition directly in the encrypted domain, or it can first decrypt the query image and then perform facial recognition. If recognition is performed in the encrypted domain, the encrypted query image is compared to encrypted images in a reference database 812. Otherwise, the decrypted image is compared to conventional images in the reference database 812. The image data is compared to the images in the reference database and a similarity score 814 and / or image match 816 is output. Because the optical encryption camera 102 encrypts image data before image capture, this implementation can thwart data sniffing attacks that attempt to access the raw image data.

[0071] Facial recognition can be used as a security measure in payment applications, healthcare applications, building and other access applications, etc. Image data stored for facial, retinal, or other personal data collection is kept encrypted and can only be decrypted if the encryption key is known.

[0072] Optical encryption according to this embodiment can employ standard optical masks, which are cost-effective and easy to mass-produce. For example, an optical scaling mask can be used to apply pixel gain without modifying the camera hardware. Lensless imaging can be used to provide a separable coded aperture mask over the bare sensor array, enabling imaging devices with thin, flat form factors that can simulate conventional cameras by reconstructing conventional images from coded measurements. Coded phase masks can also be used to improve optical efficiency and reconstruction quality. In a useful embodiment, a coded optical mask is used to capture the coded measurements, enabling high-quality reconstruction of the image while also preventing decoding attacks. A second optical scaling mask is placed coplanar with the bare sensor array, and the mask design for the multiplexing and scaling masks is trained to enhance security by making the underlying image undetectable.

[0073] Image reconstruction is a core problem in computational imaging and plays a key role in lensless imaging. In accordance with embodiments of the present invention, both regularized least squares and deep learning-based reconstruction methods are used.

[0074] Referring to FIG. 7, an exemplary method for optical encryption according to an embodiment of the present invention is illustrated. At block 900, a mask for encrypting image data is generated. The multiplexing mask and scaling mask can be generated using randomly generated seeds that can be used to calculate the mask using a mask generation calculation. In one embodiment, the mask can be generated using deep learning neural networks. These networks can use simulated images to determine the most effective mask for protecting the image. The parameters of the mask are trained to provide the most robust mask against a particular threat or a range of threats.

[0075] The mask generator training module can be trained using keyed and blind simulations of input images to generate masks that are resistant to various cyber attacks. The mask generator training module can be optimized according to an adversarial loss function, where the adversarial loss function is minimized with respect to reconstruction error for measuring decoding accuracy and maximized with respect to reconstruction error for blind decoding. The mask generator training module can further include one or more mask generators trained using a deep learning neural network.

[0076] In block 902, the optical encryption camera encrypts the image by applying a multiplexing mask and then a scaling mask to the image to generate an encrypted image. This encryption is performed before the image is captured by the sensor array in the camera. For example, the optical encryption camera can capture medical images containing sensitive data, which is stored in encrypted form in memory. The optical encryption camera can also capture facial images and perform facial recognition using the encrypted images. Other applications are also possible. It should be understood that the image can also be decrypted and stored in a decrypted (unencrypted) form.

[0077] In block 904, an encrypted image is received on the sensor array and combined with a multiplexing mask and a scaling mask to provide an encryption key for encrypting the image data before capture. In block 906, the encrypted image is stored. This may include storing the image in camera memory or sending the image to a stored image database. The image may be stored in an encrypted state.

[0078] At block 908, a connectable decoding module can be used to decode the captured image by inverting the multiplexing and scaling masks. In other embodiments, the decoding model can be provided in the camera itself. At block 910, the decoded image can be displayed.

[0079] Simulations and real-world experiments have shown that optical encryption cameras are more robust against various cyber attacks. For example, embodiments of the present invention provide resistance against autocorrelation attacks, bright source attacks, transformer-based ciphertext attacks, etc.

[0080] As used herein, the terms “hardware processor subsystem” or “hardware processor” may refer to a processor, memory, software, or combination thereof that cooperate to perform one or more specific tasks. In useful embodiments, a hardware processor subsystem may include one or more data processing elements (e.g., logic circuits, processing circuits, instruction execution units, etc.). The one or more data processing elements may be included in a central processing unit, a graphics processing unit, and / or a separate processor (or computing element)-based controller (e.g., logic gates, etc.). A hardware processor subsystem may include one or more on-board memories (e.g., caches, dedicated memory arrays, read-only memories, etc.). In some embodiments, a hardware processor subsystem may include one or more memories that may be on-board or off-board or dedicated for use by the hardware processor subsystem (e.g., ROM, RAM, basic input / output system (BIOS), etc.).

[0081] In some embodiments, the hardware processor subsystem may include and execute one or more software elements, which may include an operating system and / or one or more applications and / or specific code to achieve a particular result.

[0082] In other embodiments, the hardware processor subsystem may include specialized circuitry dedicated to performing one or more electronic processing functions to achieve a particular result. Such circuitry may include one or more application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and / or programmable logic arrays (PLAs).

[0083] These and other variations of the hardware processor subsystem are also contemplated in accordance with embodiments of the present invention.

[0084] References herein to "one embodiment" or "an embodiment" of the present invention, as well as other variations, mean that a particular feature, structure, characteristic, etc. described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrase "in one embodiment" or "in an embodiment," as well as any other variations, in various places throughout this specification do not necessarily all refer to the same embodiment. However, it should be understood that features of one or more embodiments may be combined given the teachings of the present invention provided herein.

[0085] For example, the use of any of " / ," "and / or," or "at least one" in the cases of "A / B," "A and / or B," and "at least one of A and B" will be understood to be intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of both alternatives (A and B). As a further example, in the cases of "A, B, and / or C" and "at least one of A, B, and C," such language is intended to encompass the selection of only the first listed alternative (A), or the selection of only the second listed alternative (B), or the selection of only the third listed alternative (C), or the selection of only the first and second listed alternatives (A and B), the selection of only the first and third listed alternatives (A and C), the selection of only the second and third listed alternatives (B and C), or the selection of all three alternatives (A, B, and C). This can be expanded as many times as there are listed items.

[0086] The foregoing is understood in all respects to be illustrative and exemplary, but not restrictive, and the scope of the invention disclosed herein is to be determined not from the detailed description, but from the claims which are to be interpreted in accordance with the full breadth and spirit of the patent laws. It will be understood that the embodiments shown and described herein are merely exemplary of the invention, and that those skilled in the art could make various modifications without departing from the scope and spirit of the invention. Various other feature combinations could be implemented by those skilled in the art without departing from the scope and spirit of the invention. Having thus described aspects of the invention with the detail and particularity required by the patent laws, what is desired to be claimed and protected by Letters Patent is set forth in the appended claims.

Claims

1. a sensor array (404); a filter (402) disposed above the sensor array and receiving light in front of the sensor array; The filter is An optical encryption camera comprising, in order, a multiplexing mask (106) and a scaling mask (108), said multiplexing mask and said scaling mask combined to provide an encryption key for encrypting image data prior to capture.

2. The optical encryption camera of claim 1 , further comprising a connectable decryption module for decrypting encrypted image data by inverse operations of the multiplexing mask and the scaling mask.

3. The optical encryption camera of claim 1 , wherein the multiplexing mask and the scaling mask are randomly generated using a seed.

4. The optical encryption camera of claim 1 , further comprising a mask generator training module for generating masks that are resistant to various cyber-attacks.

5. The optical encryption camera of claim 4 , wherein the mask generator training module is optimized according to an adversarial loss function.

6. The optical encryption camera of claim 5 , wherein the adversarial loss function is minimized with respect to a reconstruction error to measure decoding accuracy.

7. The optical encryption camera of claim 5 , wherein the adversarial loss function is maximized with respect to a reconstruction error for blind decoding.

8. The optical encryption camera of claim 4 , wherein the mask generator training module further comprises at least one mask generator trained with a deep learning neural network.

9. The optical encryption camera of claim 4 , wherein the mask generator training module further comprises a blind decoding module trained with a deep learning neural network.

10. The optical encryption camera of claim 1 , wherein the optical encryption camera captures medical images containing sensitive data, and the sensitive data is stored in memory in an encrypted state.

11. The optical encryption camera according to claim 1 , wherein the optical encryption camera captures a facial image and performs facial recognition using the encrypted image.

12. applying a multiplexing mask to an image and encrypting (902) the image by applying a scaling mask to the image with an optical encryption camera to provide an encrypted image; receiving 904 the encrypted image on a sensor array, the multiplexing mask and the scaling mask combined to provide an encryption key for encrypting image data prior to capture; storing the encrypted image.

13. 13. The method of claim 12, further comprising: decrypting the encrypted image with a connectable decryption module by using an inverse operation of the multiplexing mask and the scaling mask to provide a decrypted image.

14. The method of claim 13 further comprising displaying the decoded image.

15. The method of claim 12 , wherein the multiplexing mask and the scaling mask are randomly generated using a seed.

16. The method of claim 12 , further comprising training a mask generator training module using keyed and blind simulations of input images to generate masks that are resistant to various cyber-attacks.

17. the mask generator training module is optimized according to an adversarial loss function; The adversarial loss function is minimized with respect to a reconstruction error to measure decoding accuracy; The method of claim 16 , wherein the adversarial loss function is maximized with respect to a reconstruction error for blind decoding.

18. The method of claim 16 , wherein the mask generator training module further comprises at least one mask generator trained with a deep learning neural network.

19. The method of claim 12 , wherein the optical encryption camera captures medical images containing sensitive data, and the sensitive data is stored in memory in an encrypted form.

20. The method of claim 12 , wherein the optical encryption camera captures a facial image and performs facial recognition using the image in an encrypted state.

Images