Two-dimensional photonic integrated circuit optical phased array for imaging systems

Abstract

To provide a two-dimensional photonic integrated circuit optical phased array for imaging systems.SOLUTION: Aspects of the disclosure provide a device and a method for capturing one or more images using a photonic integrated circuit (PIC) optical phased array (OPA) based imaging system. The method may include processing the environmentally or artificially lit received image using one or more injected control wavelengths or wavefronts to enable electronic control of functions such as real time color filtering, glint removal or other intensity image manipulation, image stabilization, collection of multiple images for panoramic composites, focal plane adjustment and / or other image optimization manipulations based on user input. These system level functions may be accomplished with a device utilizing a plurality of optical nano-antenna elements, a plurality of phase and amplitude modulators, a plurality of measurement photodiodes and a plurality of other associated photonic constructs along with one or more CMOS processors to process the received image.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] Cross - Reference to Related Applications

[0001] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 609,997, filed on December 14, 2023, which is hereby incorporated by reference in its entirety.

Background Art

[0002] Background

[0002] Mobile phones, tablets, laptops, and other computers, and image systems such as cameras can utilize optomechanical systems for image capture and processing. Such optomechanical systems enable post - processing of images, but may require physically moving the imaging system to capture multiple images and also stitching for post - capture panorama synthesis required for post - processing.

[0003]

[0003] Generally, current imaging technologies can be limited in capabilities due to the optomechanical systems used, which can affect functionality and / or key technical performance metrics. Examples can include limitations or lack of the ability to perform real - time color filtering and intensity manipulation, limitations in the bandwidth of image stabilization for compensating external vibrations and disturbances, the need to physically move the camera to capture multiple images with post - stitching for panorama synthesis, and limitations in the ability to collect and correct the focal plane of an image in real - time (focal plane adjustment).

Summary of the Invention

Means for Solving the Problems

[0004] Brief Summary

[0004] Aspects of the present disclosure are directed to a method of capturing one or more images using an imaging device. The method includes generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs; receiving, in an optical phased array (OPA), light from the environment of the imaging device; driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA by applying the one or more control wavelengths to process the received light; and capturing, by one or more processors of the imaging device, an image based on the processed received light.

[0005]

[0005] In one example, generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs includes generating a first control wavelength and a second control wavelength; receiving, in the OPA, light from the environment of the imaging device includes receiving light in a first time step and a second time step; driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA by applying the one or more control wavelengths to process the received light includes driving, in a first time step, the plurality of phase and amplitude modulators by applying the first control wavelength to process the light received in the first time step, and driving, in a second time step, the plurality of phase and amplitude modulators by applying the second control wavelength to process the light received in the second time step; and capturing, by one or more processors of the imaging device, an image based on the processed received light includes capturing a first image in a first time step based on the processed light received in the first time step, and capturing a second image in a second time step based on the processed light received in the second time step. Additionally, the method further includes generating, by one or more processors of the imaging device, a composite image of the first image and the second image.

[0006]

[0006] In another example, the method further includes transmitting, by an OPA of the imaging device, one or more control wavelengths; reflecting, by the OPA of the imaging device, one or more control wavelengths back; and receiving, by the OPA of the imaging device, one or more control wavelengths.

[0007]

[0007] In another example, one or more user inputs are received from a user interface.

[0008]

[0008] In a further example, driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA to process received light by applying one or more control wavelengths includes measuring, by one or more photodiodes, one or more measurements associated with the one or more control wavelengths; and driving the plurality of phase and amplitude modulators based on the one or more measurements associated with the one or more control wavelengths.

[0009]

[0009] In another example, driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA to process received light by applying one or more control wavelengths includes measuring, by one or more photodiodes, one or more measurements associated with the received light; and driving the plurality of phase and amplitude modulators based on the one or more measurements associated with the received light.

[0010]

[0010] In another example, driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA to process received light by applying one or more control wavelengths includes measuring, by one or more photodiodes, one or more measurements associated with the one or more control wavelengths and the received light; and driving the plurality of phase and amplitude modulators based on the one or more measurements associated with the one or more control wavelengths and the received light. Additionally, the one or more measurements can be at least one of i) intensity, ii) power, and iii) relative phase.

[0011]

[0011] In an additional example, the method further includes transmitting, by an OPA, the processed received light to a second OPA, and transmitting, by the second OPA, the processed received light to a focal plane array (FPA) of an imaging device. Additionally, the method may further include adjusting, by an amplitude modulator array of the FPA, the amount of light received by one or more elements of the FPA. Additionally or alternatively, the method may further include driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the second OPA by applying one or more control wavelengths to further process the processed received light. Additionally or alternatively, the one or more control wavelengths may be a plurality of control wavelengths. Additionally or alternatively, driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of an OPA by applying a plurality of control wavelengths to process received light may include applying a first control wavelength among the plurality of control wavelengths, and driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of a second OPA by applying one or more control wavelengths to further process the processed received light may include applying a second control wavelength among the plurality of control wavelengths, and the second control wavelength is different from the first control wavelength.

[0012]

[0012] Another aspect of the present disclosure is directed to an imaging device configured to capture one or more images. The imaging device includes an optical phased array (OPA) receiver, the OPA receiver including a plurality of emitters configured to transmit and receive light, a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device, and one or more photodiodes configured to measure one or more values associated with the light propagating through the imaging device. The imaging device further includes a focal plane array (FPA) OPA, the FPA OPA including a plurality of emitters configured to transmit and receive light, a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device, and one or more photodiodes configured to measure one or more values associated with the light propagating through the imaging device. The imaging device further includes a control unit, the control unit including one or more processors and one or more light sources configured to generate light. The imaging device further includes an FPA configured to receive light from the FPA OPA and record an image based on the received light, and the OPA receiver, the FPA OPA, and the control unit are disposed on a photonic integrated circuit (PIC).

[0013]

[0013] In one example, the OPA receiver further includes one or more reflectors associated with an emitter of the plurality of emitters, the one or more reflectors configured to reflect one or more control wavelengths back to that emitter of the plurality of emitters.

[0014]

[0014] In another example, the imaging device further includes one or more microlens arrays associated with at least one of (i) the OPA receiver and (ii) the FPA OPA.

[0015]

[0015] In another example, the imaging device additionally includes one or more sensors configured to collect one or more sensor measurements, the one or more sensors including at least one of (i) a gyroscope, an accelerometer, and an inertial measurement unit (IMU).

[0016]

[0016] In additional examples, the imaging device is arranged in a stacked configuration. In another example, one or more processors generate one or more control wavelengths based on one or more user inputs and i) apply one or more control wavelengths to a plurality of phase and amplitude modulators of an OPA receiver for processing received light, and ii) apply one or more control wavelengths to drive at least one of a plurality of phase and amplitude modulators of an FPA OPA for processing received light, and capture an image based on the processed received light.

Brief Description of the Drawings

[0017] Brief Description of the Drawings

Figure 1A

[0017] It is an image diagram of an imaging device according to an aspect of the present disclosure.

Figure 1B

[0017] It is an image diagram of an imaging device according to an aspect of the present disclosure.

Figure 2

[0018] It is a block diagram of an imaging device according to an aspect of the present disclosure.

Figure 3

[0019] It represents the characteristics of an optical phased array architecture according to an aspect of the present disclosure.

Figure 4

[0020] It represents the characteristics of an optical control architecture according to an aspect of the present disclosure.

Figure 5

[0021] It is a block diagram of a user device according to an aspect of the present disclosure.

Figure 6

[0022] It is a block diagram according to an aspect of the present disclosure.

Modes for Carrying Out the Invention

[0018] Detailed Description Overview

[0023] The present technology relates to an imaging device implemented using an optical phased array (OPA). The imaging device can be used for image capture and related image processing. The imaging device can include one or more OPAs and one or more processors. One or more OPAs and / or one or more processors can be included within a photonic integrated circuit (PIC). One or more processors can be one or more complementary metal-oxide-semiconductor (CMOS) processors. The imaging device can be configured to process received light based on one or more control wavelengths or wavefronts for phase and wavefront control of the received light and capture an image based on the processed light.

[0019]

[0024] Generally, current imaging technologies can be limited in capabilities due to the optical-mechanical systems used, which can affect functionality and / or key technical performance metrics. Examples can include limitations or lack of the ability to perform real-time color filtering and intensity manipulation, limitations in the bandwidth of image stabilization for compensating external vibrations and disturbances, the need to physically move a camera to capture multiple images with subsequent stitching for panorama synthesis, and limitations in the ability to collect and correct (focus plane adjustment) the focus plane of an image in real time.

[0020]

[0025] To address this, as pointed out above, the imaging device can be implemented using a PIC with an OPA. In this regard, the imaging device can utilize the functional capabilities of the OPA to perform complex image processing and provide improved image capture capabilities. In this regard, an imaging device implemented using a PIC-OPA can perform real-time color filtering and intensity manipulation, perform image stabilization to compensate for external vibrations and disturbances, not require its physical movement to capture multiple images for panorama synthesis, and be configured to collect and correct (focus plane adjustment) the focus plane of an image in real time.

[0021] Exemplary System

[0026] As discussed above, the imaging device may include one or more OPAs and one or more processors (e.g., a CMOS processor). FIGS. 1A - 1B illustrate an exemplary imaging device 101. FIG. 1A shows a side view of the imaging device 101, and FIG. 1B shows a top view of the imaging device 101. The imaging device 101 includes a first OPA or OPA receiver 103, a second OPA or focal plane array (FPA) OPA 104, an FPA 105, a microlens array 106 associated with the FPA OPA 104, a microlens array 107 associated with the OPA receiver 103, and a block or control unit 108. The block 108 may include one or more processors (e.g., a CMOS processor). Additionally, the block 108 may include one or more light sources (e.g., light - emitting diodes (LEDs), PIC - integrated lasers, externally coupled lasers, etc.). The OPA receiver 103, the FPA OPA 104, and the block 108 of the imaging device 101 are shown as being included on a photonic integrated circuit (PIC) 102. However, in some implementations, one or more of these components may not be included on the PIC 102 and / or may be included on a different PIC separate from the PIC 102.

[0022]

[0027] In some implementations, the microlens array 106 can be a lens, a metasurface, a microlens array, or any combination thereof. Additionally or alternatively, the microlens array 107 can be a lens, a metasurface, a microlens array, or any combination thereof.

[0023]

[0028] As shown in the side view of FIG. 1A, the imaging device can be arranged in a stacked configuration. In this regard, the microlens array 107 associated with the OPA receiver 103 can be disposed on the first surface of the imaging device 101. During operation, the first surface can face the environment of the imaging device. This environment is the object of the captured image. The microlens array 106 associated with the FPA OPA 104 can be disposed on the second surface of the imaging device 101. The second surface can be on the opposite side of the first surface. The FPA 105 can be disposed at a distance from the microlens array 106 on the second side of the imaging device 101. This distance can correspond to the focal plane of the imaging device 101. The OPA receiver 103 and the FPA OPA 104 are shown as circular, but one or both of them can be configured to be any geometric shape including ellipse, square, rectangle, hexagon, octagon, etc. Additionally or alternatively, the shapes of the OPA receiver 103 and the FPA OPA 104 can be modified to change the relative sizes and / or pixel pitches of the OPA receiver 103 input and the FPA OPA 104 output.

[0024]

[0029] The thickness of the imaging device 101 shown in FIG. 1A can be about 4 mm or more or less. The height and width of the imaging device shown in FIG. 1B can be about 10 - 24 mm or more or less.

[0025]

[0030] Figure 2 is a block diagram of an exemplary imaging device 201. The imaging device 201 includes additional components related to the imaging device 101 that can assist in image processing. The imaging device 201 includes a first OPA or OPA receiver 203, a second OPA or FPA OPA 204, an FPA 205, a microlens array 206 that may be associated with the FPA OPA 204, a microlens array 207 that may be associated with the OPA receiver 203, a block 208, and one or more sensors 209. The OPA receiver 203, the FPA OPA 204, and the block or control unit 208 of the imaging device 201 are shown as being included on the PIC 202. However, in some implementations, one or more of these components may not be included on the PIC 202 and / or may be included on a different PIC separate from the PIC 202. In some cases, the microlens arrays 206, 207, and / or one or more sensors may or may not be included on the PIC 202 and may or may not be included on a different PIC separate from the PIC 202.

[0026]

[0031] In some implementations, the microlens array 206 can be a lens, a metasurface, a microlens array, or any combination thereof. Additionally or alternatively, the microlens array 207 can be a lens, a metasurface, a microlens array, or any combination thereof.

[0027]

[0032] The OPA receiver 203 can be configured to receive light from the environment of the imaging device. This environment is the object of the captured image. The OPA receiver 203 includes one or more reflectors, partial reflectors or retroreflective photonic structures 210, a plurality of emitters 211, a plurality of phase and amplitude modulators 212, and a plurality of photodiodes (PDs) 213. The components of the OPA receiver 203 can be connected via a plurality of waveguides or optical fibers. Additionally, a microlens array 207 can be associated with the OPA receiver 203. The microlens array 207 can be a microlens layer adjacent to an emitter layer including a plurality of emitters 211. In this regard, each emitter of the emitter layer can correspond to a microlens of the microlens layer. In some cases, the plurality of emitters 211 can be a plurality of optical antennas. In some cases, the microlens array 207 can be a single lens, such as a metasurface or a photonic structure that emulates the functionality of a microlens array, for example. One or more reflectors 210 can correspond to the emitters among the plurality of emitters 211.

[0028]

[0033] The plurality of phase and amplitude modulators 212 can be formed within one or more modulator layers and can be configured to modify the phase and / or amplitude of the received (Rx) light. In this regard, the plurality of phase and amplitude modulators 212 can enable control of the Rx light characteristics. In some cases, each layer can be a layer of one or more phase modulators or one or more amplitude modulators. In some cases, one or more modulator layers can include one or more multiplexers or demultiplexers. In some cases, one or more modulator layers can include a plurality of PDs 213. The plurality of PDs 213 can be coupled to each phase and amplitude modulator within the layer of phase and amplitude modulators. In some cases, a single PD can be coupled to two or more phase and amplitude modulators of the layer via a waveguide tap coupler. The plurality of PDs 213 can be configured to measure one or more values, such as the intensity, power and / or relative phase or wavefront of the injected control optical wavelength. The measured values can then be used to control and / or analyze the Rx light (e.g., as feedback).

[0029]

[0034] The plurality of emitters 211 or optical antennas may be configured to transmit and receive light. In this regard, the plurality of emitters may be configured to receive incident light (e.g., Rx light). Additionally, the plurality of emitters may be configured to transmit one or more control wavelengths. The one or more control wavelengths may be reflected back to the plurality of emitters 211 via one or more reflectors 210. In this regard, the reflected-back control wavelengths may be used for phase and wavefront control of the Rx light.

[0030]

[0035] The FPA OPA 204 may be configured to receive light from the OPA receiver 203. Like the OPA receiver 203, the FPA OPA includes one or more reflectors, partial reflectors or retroreflective photonic structures 214, a plurality of emitters 215, a plurality of phase and amplitude modulators 216, and a plurality of PDs 217. The components of the FPA OPA 204 may be connected via a plurality of waveguides or optical fibers. Additionally, the microlens array 206 may be associated with the FPA OPA 204. The microlens array 206 may be a microlens layer adjacent to an emitter layer including the plurality of emitters 215. In this regard, each emitter of the emitter layer may correspond to a microlens of the microlens layer. In some cases, the plurality of emitters 215 may be a plurality of optical antennas. In some cases, the microlens array 206 may be a single lens such as, for example, a metasurface or a photonic structure that emulates the functionality of a microlens array. The one or more reflectors 214 may correspond to the emitters of the plurality of emitters 215.

[0031]

[0036] The plurality of phase and amplitude modulators 216 can be formed within one or more modulator layers and can be configured to modify the phase and / or amplitude of received (Rx) light. In this regard, the plurality of phase and amplitude modulators 216 can enable control of Rx light characteristics. In some instances, each layer can be a layer of one or more phase modulators or one or more amplitude modulators. In some instances, one or more modulator layers can include one or more multiplexers or demultiplexers. In some instances, one or more modulator layers can include a plurality of PDs 217. The plurality of PDs 217 can be coupled to each phase and amplitude modulator within the phase and amplitude modulator layer. In some instances, a single PD can be coupled to two or more phase and amplitude modulators of the layer via a waveguide tap coupler. The plurality of PDs 217 can be configured to measure one or more values such as the intensity, power, and / or relative phase of a control wavelength. The measured values can then be used to control and / or analyze the Rx light (e.g., as feedback).

[0032]

[0037] The plurality of emitters 215 or optical antennas can be configured to transmit and receive light. In this regard, the plurality of emitters can be configured to receive incident light (e.g., Rx light). Additionally, the plurality of emitters can be configured to transmit light to the FPA 205. Additionally, the plurality of emitters can be configured to transmit one or more control wavelengths. The one or more control wavelengths can be reflected back to the plurality of emitters 215 via one or more reflectors 214. In this regard, the reflected control wavelengths can be used for phase and wavefront control of the Rx light.

[0033]

[0038] FPA205 can be the image sensor of the imaging device 201. In this regard, FPA205 can be configured to receive light from FPA OPA204 and record an image based on the received light. The recorded image can be stored on the memory of the imaging device 201 (e.g., memory 220). In some cases, PIC202 or FPA205 can additionally include an array of amplitude modulators. The array of amplitude modulators can be used to adjust the amount of light directed to the elements of FPA205 on a per-element basis. This adjustment can also be utilized, for example, for brightness adjustment, glint removal, aperture adjustment, and other amplitude-dependent operations.

[0034]

[0039] Sensor 209 can be configured to collect one or more sensor measurements. The one or more sensors can include various types of sensors such as, for example, gyroscopes, accelerometers, inertial measurement units (IMUs), etc. The one or more sensors 209 can be configured to collect one or more sensor measurements at multiple time steps. The one or more measurements can be stored in the memory of the imaging device 201 (e.g., memory 220).

[0035]

[0040] Block 208 includes one or more processors 218 (e.g., CMOS processors), one or more light sources 219, and memory 220. The one or more light sources 219 can be light emitting diodes (LEDs), PIC integrated lasers, externally coupled lasers, etc. The one or more processors 218 can be CMOS processors. The one or more processors can be or include any conventional processor, such as a commercially available CPU. Additionally or alternatively, the one or more processors can be or include a dedicated device, such as an application specific integrated circuit (ASIC), or another hardware-based processor, such as a field programmable gate array (FPGA). FIG. 2 functionally shows the processor 218 and memory 220 within one or more blocks 208, but the one or more processors 218 and memory 220 can actually include multiple processors and memories that may or may not be placed on the PIC 202. Thus, references to processors or computers are to be understood to include references to collections of processors, or computers, or memories that may or may not operate in parallel.

[0036]

[0041] Memory 220 can store information accessible by one or more processors 218, including data and instructions that can be executed by one or more processors 218. The memory can be any type of memory that can store information accessible by a processor, including computer-readable media such as hard drives, memory cards, ROM, RAM, DVDs, or other optical disks, and other writable and read-only memories. The system and method can include various combinations of the foregoing, whereby various portions of data and instructions are stored on various types of media.

[0037]

[0042] Data can be obtained, stored, or modified by one or more processors 218 according to instructions. For example, although the present system and method are not limited by any particular data structure, data can be stored in a computer register within a relational database as a table having a plurality of different fields and records, an XML document, or a flat file. The data can be formatted in any computer-readable format, such as, but not limited to, binary or Unicode. Merely as a further example, image data can include a bitmap including a grid of pixels stored according to a lossless (e.g., BMP) or lossy (e.g., JPEG) and bitmap or vector-based (e.g., SVG) format that is compressed or decompressed, as well as computer instructions for drawing a graphic. The data can include any information sufficient to identify related information, such as numbers, descriptive text, proprietary specification code, references to other data stored in the same memory or other areas of different memory (including other network locations), or information used by a function that calculates related data.

[0038]

[0043] Instructions can be any combination of instructions (such as machine code) directly executed by one or more processors 218 or instructions (such as scripts) indirectly executed. For example, instructions can be stored as computer code on a computer-readable medium. In this regard, the terms "instructions" and "program" can be used interchangeably herein. Instructions can be stored in object code form for direct processing by one or more processors 218 or in any other computer language including scripts or collections of independent source code modules that are interpreted as needed or pre-compiled.

[0039]

[0044] In some cases, one or more processors 218 may be operably connected to at least one of the PDs and a plurality of PDs 213 of the OPA203, a plurality of PDs 217 of the FPA OPA204, or at least one layer of both the phase and amplitude modulators. In this regard, one or more processors 218 may be configured to drive (e.g., modify the output phase and / or output amplitude) at least one layer of the plurality of phase and amplitude modulators 212 of the OPA203, the plurality of phase and amplitude modulators 216 of the FPA OPA204, or both. Additionally, one or more processors 218 may be configured to utilize the values measured by the PDs to drive at least one layer of the OPA203, the FPA OPA204, or both phase and amplitude modulators.

[0040]

[0045] One or more processors 218 may be additionally configured to generate one or more control wavelengths via one or more light sources 219 (e.g., light emitting diodes (LEDs), PIC integrated lasers, externally coupled lasers, etc.). One or more processors 218 may be further configured to apply one or more control wavelengths to an optical waveguide that co-propagates with the Rx light within the OPA203, the FPA OPA204, or both. In this regard, one or more processors 218 may drive the plurality of phase and amplitude modulators 212 and 216 to apply one or more control wavelengths to facilitate functional processing of the Rx light. In this regard, one or more processors 218 of the imaging device 201 may record or capture an image of the processed light. The imaging device 201 may record the image at the FPA 205 of the imaging device 201. The recorded image may be stored on the memory 220 of the imaging device 201.

[0041]

[0046] For example, the control wavelength can be within a specific color band (e.g., "blue" band, "green" band, "red" band, "infrared" band, etc.) in order to closely match the optical propagation characteristics of the Rx spectral band. In such an example, if the control wavelength is within the red band, then the Rx optical red band exhibits the same functional performance as the associated red control wavelength. The band can be decomposed into one or more sub-bands to improve this correlation. The control wavelength can be spectrally multiplexed / demultiplexed around the Rx optical band, time-multiplexed by a higher peak power, and / or coherently mixed with a local oscillator by a PD to provide a higher signal contrast to the Rx light. This can enable one or more processors 218 of the imaging device 201 to record or capture an image based on the processed light. The recorded image can be stored on the memory 220 of the imaging device 201.

[0042]

[0047] Additionally or alternatively, feedback on the Rx light using one or more control wavelengths can be generated based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscope accelerometer, inertial measurement unit (IMU), etc.). In some cases, the feedback based on one or more sensor measurements can be updated during each time step (e.g., the time corresponding to the reception of one or more measurements). In this regard, one or more control wavelengths can process the Rx light to cancel or compensate for noise or motion detected by one or more sensors 209. In some cases, the feedback on the Rx light using one or more control wavelengths can be updated based on one or more sensor measurements during each time step (e.g., the time corresponding to the reception of one or more measurements).

[0043]

[0048] Additionally or alternatively, one or more control wavelengths may be used to view Rx light within a portion of the field of view (FOV) on the imaging device. In this regard, the one or more control wavelengths may process the Rx light such that only Rx light within a particular portion of the FOV is routed through the imaging device and captured as an image. Thus, a portion of the entire FOV or region of interest (FOR) may be captured without physically moving the imaging device 201.

[0044]

[0049] Additionally or alternatively, one or more control wavelengths may be used to set the image plane. The image plane may be at a particular distance from the imaging device. In this regard, feedback on the Rx light using one or more control wavelengths may process the Rx light such that the light at a particular image plane is in focus on the imaging device. A series of these images at various planes may be saved in rapid temporal succession to form an image hypercube from which an image plane of interest may be selected or a three-dimensional image may be rendered.

[0045]

[0050] Additionally or alternatively, one or more control wavelengths may be used as feedback for a particular intensity or may be generated based on a target pattern. The target pattern may include an exemplary image having particular intensity measurements that are compared to or correlated with the Rx light. In this regard, the one or more control wavelengths may process the Rx light such that glints or high intensity values cannot persist during its processing.

[0046]

[0051] In addition, one or more control wavelengths and functionalities enabled thereby can be determined based on one or more inputs from a user. The one or more inputs can be received at a user interface (not shown). In this regard, the user can select a mode or feature that determines which wavelengths are controlled and how the corresponding processing is performed on the incident light. In some cases, multiple control wavelengths can be repeatedly utilized to process Rx light according to an input from the user. For example, when the user selects a full-color image or an image that includes all colors of the visible light spectrum, each control wavelength corresponding to each band of the visible light can be used to repeatedly process the Rx light. In this regard, one or more processors 218 of the imaging device 201 can record or collect one to N images. Each of the one to N images is within a different visible light band. The recorded images can be stored in the memory 220 of the imaging device 201. The one or more processors 218 of the imaging device can be further configured to generate a full-color image using the one to N stored images.

[0047]

[0052] In some cases, the control wavelength phase or the wavelength itself associated with the Rx band can be dithered. In this regard, the control wavelength can be dithered according to a function of an orthogonal set (e.g., an orthogonal basis function). The dithering can be time-division dithering, frequency-division dithering, or some combination thereof. The dither can be applied to various subsets of the plurality of phase and amplitude modulators 212 and 216. Each subset of the phase and amplitude modulators can correspond to a function among the functions of an orthonormal set. In some implementations, the function of the orthonormal set can be a Walsh function.

[0048]

[0053] Regarding time-division dithering, each dither can be applied sequentially. In this regard, the dither can be applied to different subsets of the phase and amplitude modulators of each successive signal. The dither frequency of each dither can be selected from a predetermined set of frequencies. The frequencies of each dither can be the same or different.

[0049]

[0054] Regarding frequency division dithering, each dither can be applied simultaneously. In this regard, the dither can be applied to different subsets of the phase and amplitude modulators of each signal. The dither frequency of each signal can be selected from a predetermined set of frequencies. In some cases, each of the frequencies in the predetermined set of frequencies can be unique. In such cases, the frequencies in the predetermined set of frequencies can be selected so as not to interfere with each other. Additionally or alternatively, in some cases, each of the plurality of predetermined frequencies may not be unique. In such cases, among the plurality of perturbations, the perturbations that utilize the same frequency or frequencies that can interfere can be selected so that the frequencies do not interfere. For example, if two perturbations utilize the same frequency, one perturbation can be utilized via a sine function and the other can be utilized via a cosine function, and one of these functions can be shifted by π / 2 so that the perturbations are orthogonal and / or the phases of the perturbations are shifted.

[0050]

[0055] Figure 3 depicts the characteristics of an OPA architecture represented as an exemplary OPA architecture 300. The OPA architecture 300 can represent the components of the OPA receivers 103, 203, the FPA OPAs 104, 204, and their associated components. The OPA architecture 300 includes a representation of a microlens array 310, a plurality of emitters or optical antennas 320, and a plurality of phase and amplitude modulators 330. For clarity and ease of understanding, additional waveguides and other features are not depicted. Arrows 340 and 342 represent the general directions in which Tx light (e.g., control wavelength, light to the FPA, etc.) and Rx light (e.g., light from the environment, reflected control wavelength, etc.) pass through or move through the OPA architecture 300.

[0051]

[0056] The microlens array 310 may include a plurality of convex microlenses 311 to 315 that focus Rx light onto each of a plurality of emitters positioned at the foci of the microlens array 310. In this regard, the dashed line 350 represents the focal plane of the microlenses 311 to 315 of the array 310. The microlens array 310 may be arranged in a grid pattern having a consistent pitch or distance between adjacent lenses. In other examples, the microlens array 310 may be in various arrangements having various numbers of rows and columns of various lenses, various shapes, and / or various pitches (consistent or inconsistent).

[0052]

[0057] Each microlens of the microlens array may have a diameter and / or height of several micrometers, several tens of micrometers, or several hundreds of micrometers. In addition, each microlens of the microlens array may be manufactured by directly molding, printing, or etching the lens within the wafer of the OPA architecture 300. Alternatively, the microlens array 310 may be formed as a separately fabricated microlens array. In this example, the microlens array 310 may be a rectangular or square plate of glass or silica having a length and width of several millimeters (for example, around 10 mm) and a thickness of around 0.2 mm. Integrating the microlens array into the OPA architecture 300 may enable reduction of the grid emitter size and increase of the space between emitters. In this way, two-dimensional waveguide routing within the OPA architecture may better conform to a single-layer optical phased array. In other cases, rather than a physical microlens array, the function of the microlens array may be repeated using an array of diffractive optical elements (DOEs).

[0053]

[0058] Each microlens of the microlens array can be associated with each emitter of the plurality of emitters 320. For example, each microlens can have an emitter where the Tx signal is received and the Rx signal is focused. As an example, microlens 311 is associated with emitter 321. Similarly, each of the microlenses 312 - 315 also has its respective emitter 322 - 325. In this regard, for a given pitch (i.e., the edge length of the microlens), the microlens focal length can be optimized for the best transmission and reception coupling to the underlying emitter. Thus, this arrangement can increase the effective fill factor of the Rx light at each emitter while also expanding the Tx light received by the microlens from each emitter before the Tx light leaves the OPA architecture 300.

[0054]

[0059] The plurality of emitters 320 can be configured to convert light emission from the waveguide to free space and vice versa. The emitters can also generate specific phase and intensity profiles to further increase the effective fill factor of the Rx light and improve the wavefront of the Tx light. The phase and intensity profiles can be determined using inverse design or other techniques to account for how the transmitted light changes as it propagates to and through the microlens array. The phase profile can be different from the flat profile of a conventional grating emitter, and the intensity profile can be different from the Gaussian intensity profile of a conventional grating emitter. However, in some implementations, the emitter can be a Gaussian field profile grating emitter.

[0055]

[0060] The phase and amplitude modulator 330 may enable the sensing and measurement of Rx light, the modification of Tx light, and the focusing of input light into a single waveguide or fiber. Each emitter may be associated with a phase and amplitude modulator. As shown in FIG. 3, each emitter may be connected to its respective phase and amplitude modulator. As an example, emitter 320 may be associated with phase and amplitude modulator 330. The Rx light received by phase and amplitude modulators 331-335 may be provided to receiver components including sensors, and the Tx light from phase and amplitude modulators 331-335 may be provided to each of the plurality of emitters 320. The architecture of the plurality of phase and amplitude modulators 330 may include at least one layer of phase and amplitude modulators having at least one phase and amplitude modulator connected to an emitter among the plurality of emitters 320. In some examples, the phase and amplitude modulator architecture may include multiple layers of phase and amplitude modulators, and the phase and amplitude modulators in the first layer may be connected in series with one or more phase and amplitude modulators in the second layer.

[0056]

[0061] FIG. 4 shows an exemplary optical control architecture (OCA) 400 that may be included within OPA receivers 103, 203, FPA OPA 204, or both. OCA 400 includes a laser source 406, a plurality of phase and amplitude modulator layers 410, a plurality of PDs 412, one or more waveguide tap couplers 414, one or more multiplexers or demultiplexers 416, a plurality of reflectors, partial reflectors, or retroreflective photonic structures 418, a plurality of emitters 420, a photonic circulator 422, PD 424, and one or more processors 430. In some cases, one or more components of OCA 400 may be formed on a PIC.

[0057]

[0062] The laser source 406 may be used to generate one or more control wavelengths based on instructions from one or more processors 430 as discussed above. The one or more control wavelengths may propagate along the waveguides of OCA 400 as indicated by arrow 408a.

[0058]

[0063] One or more phase and amplitude modulator layers 410 include an optical control (OC) phase and amplitude modulator layer 410b. The OC phase and amplitude modulator layer 410b includes a plurality of PDs 412. As discussed above, the plurality of PDs 412 can be coupled to each phase and amplitude modulator within the phase and amplitude modulator layer. FIG. 4 shows that individual PDs of the plurality of PDs 412 can be coupled to two or more phase and amplitude modulators of the layer via a waveguide tap coupler such as waveguide tap coupler 414. The plurality of PDs 412 can be configured to measure one or more values such as the intensity, power, and / or relative phase of one or more control wavelengths. The measured values can be used for the control and / or analysis of the Rx light (e.g., as feedback). The measured values can be used for the control and / or analysis of components of the OCA 400.

[0059]

[0064] In some cases, various PDs can be configured to measure values in various waveguides of the phase and amplitude modulator layer to determine the relative phase. For example, one arrangement can include one PD that measures the intensity / power of waveguide A, one PD that measures the intensity / power of an adjacent waveguide B, and one PD that measures the relative phase between waveguide A and waveguide B by mixing two signals on one PD. Such an arrangement can provide a basic building block of a PD measurement system for a plurality of PDs. In addition, the same or a similar arrangement can be repeated for adjacent waveguides (e.g., B and C, C and D, etc.). The measured values measured by three exemplary PDs of waveguides A - B can be expressed as A 2 , B 2 and A 2 + B 2 + 2ABcosΦ AB respectively. The three measured values can be used to determine Φ AB, that is, it can enable the determination of the relative phase of light or signals propagating through waveguides A and B. Absolute phase measurement is also possible by using a part of the control wavelength as a local oscillator in order to interfere coherently with the reverse reflector control wavelength propagating back and forth in the PIC with multiple PDs. This may require additional multiple phase modulators in the local oscillator path with associated feedback electronics to phase-lock the two together.

[0060]

[0065] Figure 4 additionally shows one or more phase and amplitude modulator layers 410, such as phase and amplitude modulator layer 410b. The phase and amplitude modulator layer 410b includes one or more multiplexers or demultiplexers 416 operably connected thereto. In some cases, one or more multiplexers or demultiplexers 416 may route received light or received signals to one or more of the multiple PDs 412 for detection, electronic amplification and buffering and / or analog-to-digital conversion. The received light or received signal (e.g., from the environment of the imaging device, from another OPA of the imaging device) is represented by arrow 432. In this regard, the received light 432b can be received by the multiple emitters 420 and propagated to one or more multiplexers or demultiplexers 416 via the OCA architecture as shown by arrow 432b.

[0061]

[0066] The multiple reflectors 418 can be configured to reflect light or signals transmitted from multiple emitters 420, such as one or more control wavelengths represented by arrow 408b, and return them to the multiple emitters 420. One or more of the reflected and returned control wavelengths can propagate through the OCA 400 and be directed to the PD 424 via the circulator 422. The measured value of the signal measured by the PD 424 can be processed by one or more processors 430. This measured value can be used for the control and / or analysis of the components of the OCA 400.

[0062]

[0067] One or more of the processors 430 shown are operatively connected to the plurality of phase and amplitude modulator layers 410, the plurality of PDs 412 and 424. In this regard, one or more of the processors may be configured to drive (e.g., modify the phase and / or amplitude) at least one layer of the phase and amplitude modulators. Additionally, one or more of the processors may be configured to utilize the values measured by the PDs to drive at least one layer of the modulators. In some instances, one or more multiplexers or demultiplexers may route signals to the PDs for detection, electronic amplification and buffering, and analog-to-digital conversion.

[0063]

[0068] For example, the above measurements from the plurality of PDs 412 and 424 may be used in the control of (e.g., to drive) the plurality of phase and amplitude modulator layers 410. In some instances, one or more of the processors 430 may use these measurements to compensate for the phase errors determined (e.g., in the OC phase and amplitude modulator layer 410a) and to drive the plurality of phase and amplitude modulator layers 410 (e.g., layer 410b) to provide a particular phase to the plurality of emitters 420.

[0064]

[0069] The exemplary imaging devices and exemplary architectures discussed above may be utilized as part of one or more user devices. In this regard, the exemplary imaging devices and exemplary architectures may be operatively connected within and / or incorporated into one or more user devices such as dedicated camera devices, mobile phones, tablets, laptops, desktop computers, or other computing devices capable of communicating information with an imaging device. FIG. 5 shows an exemplary user device 500 including imaging devices such as the imaging devices 101 and 201 discussed above. The imaging devices 101 and 201 may include architectures as described with reference to FIGS. 3 and 4.

[0065]

[0070] The user device 500 may include one or more processors, memory, data, and instructions. The memory stores information accessible by one or more processors, including instructions and data that may be executed or otherwise used by the processor. The memory can be any type of memory that can store information accessible by a processor, including a computing device-readable medium. The memory can be non-transitory media such as a hard drive, memory card, optical disk, solid state, etc. Some systems may include various combinations of the foregoing, whereby various portions of instructions and data are stored on various types of media. The instructions can be any set of instructions directly executable by a processor (such as machine code) or indirectly executable (such as a script). For example, the instructions can be stored as computing device code on a computing device-readable medium. In this regard, the terms "instructions", "modules", and "programs" may be used interchangeably herein. The instructions can be stored in object code form for direct processing by a processor or in any other computing device language including scripts or sets of independent source code modules that are interpreted as needed or pre-compiled.

[0066]

[0071] The processor can be any conventional processor, such as a commercially available CPU. Alternatively, each processor can be a dedicated device such as an ASIC, a graphics processing unit (GPU), a tensor processing unit (TPU), or other hardware-based processor. FIG. 5 functionally shows the processor, memory, and other elements of a given computing device as being within the same block, although such a device can actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical enclosure. Similarly, the memory can be a hard drive or other storage medium located within a housing different from that of the processor, such as within a cloud computing system of server 802. Accordingly, references to the processor or memory should be understood to include references to a collection of processors or memories that may or may not operate in parallel.

[0067]

[0072] The user device 500 can include all of the components typically used in connection with a computing device, such as the above-described processor and memory, as well as a user interface subsystem that receives input from a user and presents information (e.g., text, images, and / or other graphic elements via acoustic and / or tactile feedback) to the user. The user interface subsystem can include one or more user inputs (e.g., at least one front (user)-facing camera, a mouse, a keyboard, a touch screen, and / or a microphone) and one or more display devices operable to display information (e.g., text, images, and / or other graphic elements). Other output devices, such as speakers, can also provide information to the user.

[0068]

[0073] Alternatively, imaging devices such as imaging devices 101 and 201 can be stand-alone devices that can communicate with the processor of the user device via a wireless or wired connection. Such user devices can communicate with the imaging device via one or more networks. The one or more networks can include various configurations and protocols including short-range communication protocols, such as Bluetooth (trademark), Bluetooth LE (trademark), the Internet, the World Wide Web, intranets, virtual private networks, wide area networks, local area networks, private networks using the proprietary communication protocols of one or more companies, Ethernet, WiFi, and HTTP, as well as various combinations of the foregoing. Such communication can be facilitated by any device capable of transmitting data between other computing devices such as modems and wireless interfaces.

[0069] Exemplary method

[0074] The imaging devices 101 and 201 discussed above can be used in a method of capturing one or more images using the imaging device. FIG. 6 shows an exemplary method 600 of capturing one or more images using an imaging device. At block 610, the method includes generating one or more control wavelengths by one or more processors of the imaging device based on one or more user inputs. For example, one or more processors 218 and 430 (e.g., CMOS processors) of imaging device 201 can be configured to generate one or more control wavelengths 408 via one or more light sources 219 and 406 (e.g., light emitting diodes (LEDs), PIC integrated lasers, external cavity lasers, etc.). As discussed above, the one or more control wavelengths can be within a particular wavelength band (e.g., color band), can be based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscope accelerometers, IMUs, etc.), can correspond to a portion of the FOV on imaging devices 101 and 201, can be within a particular focal plane in relation to imaging devices 101 and 201, and / or can correspond to a particular intensity.

[0070]

[0075] In some cases, one or more control wavelengths can be determined based on one or more inputs from a user. The one or more inputs can be received at a user interface of a user computing device (e.g., user device 500) such as a dedicated camera device, a mobile phone, a tablet, a laptop, a desktop computer, or another computing device capable of communicating information with an imaging device. In this regard, imaging devices 101 and 201 can be incorporated into any of the aforementioned devices. Alternatively, imaging devices 101 and 201 can be stand-alone devices capable of communicating with a processor of the aforementioned devices via a wireless or wired connection. For example, a user can select a mode or feature that determines which control wavelengths and corresponding processing are performed on incident light. For example, a user can select infrared image capture. In such an example, one or more control wavelengths can be within the infrared band.

[0071]

[0076] At block 620, the method further includes receiving light from the environment of the imaging device at an optical phased array (OPA). For example, the OPA receivers 103 and 203 of imaging devices 101 and 201 can be configured to receive light from the environment of imaging devices 101 and 201. Specifically, a plurality of emitters or optical antennas 215, 320, 420 of the OPA can be configured to receive light from the environment of imaging devices 101 and 201.

[0072]

[0077] At block 630, the method further includes driving a plurality of phase and amplitude modulators of an optical phased array (OPA) by applying one or more control wavelengths to process the received light by one or more processors of the imaging device. For example, one or more processors 218 and 430 of the imaging device can be configured to apply one or more control wavelengths to Rx light by driving a plurality of phase and amplitude modulators 212 and 330.

[0073]

[0078] As discussed above, the control wavelength can be within a specific color band (e.g., blue band, green band, red band, infrared band, etc.) in order to closely match the optical propagation characteristics of the Rx spectral band. In such an example, when the control wavelength is within the red band, the Rx light can be processed using the control wavelength within the red band such that the Rx light red band exhibits the same functional performance as the associated red control wavelength. These bands can be decomposed into even more sub-bands to improve this correlation. The control wavelength can be spectrally multiplexed / demultiplexed around the Rx light band to provide a higher signal contrast for the Rx light, can be multiplexed in time with a higher peak power, and / or can be coherently mixed with a local oscillator at the PD.

[0074]

[0079] Additionally or alternatively, the feedback on the Rx light using one or more control wavelengths can be based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscope accelerometer, inertial measurement unit (IMU), etc.). In some cases, the feedback based on one or more sensor measurements can be updated during each time step (e.g., the time corresponding to the reception of one or more measurements). In this regard, the one or more control wavelengths can process the Rx light to cancel or compensate for noise or motion detected by the one or more sensors 209. In some cases, the feedback on the Rx light using one or more control wavelengths can be updated based on one or more sensor measurements during each time step (e.g., the time corresponding to the reception of one or more measurements).

[0075]

[0080] Additionally or alternatively, feedback on the Rx light using one or more control wavelengths can be generated based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscope accelerometers, inertial measurement units (IMUs), etc.). In some cases, the feedback based on one or more sensor measurements can be updated during each time step (e.g., the time corresponding to the receipt of one or more measurements). In this regard, the one or more control wavelengths can process the Rx light to cancel or compensate for noise or motion detected by the one or more sensors 209. In some cases, the feedback on the Rx light using one or more control wavelengths can be updated based on one or more sensor measurements during each time step (e.g., the time corresponding to the receipt of one or more measurements).

[0076]

[0081] Additionally or alternatively, one or more control wavelengths can be used to view Rx light within a portion of the FOV on the imaging device. In this regard, the one or more control wavelengths can process the Rx light such that only the Rx light within a particular portion of the FOV is routed through the imaging device and captured as an image. Thus, all or a portion of the FOV or FOR can be captured without the need to physically move the imaging device 201.

[0077]

[0082] Additionally or alternatively, one or more control wavelengths can be used to set the image plane. The image plane can be at a particular distance from the imaging device. In this regard, the feedback on the Rx light using one or more control wavelengths can process the Rx light such that the light at the particular image plane is in focus on the imaging device. A series of these images at various planes can be saved in rapid temporal succession to form an image hypercube from which an image plane of interest can be selected or a three-dimensional image can be rendered.

[0078]

[0083] Additionally or alternatively, one or more control wavelengths can be used as feedback for a particular intensity or can be generated based on a target pattern. The target pattern can include an exemplary image having specific intensity measurements that are compared to or correlated with the Rx light. In this regard, one or more control wavelengths can process the Rx light such that glints or high-intensity values cannot persist during the process.

[0079]

[0084] As discussed above, one or more control wavelengths and functionalities thereby enabled can be determined based on one or more inputs from a user. The one or more inputs can be received at a user interface. In this regard, the user can select a mode or feature that determines which control wavelengths are generated and how the received light is processed as an extension.

[0080]

[0085] In some cases, driving multiple phases and amplitudes can be based on one or more measurements from one or more PDs. In this regard, the multiple phase and amplitude modulators 212 and 330 of the OPA receivers 103 and 203 can be driven based on feedback related to measurements associated with one or more propagating control wavelengths 408. In some cases, the feedback can additionally relate to measurements associated with the received light 432. The above measurements from the multiple PDs 213, 412, and PD424 can be used in the control of the multiple phase and amplitude modulators 212, 330, and their layer 410 (e.g., to drive them). In some cases, one or more processors 218 and 430 can use the measurements to compensate for the determined phase error and drive the multiple phase and amplitude modulator layer 410.

[0081]

[0086] In block 640, the method further includes capturing an image by one or more processors of the imaging device based on the processed received light. For example, one or more processors 218 and 430 may use FPAs 105 and 205 of imaging devices 101 and 201 to capture or record an image of the processed light. In some cases, the captured image may be stored in memories 220 of imaging devices 101 and 201.

[0082]

[0087] In some cases, the method may further include transmitting one or more control wavelengths by the OPA or its emitter, reflecting one or more control wavelengths back by the OPA or one or more of its reflectors, and receiving one or more control wavelengths by the OPA or its emitter. In this regard, a plurality of emitters or antennas 211, 320, 420 of OPA receivers 103 and 203 may be configured to transmit one or more control wavelengths 408. One or more control wavelengths 408b may be reflected back to the plurality of emitters 211, 320, 420 via one or more reflectors 210 and 418. In this regard, the reflected control wavelengths may be used for phase and wavefront control of the Rx light as discussed above.

[0083]

[0088] In some cases, the method may further include transmitting the processed received light to a second OPA by the OPA, and transmitting the processed received light to the FPA of the imaging device by the second OPA. In this regard, OPA receivers 103 and 203 may transmit the processed received light to FPA OPAs 104 and 204. The processed received light may be received by emitters 215, 320, 420 of FPA OPAs 104 and 204. In addition, the processed received light may be further transmitted by FPA OPAs 104 and 204 to FPAs 105 and 205 by their emitters 215, 320, 420. In this regard, FPAs 105 and 205 may receive light from FPA OPAs 104, 204 and record an image based on the received light. The recorded image may be stored on a memory (e.g., memory 220) of imaging device 201.

[0084]

[0089] In some cases, the amplitude modulator arrays of FPA105 and 205 can be used to adjust the amount of light directed at the elements of FPA205 on a per-element basis. This adjustment can also be utilized, for example, for brightness adjustment, speckle removal, aperture adjustment, and other amplitude-dependent operations.

[0085]

[0090] In some cases, the method can further include transmitting one or more control wavelengths by the second OPA or its emitter, reflecting one or more control wavelengths back by the second OPA or one or more of its reflectors, and receiving one or more control wavelengths by the second OPA or its emitter. In this regard, the multiple emitters or antennas 215, 320, 420 of FPA OPAs 104 and 204 can be configured to transmit one or more control wavelengths 408. One or more control wavelengths 408b can be reflected back to the multiple emitters 215, 320, 420 via one or more reflectors 214 and 418. In this regard, the reflected control wavelengths can be used for phase and wavefront control of the processed Rx light as discussed above.

[0086]

[0091] In some cases, the method can further include driving the multiple phase and amplitude modulators of the second OPA by one or more processors of the imaging device to apply one or more control wavelengths to further process the processed received light. In this regard, one or more processors 218 and 430 of the imaging device can be configured to apply one or more control wavelengths to the Rx light at FPA OPAs 104, 204 by driving the multiple phase and amplitude modulators 216 and 330.

[0087]

[0092] As described above, one or more control wavelengths can be determined based on one or more inputs from a user. The one or more inputs can be received at a user interface. In this regard, the user can select which control wavelengths are generated and the mode or feature for determining the processing of the received light as an extension.

[0088]

[0093] In some cases, driving a plurality of phases and amplitudes can be based on one or more measurements from one or more PDs. In this regard, the plurality of phase and amplitude modulators 216 and 330 of the FPA OPAs 104, 204 can be driven based on feedback related to measurements associated with one or more propagating control wavelengths 408. In some cases, the feedback can additionally relate to measurements associated with the received light 432. The above measurements from the plurality of PDs 217, 412 and PD 424 can be used in the control of the plurality of phase and amplitude modulators 216, 330 and their layers 410 (e.g., to drive them). In some cases, one or more processors 218 and 430 can use the measurements to compensate for the determined phase error and drive the plurality of phase and amplitude modulator layers 410.

[0089]

[0094] In some cases, the OPA receivers 103, 203 and the FPA OPAs 104, 204 may be configured to perform various forms of Rx light processing as discussed above in conjunction with one or more processors 218 and 430 of the imaging devices 101 and 201. In this regard, one or more processors 218 and 430 of the imaging devices 101 and 201 may be configured to generate one or more control wavelengths via one or more light sources 219 and 406. In one example, one or more control wavelengths may be transmitted based on one or more sensor measurements of the sensor 209, may correspond to a portion of the FOV of the imaging device, and / or may be within a particular focal plane, and are reflected back and used by the OPA receivers 103 and 203 for phase and wavefront control of the Rx light. In such an example, one or more control wavelengths that may be within a particular color band and / or may correspond to a particular intensity or may be generated based on a target pattern are transmitted, reflected back, and used by the FPA OPAs 104 and 204 to process the Rx light.

[0090]

[0095] In some cases, the one or more control wavelengths may be multiple control wavelengths such as a first control wavelength and a second control wavelength. In such cases, the method steps discussed above may be repeated based on the number of control wavelengths. For example, if a user selects a full-color image or an image that includes all colors of the visible light spectrum, the control wavelengths corresponding to each band of the visible light may be used to repeatedly process the Rx light. In this regard, one or more processors of the imaging device may record and collect 1 to N images in consecutive time steps. Each of the 1 to N images is within a different visible light band. The recorded images may be stored in the memory of the imaging device. In such cases, the method may further include generating a composite image by one or more processors of the imaging device. In this regard, one or more processors of the imaging device may be further configured to generate a full-color image using the 1 to N stored images.

[0091]

[0096] The features and methodologies described herein can provide a scalable imaging device that can perform complex image processing using one or more injected or applied control wavelengths and wavefronts to maintain an appropriate relationship between the received optical components and the image, resulting in improved image capture capabilities. In this regard, this imaging device can implement real-time color processing and intensity manipulation, image stabilization to compensate for external vibrations and disturbances, capture panoramic images without moving the imaging device, and correct the focal plane of the image in real time (focal plane adjustment).

[0092]

[0097] Unless otherwise stated, the foregoing alternatives are not mutually exclusive and can be implemented in various combinations to achieve their respective advantages. These and other variations and combinations of the features discussed above can be utilized without departing from the subject matter defined by the claims. Therefore, the foregoing description of the embodiments should be construed as illustrative rather than limiting of the subject matter defined by the claims. In addition, the provision of some examples described herein and clauses expressed as "such as," "including," etc. should not be construed as limiting the subject matter of the claims to specific examples. Rather, the examples are intended to show only some of the many possible embodiments. Further, the same reference numbers in different drawings can identify the same or similar elements.

Description of Reference Numerals

[0093] 101 Imaging device 102 Photonics integrated circuit 103 OPA receiver 104 Focal plane array optical phased array 105 Focal plane array 106 Microlens array 107 Microlens array 108 Control unit 201 Imaging device 202 Photonics integrated circuit 203 Optical phased array receiver 204 Focus plane array optical phased array 205 Focus plane array 206 Microlens array 207 Microlens array 208 Control unit 209 Sensor 210 Reflector 211 Emitter 212 Phase and amplitude modulator 213 Photodiode 214 Reflector 215 Emitter 216 Phase and amplitude modulator 217 Photodiode 218 Processor 219 Light source 220 Memory 300 Optical phased array architecture 310 - 315 Microlens array 320 - 325 Emitter 330 - 335 Phase and amplitude modulator 340 Arrow 342 Arrow 350 Dashed line 400 Optical control architecture 406 Laser source 408 Control wavelength 408a Control wavelength 408b Control wavelength 410 Phase and amplitude modulator layer 410a Phase and amplitude modulator layer 410b Phase and amplitude modulator layer 412 Photodiode 414 Waveguide tap coupler 416 Demultiplexer 418 Reflector 420 Emitter 422 Photonics circulator 424 Photodiode 430 Processor 432 Received light 432a Received light Received light of 432b 500 User device 610 Block 620 Block 630 Block 640 Block 802 Server

Claims

1. 1. A method for capturing one or more images using an imaging device, comprising: generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs; receiving light from an environment of the imaging device at an optical phased array (OPA); applying the one or more control wavelengths to drive a plurality of phase and amplitude modulators of the OPA to process the received light, by the one or more processors of the imaging device; capturing, by the one or more processors of the imaging device, an image based on the processed received light; The method includes:

2. generating, by the one or more processors of the imaging device, the one or more control wavelengths based on the one or more user inputs includes generating a first control wavelength and a second control wavelength; receiving light from the environment of the imaging device in the OPA includes receiving light at a first time step and a second time step; Driving the multiple phase and amplitude modulators of the OPA to apply the one or more control wavelengths to process the received light, by the one or more processors of the imaging device, includes driving the multiple phase and amplitude modulators at the first time step to apply the first control wavelength to process the light received at the first time step, and driving the multiple phase and amplitude modulators at the second time step to apply the second control wavelength to process the light received at the second time step; and 2. The method of claim 1, wherein capturing an image based on the processed received light by the one or more processors of the imaging device includes capturing a first image at the first time step based on the processed light received at the first time step, and capturing a second image at the second time step based on the processed light received at the second time step.

3. The method of claim 2 , further comprising generating, by the one or more processors of the imaging device, a composite image of the first image and the second image.

4. transmitting, by the OPA of the imaging device, the one or more control wavelengths; reflecting the one or more control wavelengths back through the OPA of the imaging device; receiving, by an OPA of the imaging device, the one or more control wavelengths; The method of claim 1 further comprising:

5. The method of claim 1 , wherein the one or more user inputs are received from a user interface.

6. applying the one or more control wavelengths to drive the plurality of phase and amplitude modulators of the OPA to process the received light, by the one or more processors of the imaging device, measuring, with one or more photodiodes, one or more measurements associated with the one or more control wavelengths; driving the plurality of phase and amplitude modulators based on the one or more measurements associated with the one or more control wavelengths; The method of claim 1 , comprising:

7. applying the one or more control wavelengths to drive the plurality of phase and amplitude modulators of the OPA to process the received light, by the one or more processors of the imaging device, measuring, with one or more photodiodes, one or more measurements associated with the received light; driving the plurality of phase and amplitude modulators based on the one or more measurements associated with the received light; and The method of claim 1 , comprising:

8. applying the one or more control wavelengths to drive the plurality of phase and amplitude modulators of the OPA to process the received light, by the one or more processors of the imaging device, measuring, with one or more photodiodes, one or more measurements associated with the one or more control wavelengths and the received light; driving the plurality of phase and amplitude modulators based on the one or more control wavelengths and the one or more measurements associated with the received light; The method of claim 1 , comprising:

9. The method of claim 8 , wherein the one or more measurements are at least one of: i) intensity, ii) power, and iii) relative phase.

10. transmitting, by the OPA, the processed received light to a second OPA; transmitting, by the second OPA, the processed received light to a focal plane array (FPA) of the imaging device; The method of claim 1 further comprising:

11. The method of claim 10 , further comprising adjusting an amount of light received at one or more elements of the FPA with an amplitude modulator array of the FPA.

12. 11. The method of claim 10, further comprising applying, by the one or more processors of the imaging device, the one or more control wavelengths to drive a plurality of phase and amplitude modulators of the second OPA to further process the processed received light.

13. The method of claim 10 , wherein the one or more control wavelengths is a plurality of control wavelengths.

14. applying the plurality of control wavelengths to drive the plurality of phase and amplitude modulators of the OPA to process the received light, by the one or more processors of the imaging device, includes applying a first control wavelength of the plurality of control wavelengths; 14. The method of claim 13, wherein applying the one or more control wavelengths by the one or more processors of the imaging device to drive a plurality of phase and amplitude modulators of the second OPA to further process the processed received light includes applying a second control wavelength of the plurality of control wavelengths, the second control wavelength being different from the first control wavelength.

15. 1. An imaging device configured to capture one or more images, comprising:

1. An optical phased array (OPA) receiver, comprising: a plurality of emitters configured to transmit and receive light; a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device; and one or more photodiodes configured to measure one or more values ​​associated with light propagating through the imaging device; An optical phased array (OPA) receiver including: A focal plane array (FPA) OPA, comprising: a plurality of emitters configured to transmit and receive light; a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device; and one or more photodiodes configured to measure one or more values ​​associated with light propagating through the imaging device; A focal plane array (FPA) OPA including A control unit, one or more processors; and One or more light sources configured to generate light a control unit including: an FPA configured to receive light from the FPA OPA and record an image based on the received light; wherein the OPA receiver, the FPA OPA and the control unit are disposed on a Photonics Integrated Circuit (PIC).

16. 16. The imaging device of claim 15, wherein the OPA receiver further comprises one or more reflectors associated with an emitter of the plurality of emitters and configured to reflect one or more control wavelengths back to the emitter of the plurality of emitters.

17. The imaging device of claim 15 , further comprising: i) the OPA receiver; and ii) one or more microlens arrays associated with at least one of the FPA OPAs.

18. 16. The imaging device of claim 15, further comprising one or more sensors configured to collect one or more sensor measurements, the one or more sensors comprising: i) at least one of a gyroscope, an accelerometer, and an inertial measurement unit (IMU).

19. The imaging device of claim 15 arranged in a stacked configuration.

20. The one or more processors: generating one or more control wavelengths based on one or more user inputs; i) driving the multiple phase and amplitude modulators of the OPA receiver to apply the one or more control wavelengths to process received light, and ii) driving at least one of the multiple phase and amplitude modulators of the FPA OPA to apply the one or more control wavelengths to process received light; capturing an image based on the processed received light; The imaging device of claim 15 , configured to:

Images