Two-dimensional photonics integrated circuit optical phased array for imaging systems

Abstract

To provide a two-dimensional photonic integrated circuit optical phased array for imaging systems.SOLUTION: Aspects of the disclosure provide a device and a method for capturing one or more images using a photonic integrated circuit (PIC) optical phased array (OPA) based imaging system. The method may include processing the environmentally or artificially lit received image using one or more injected control wavelengths or wavefronts to enable electronic control of functions such as real time color filtering, glint removal or other intensity image manipulation, image stabilization, collection of multiple images for panoramic composites, focal plane adjustment and / or other image optimization manipulations based on user input. These system level functions may be accomplished with a device utilizing a plurality of optical nano-antenna elements, a plurality of phase and amplitude modulators, a plurality of measurement photodiodes and a plurality of other associated photonic constructs along with one or more CMOS processors to process the received image.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of the filing date of U.S. Provisional Patent Application No. 63 / 609,997, filed December 14, 2023, which is incorporated by reference in its entirety into this specification. [Background technology]

[0002] background

[0002] Imaging systems such as mobile phones, tablets, laptops, and other computers, cameras, etc. may utilize opto-mechanical systems for image capture and processing. Such opto-mechanical systems allow for post-processing of images, but may also require physical movement of the imaging system to capture multiple images, as well as stitching for post-processing panoramic composition after image capture.

[0003]

[0003] In general, current imaging technologies may be limited in capabilities due to the opto-mechanical systems used, which affect functionality and / or key technical performance indicators. Examples may include limited or no ability to perform real-time color filtering and intensity manipulation, limited bandwidth of image stabilization to compensate for external vibrations and disturbances, the need to physically move the camera to capture multiple images with post-stitching for panoramic composition, and limited ability to collect and correct the focal plane of images in real time (focal plane adjustment). Summary of the Invention [Means for solving the problem]

[0004] Quick Overview Aspects of the present disclosure are directed to a method for capturing one or more images using an imaging device, the method including: generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs; receiving light from an environment of the imaging device at an optical phased array (OPA); applying, by the one or more processors of the imaging device, the one or more control wavelengths to drive multiple phase and amplitude modulators of the OPA to process the received light; and capturing, by the one or more processors of the imaging device, an image based on the processed received light.

[0005]

[0005] In one example, generating one or more control wavelengths based on one or more user inputs by one or more processors of the imaging device includes generating a first control wavelength and a second control wavelength; receiving light from an environment of the imaging device in the OPA includes receiving light at a first time step and a second time step; applying one or more control wavelengths to drive multiple phase and amplitude modulators of the OPA to process the received light by one or more processors of the imaging device includes applying a first control wavelength to drive the multiple phase and amplitude modulators at a first time step to process the light received at the first time step and applying a second control wavelength to drive the multiple phase and amplitude modulators at a second time step to process the light received at the second time step; and capturing an image based on the processed received light by one or more processors of the imaging device includes capturing a first image at a first time step based on the processed light received at the first time step and capturing a second image at a second time step based on the processed light received at the second time step. Additionally, the method further includes generating, by one or more processors of the imaging device, a composite image of the first image and the second image.

[0006]

[0006] In another example, the method additionally includes transmitting one or more control wavelengths by the OPA of the imaging device, reflecting the one or more control wavelengths back by the OPA of the imaging device, and receiving the one or more control wavelengths by the OPA of the imaging device.

[0007] In another example, the one or more user inputs are received from a user interface.

[0008]

[0008] In a further example, driving multiple phase and amplitude modulators of an OPA by one or more processors of an imaging device to apply one or more control wavelengths to process received light includes measuring one or more measurements associated with the one or more control wavelengths by one or more photodiodes, and driving the multiple phase and amplitude modulators based on the one or more measurements associated with the one or more control wavelengths.

[0009]

[0009] In another example, applying one or more control wavelengths by one or more processors of the imaging device to drive multiple phase and amplitude modulators of the OPA to process the received light includes measuring one or more measurements associated with the received light by one or more photodiodes, and driving the multiple phase and amplitude modulators based on the one or more measurements associated with the received light.

[0010] In another example, driving, by one or more processors of the imaging device, multiple phase and amplitude modulators of the OPA to apply one or more control wavelengths to process the received light includes measuring, by one or more photodiodes, one or more measurements associated with the one or more control wavelengths and the received light, and driving the multiple phase and amplitude modulators based on the one or more measurements associated with the one or more control wavelengths and the received light. Additionally, the one or more measurements can be at least one of i) intensity, ii) power, and iii) relative phase.

[0011]

[0011] In a further example, the method further includes transmitting, by the OPA, the processed received light to a second OPA, and transmitting, by the second OPA, the processed received light to a focal plane array (FPA) of the imaging device. In addition, the method may further include adjusting, by an amplitude modulator array of the FPA, an amount of light received at one or more elements of the FPA. Additionally or alternatively, the method may further include applying, by one or more processors of the imaging device, one or more control wavelengths to drive multiple phase and amplitude modulators of the second OPA to further process the processed received light. Additionally or alternatively, the one or more control wavelengths may be multiple control wavelengths. Additionally or alternatively, applying the plurality of control wavelengths by one or more processors of the imaging device to drive the plurality of phase and amplitude modulators of the OPA to process the received light may include applying a first control wavelength of the plurality of control wavelengths, and applying the one or more control wavelengths by the one or more processors of the imaging device to drive the plurality of phase and amplitude modulators of the second OPA to further process the processed received light includes applying a second control wavelength of the plurality of control wavelengths, wherein the second control wavelength is different from the first control wavelength.

[0012] Another aspect of the present disclosure is directed to an imaging device configured to capture one or more images. The imaging device includes an optical phased array (OPA) receiver, where the OPA receiver includes a plurality of emitters configured to transmit and receive light, a plurality of phase and amplitude modulators configured to modulate the light propagating through the imaging device, and one or more photodiodes configured to measure one or more values ​​associated with the light propagating through the imaging device. The imaging device further includes a focal plane array (FPA) OPA, where the FPA OPA includes a plurality of emitters configured to transmit and receive light, a plurality of phase and amplitude modulators configured to modulate the light propagating through the imaging device, and one or more photodiodes configured to measure one or more values ​​associated with the light propagating through the imaging device. The imaging device further includes a control unit, where the control unit includes one or more processors and one or more light sources configured to generate light. The imaging device further includes an FPA configured to receive light from the FPA OPA and record an image based on the received light, and the OPA receiver, the FPA OPA and the control unit are arranged on a photonics integrated circuit (PIC).

[0013]

[0013] In one example, the OPA receiver further includes one or more reflectors associated with an emitter among the plurality of emitters and configured to reflect one or more control wavelengths back to that emitter among the plurality of emitters.

[0014] In another example, the imaging device further includes one or more microlens arrays associated with at least one of: i) the OPA receiver; and ii) the FPA OPA.

[0015]

[0015] In another example, the imaging device additionally includes one or more sensors configured to collect one or more sensor measurements, the one or more sensors including i) at least one of a gyroscope, an accelerometer, and an inertial measurement unit (IMU).

[0016] In a further example, the imaging devices are arranged in a stacked configuration. In another example, the one or more processors are configured to generate one or more control wavelengths based on one or more user inputs, and to drive at least one of i) a plurality of phase and amplitude modulators of the OPA receiver to apply the one or more control wavelengths to process the received light, and ii) a plurality of phase and amplitude modulators of the FPA OPA to apply the one or more control wavelengths to process the received light, and capture an image based on the processed received light. [Brief explanation of the drawings]

[0017] BRIEF DESCRIPTION OF THE DRAWINGS [Figure 1A]

[0017] FIG. 1 is an image diagram of an imaging device according to an aspect of the present disclosure. [Figure 1B]

[0017] FIG. 1 is an image diagram of an imaging device according to an aspect of the present disclosure. [Figure 2]

[0018] FIG. 1 is a block diagram of an imaging device according to aspects of the present disclosure. [Figure 3]

[0019] 1 illustrates features of an optical phased array architecture according to aspects of the present disclosure. [Figure 4]

[0020] 1 illustrates features of an optical control architecture according to aspects of the present disclosure. [Figure 5]

[0021] FIG. 1 is a block diagram of a user device according to an aspect of the present disclosure. [Figure 6]

[0022] FIG. 1 is a block diagram according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] Detailed Description overview

[0023] The present technology relates to an imaging device implemented using an optical phased array (OPA). The imaging device may be used for image capture and related image processing. The imaging device may include one or more OPAs and one or more processors. The one or more OPAs and / or the one or more processors may be included in a photonics integrated circuit (PIC). The one or more processors may be one or more complementary metal-oxide semiconductor (CMOS) processors. The imaging device may be configured to process received light based on one or more control wavelengths or wavefronts for phase and wavefront control of the received light, and capture an image based on the processed light.

[0019]

[0024] In general, current imaging technologies may be limited in capabilities due to the opto-mechanical systems used, which affect functionality and / or key technical performance indicators. Examples may include limited or no ability to perform real-time color filtering and intensity manipulation, limited bandwidth of image stabilization to compensate for external vibrations and disturbances, the need to physically move the camera to capture multiple images with post-stitching for panoramic composition, and limited ability to collect and correct the focal plane of images in real time (focal plane adjustment).

[0020]

[0025] To address this, as noted above, imaging devices can be implemented using PICs with OPAs. In this regard, the imaging device can utilize the functional capabilities of the OPA to perform complex image processing and provide improved image capture capabilities. In this regard, imaging devices implemented using PICs and OPAs can be configured to perform real-time color filtering and intensity manipulation, perform image stabilization to compensate for external vibrations and disturbances, and collect and correct the focal plane of an image in real time (focal plane adjustment) without requiring its physical movement to capture multiple images for panoramic composition.

[0021] Exemplary System

[0026] As discussed above, an imaging device may include one or more OPAs and one or more processors (e.g., CMOS processors). FIGS. 1A-1B illustrate an exemplary imaging device 101. FIG. 1A illustrates a side view of the imaging device 101, and FIG. 1B illustrates an overhead view of the imaging device 101. The imaging device 101 includes a first OPA or OPA receiver 103, a second OPA or focal plane array (FPA) OPA 104, an FPA 105, a microlens array 106 associated with the FPA OPA 104, a microlens array 107 associated with the OPA receiver 103, and a block or control unit 108. The block 108 may include one or more processors (e.g., CMOS processors). Additionally, the block 108 may include one or more light sources (e.g., light-emitting diodes (LEDs), PIC-integrated lasers, externally coupled lasers, etc.). The OPA receiver 103, FPA OPA 104, and block 108 of the imaging device 101 are shown as being included on a photonics integrated circuit (PIC) 102. However, in some implementations, one or more of these components may not be included on the PIC 102 and / or may be included on a different PIC separate from the PIC 102.

[0022]

[0027] In some implementations, microlens array 106 can be a lens, a metalens, a microlens array, or any combination thereof. Additionally or alternatively, microlens array 107 can be a lens, a metalens, a microlens array, or any combination thereof.

[0023]

[0028] As shown in the side view of FIG. 1A , the imaging devices may be arranged in a stacked configuration. In this regard, the microlens array 107 associated with the OPA receiver 103 may be disposed on a first surface of the imaging device 101. During operation, the first surface may face the environment of the imaging device, which is the target of the captured image. The microlens array 106 associated with the FPA OPA 104 may be disposed on a second surface of the imaging device 101. The second surface may be opposite the first surface. The FPA 105 may be disposed a distance from the microlens array 106 on the second side of the imaging device 101. This distance may correspond to the focal plane of the imaging device 101. While the OPA receiver 103 and the FPA OPA 104 are shown as circular, one or both of them may be configured to be any geometric shape, including an ellipse, a square, a rectangle, a hexagon, an octagon, etc. Additionally or alternatively, the geometry of OPA receiver 103 and FPA OPA 104 may be modified to change the relative size and / or pixel spacing of the OPA receiver 103 input and FPA OPA 104 output.

[0024]

[0029] The thickness of the imaging device 101 shown in Fig. 1A may be about 4 mm or more or less, and the height and width of the imaging device shown in Fig. 1B may be about 10 to 24 mm or more or less.

[0025]

[0030] 2 is a block diagram of an exemplary imaging device 201. The imaging device 201 includes additional components relative to the imaging device 101 that may assist in image processing. The imaging device 201 includes a first OPA or OPA receiver 203, a second OPA or FPA OPA 204, an FPA 205, a microlens array 206 that may be associated with the FPA OPA 204, a microlens array 207 that may be associated with the OPA receiver 203, a block 208, and one or more sensors 209. The OPA receiver 203, the FPA OPA 204, and the block or control unit 208 of the imaging device 201 are shown as being included on the PIC 202. However, in some implementations, one or more of these components may not be included on the PIC 202 and / or may be included on a different PIC separate from the PIC 202. In some cases, the microlens arrays 206, 207 and / or one or more sensors may or may not be included on the PIC 202, or may or may not be included on a different PIC separate from the PIC 202.

[0026]

[0031] In some implementations, microlens array 206 can be a lens, a metalens, a microlens array, or any combination thereof. Additionally or alternatively, microlens array 207 can be a lens, a metalens, a microlens array, or any combination thereof.

[0027]

[0032] The OPA receiver 203 may be configured to receive light from an environment of the imaging device. This environment is the subject of the captured image. The OPA receiver 203 includes one or more reflector, partial reflector, or retroreflector photonics structures 210, multiple emitters 211, multiple phase and amplitude modulators 212, and multiple photodiodes (PDs) 213. The components of the OPA receiver 203 may be connected via multiple waveguides or optical fibers. Additionally, a microlens array 207 may be associated with the OPA receiver 203. The microlens array 207 may be a microlens layer adjacent to an emitter layer including multiple emitters 211. In this regard, each emitter in the emitter layer may correspond to a microlens in the microlens layer. In some cases, the multiple emitters 211 may be multiple optical antennas. In some cases, the microlens array 207 may be a single lens, such as a metalens or photonics structure that emulates the functionality of a microlens array. One or more reflectors 210 may correspond to an emitter from the plurality of emitters 211 .

[0028]

[0033] The multiple phase and amplitude modulators 212 may be formed in one or more modulator layers and configured to modify the phase and / or amplitude of the received (Rx) light. In this regard, the multiple phase and amplitude modulators 212 may enable control of Rx light characteristics. In some cases, each layer may be a layer of one or more phase modulators or one or more amplitude modulators. In some cases, the one or more modulator layers may include one or more multiplexers or demultiplexers. In some cases, the one or more modulator layers may include multiple PDs 213. The multiple PDs 213 may be coupled to each phase and amplitude modulator in a phase and amplitude modulator layer. In some cases, a single PD may be coupled to two or more phase and amplitude modulators in a layer via a waveguide tap coupler. The multiple PDs 213 may be configured to measure one or more values, such as the intensity, power, and / or relative phase or wavefront, of the injected control light wavelength. The measurements can then be used to control and / or analyze the Rx light (eg, as feedback).

[0029]

[0034] The multiple emitters 211 or optical antennas may be configured to transmit and receive light. In this regard, the multiple emitters may be configured to receive incident light (e.g., Rx light). In addition, the multiple emitters may be configured to transmit one or more control wavelengths. The one or more control wavelengths may be reflected back to the multiple emitters 211 via one or more reflectors 210. In this regard, the reflected back control wavelengths may be used for phase and wavefront control of the Rx light.

[0030]

[0035] The FPA OPA 204 may be configured to receive light from the OPA receiver 203. Like the OPA receiver 203, the FPA OPA includes one or more reflector, partial reflector, or retroreflector photonics structures 214, multiple emitters 215, multiple phase and amplitude modulators 216, and multiple PDs 217. The components of the FPA OPA 204 may be connected via multiple waveguides or optical fibers. Additionally, a microlens array 206 may be associated with the FPA OPA 204. The microlens array 206 may be a microlens layer adjacent to an emitter layer including multiple emitters 215. In this regard, each emitter in the emitter layer may correspond to a microlens in the microlens layer. In some cases, the multiple emitters 215 may be multiple optical antennas. In some cases, the microlens array 206 may be a single lens, such as a metalens or a photonics structure that emulates the functionality of a microlens array. One or more reflectors 214 may correspond to an emitter of the plurality of emitters 215 .

[0031]

[0036] The multiple phase and amplitude modulators 216 may be formed in one or more modulator layers and configured to modify the phase and / or amplitude of the receive (Rx) light. In this regard, the multiple phase and amplitude modulators 216 may enable control of Rx light characteristics. In some cases, each layer may be a layer of one or more phase modulators or one or more amplitude modulators. In some cases, the one or more modulator layers may include one or more multiplexers or demultiplexers. In some cases, the one or more modulator layers may include multiple PDs 217. Multiple PDs 217 may be coupled to each phase and amplitude modulator in a phase and amplitude modulator layer. In some cases, a single PD may be coupled to two or more phase and amplitude modulators in a layer via a waveguide tap coupler. The multiple PDs 217 may be configured to measure one or more values, such as the intensity, power, and / or relative phase of the control wavelength. The measurements may then be used to control and / or analyze the Rx light (e.g., as feedback).

[0032]

[0037] The multiple emitters 215 or optical antennas may be configured to transmit and receive light. In this regard, the multiple emitters may be configured to receive incident light (e.g., Rx light). In addition, the multiple emitters may be configured to transmit light to the FPA 205. In addition, the multiple emitters may be configured to transmit one or more control wavelengths. The one or more control wavelengths may be reflected back to the multiple emitters 215 via one or more reflectors 214. In this regard, the reflected back control wavelengths may be used for phase and wavefront control of the Rx light.

[0033]

[0038] The FPA 205 may be an image sensor of the imaging device 201. In this regard, the FPA 205 may be configured to receive light from the FPA OPA 204 and record an image based on the received light. The recorded image may be stored on a memory (e.g., memory 220) of the imaging device 201. In some cases, the PIC 202 or the FPA 205 may additionally include an amplitude modulator array. The amplitude modulator array may be used to adjust the amount of light directed to the elements of the FPA 205 on an element-by-element basis. This adjustment may be utilized for, for example, brightness adjustment, glint removal, aperture adjustment, and other amplitude-dependent operations.

[0034]

[0039] The sensor(s) 209 may be configured to collect one or more sensor measurements. The one or more sensors may include various types of sensors, such as, for example, gyroscopes, accelerometers, inertial measurement units (IMUs), etc. The one or more sensors 209 may be configured to collect the one or more sensor measurements at multiple time steps. The one or more measurements may be stored in a memory (e.g., memory 220) of the imaging device 201.

[0035]

[0040] The block 208 includes one or more processors 218 (e.g., CMOS processors), one or more light sources 219, and memory 220. The one or more light sources 219 may be light-emitting diodes (LEDs), PIC-integrated lasers, externally coupled lasers, etc. The one or more processors 218 may be CMOS processors. The one or more processors may be or include any conventional processor, such as a commercially available CPU. Additionally or alternatively, the one or more processors may be or include a dedicated device, such as an application-specific integrated circuit (ASIC), or another hardware-based processor, such as a field-programmable gate array (FPGA). While FIG. 2 functionally illustrates the processor 218 and memory 220 within one or more blocks 208, the one or more processors 218 and memory 220 may actually include multiple processors and memories that may or may not be located on the PIC 202. Thus, reference to a processor or computer should be understood to include reference to a collection of processors, computers, or memories that may or may not operate in parallel.

[0036]

[0041] The memory 220 may store information accessible by one or more processors 218, including data and instructions that may be executed by the one or more processors 218. The memory may be any type of memory capable of storing information accessible by a processor, including computer-readable media such as hard drives, memory cards, ROM, RAM, DVDs or other optical disks, and other writable and read-only memory. The present systems and methods may include various combinations of the foregoing, whereby different portions of the data and instructions are stored on different types of media.

[0037]

[0042] Data may be retrieved, stored, or modified by one or more processors 218 in accordance with instructions. For example, although the present systems and methods are not limited by any particular data structure, data may be stored in computer registers in a relational database as a table with multiple different fields and records, an XML document, or a flat file. Data may also be formatted in any computer-readable format, such as, but not limited to, binary values ​​or Unicode. By way of further example only, image data may be stored as a bitmap including a grid of pixels, compressed or decompressed, stored according to lossless (e.g., BMP) or lossy (e.g., JPEG) and bitmap or vector-based (e.g., SVG) formats, and computer instructions for drawing graphics. Data may include any information sufficient to identify associated information, such as numbers, descriptive text, proprietary codes, references to data stored in other areas of the same or different memory (including other network locations), or information used by a function to calculate the associated data.

[0038]

[0043] The instructions may be any set of instructions (such as machine code) or instructions (such as a script) that are executed directly or indirectly by one or more processors 218. For example, the instructions may be stored as computer code on a computer-readable medium. In this regard, the terms "instructions" and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by one or more processors 218 or in any other computer language, including a script or collection of independent source code modules that are interpreted or pre-compiled as needed.

[0039]

[0044] In some cases, one or more processors 218 may be operably connected to at least one of the PDs and the plurality of PDs 213 of the OPA 203, the plurality of PDs 217 of the FPA OPA 204, or at least one layer of phase and amplitude modulators in both. In this regard, the one or more processors 218 may be configured to drive (e.g., modify the output phase and / or output amplitude) the plurality of phase and amplitude modulators 212 of the OPA 203, the plurality of phase and amplitude modulators 216 of the FPA OPA 204, or at least one layer of phase and amplitude modulators in both. Additionally, the one or more processors 218 may be configured to utilize values ​​measured by the PDs to drive the at least one layer of phase and amplitude modulators in the OPA 203, the FPA OPA 204, or both.

[0040]

[0045] The one or more processors 218 may additionally be configured to generate one or more control wavelengths via one or more light sources 219 (e.g., light-emitting diodes (LEDs), PIC-integrated lasers, externally coupled lasers, etc.). The one or more processors 218 may further be configured to apply the one or more control wavelengths to an optical waveguide co-propagating with the Rx light within the OPA 203, the FPA OPA 204, or both. In this regard, the one or more processors 218 may drive the multiple phase and amplitude modulators 212 and 216 to apply the one or more control wavelengths to facilitate functional processing of the Rx light. In this regard, the one or more processors 218 of the imaging device 201 may record or capture an image of the processed light. The imaging device 201 may record an image with the FPA 205 of the imaging device 201. The recorded image may be stored on a memory 220 of the imaging device 201.

[0041]

[0046] For example, the control wavelength may be within a particular color band (e.g., a "blue" band, a "green" band, a "red" band, an "infrared" band, etc.) to closely match the optical propagation characteristics of the Rx spectral band. In such an example, if the control wavelength is within the red band, the Rx light red band will exhibit the same functional performance as the associated red control wavelength. The band may be decomposed into one or more subbands to improve this correlation. The control wavelength may be spectrally multiplexed / demultiplexed around the Rx light band, temporally multiplexed with higher peak power, and / or coherently mixed with a local oscillator in the PD to provide higher signal contrast for the Rx light. This may enable one or more processors 218 of the imaging device 201 to record or capture an image based on the processed light. The recorded image may be stored on the memory 220 of the imaging device 201.

[0042]

[0047] Additionally or alternatively, feedback on the Rx light using one or more control wavelengths may be generated based on one or more sensor (e.g., gyroscope, accelerometer, inertial measurement unit (IMU), etc.) measurements from one or more sensors 209. In some cases, feedback based on the one or more sensor measurements may be updated during each time step (e.g., a time corresponding to receipt of one or more measurements). In this regard, the one or more control wavelengths may process the Rx light to cancel or compensate for noise or movement detected by the one or more sensors 209. In some cases, feedback on the Rx light using one or more control wavelengths may be updated based on the one or more sensor measurements during each time step (e.g., a time corresponding to receipt of one or more measurements).

[0043]

[0048] Additionally or alternatively, one or more control wavelengths may be used to view the Rx light within a portion of a field of view (FOV) on the imaging device. In this regard, the one or more control wavelengths may process the Rx light such that only the Rx light within a particular portion of the FOV is routed through the imaging device and captured as an image. Thus, a portion of the entire FOV or region of interest (FOR) may be captured without having to physically move the imaging device 201.

[0044]

[0049] Additionally or alternatively, one or more control wavelengths can be used to set an image plane. The image plane can be at a specific distance from the imaging device. In this regard, feedback on the Rx light using one or more control wavelengths can manipulate the Rx light so that light at a specific image plane is focused on the imaging device. A series of these images at various planes can be stored in rapid temporal succession to form an image hypercube from which an image plane of interest can be selected or a three-dimensional image can be rendered.

[0045]

[0050] Additionally or alternatively, the one or more control wavelengths can be used as feedback for a particular intensity or can be generated based on a target pattern. The target pattern can include an example image with particular intensity measurements that are compared or correlated with the Rx light. In this regard, the one or more control wavelengths can process the Rx light such that glint or high intensity values ​​cannot persist during the process.

[0046]

[0051] Additionally, the one or more control wavelengths and functionality enabled thereby may be determined based on one or more inputs from a user. The one or more inputs may be received at a user interface (not shown). In this regard, the user may select a mode or feature that determines which control wavelengths and how corresponding processing is performed on the incident light. In some cases, multiple control wavelengths may be repeatedly utilized to process the Rx light according to input from the user. For example, if the user selects a full-color image or an image containing all colors in the visible light spectrum, a respective control wavelength corresponding to each band of visible light may be used to repeatedly process the Rx light. In this regard, the one or more processors 218 of the imaging device 201 may record or collect 1 to N images, each of which is in a different visible light band. The recorded images may be stored in the memory 220 of the imaging device 201. The one or more processors 218 of the imaging device may be further configured to generate a full-color image using the 1 to N stored images.

[0047]

[0052] In some cases, the control wavelength phase associated with the Rx band or the wavelength itself may be dithered. In this regard, the control wavelength may be dithered according to an orthogonal set of functions (e.g., orthogonal basis functions). The dithering may be time-division dithering, frequency-division dithering, or some combination thereof. The dither may be applied to phase and amplitude modulators of various subsets of the plurality of phase and amplitude modulators 212 and 216. Each subset of phase and amplitude modulators may correspond to a function from an orthonormal set of functions. In some implementations, the orthonormal set of functions may be Walsh functions.

[0048]

[0053] With time-division dithering, each dither may be applied sequentially. In this regard, the dither may be applied to a different subset of the phase and amplitude modulators of each successive signal. The dither frequency of each dither may be selected from a predetermined set of frequencies. The frequencies of each dither may or may not be the same.

[0049]

[0054] With frequency-division dithering, each dither may be applied simultaneously. In this regard, dither may be applied to a different subset of the phase and amplitude modulators for each signal. The dither frequency for each signal may be selected from a predetermined set of frequencies. In some cases, each of the frequencies in the predetermined set of frequencies may be unique. In such cases, the frequencies in the predetermined set of frequencies may be selected so that they do not interfere with each other. Additionally or alternatively, in some cases, each of the multiple predetermined frequencies may not be unique. In such cases, multiple disturbances that utilize the same or potentially interfering frequencies may be selected so that their frequencies do not interfere. For example, if two disturbances utilize the same frequency, one disturbance may be utilized via a sine function and the other via a cosine function, with one of these functions being shifted by π / 2 so that the disturbances are orthogonal and / or out of phase.

[0050]

[0055] FIG. 3 illustrates features of an OPA architecture depicted as an exemplary OPA architecture 300. The OPA architecture 300 may represent components of the OPA receiver 103, 203, the FPA OPA 104, 204, and their associated components. The OPA architecture 300 includes a representation of a microlens array 310, multiple emitters or optical antennas 320, and multiple phase and amplitude modulators 330. For clarity and ease of understanding, additional waveguides and other features are not depicted. Arrows 340 and 342 represent the general directions in which Tx light (e.g., control wavelength, light to the FPA, etc.) and Rx light (e.g., light from the environment, reflected back control wavelength, etc.) travel through or through the OPA architecture 300.

[0051]

[0056] The microlens array 310 may include a plurality of convex microlenses 311-315 that focus Rx light onto each of a plurality of emitters positioned at the focal point of the microlens array 310. In this regard, dashed line 350 represents the focal plane of the microlenses 311-315 of the array 310. The microlens array 310 may be arranged in a grid pattern with a consistent pitch or distance between adjacent lenses. In other examples, the microlens array 310 may have a variety of arrangements with different numbers of rows and columns of different lenses, different shapes, and / or different pitches (consistent or inconsistent).

[0052]

[0057] Each microlens of the microlens array can be several micrometers, tens of micrometers, or hundreds of micrometers in diameter and / or height. Additionally, each microlens of the microlens array can be fabricated by molding, printing, or etching the lenses directly into the wafer of the OPA architecture 300. Alternatively, the microlens array 310 can be formed as a separately fabricated microlens array. In this example, the microlens array 310 can be a rectangular or square plate of glass or silica several millimeters in length and width (e.g., around 10 mm) and around 0.2 mm in thickness. Integrating the microlens array into the OPA architecture 300 can enable a reduction in grating emitter size and an increase in the spacing between emitters. In this way, two-dimensional waveguide routing within the OPA architecture can be better suited to a single-layer optical phased array. In other cases, rather than a physical microlens array, the function of the microlens array can be replicated using an array of diffractive optical elements (DOEs).

[0053]

[0058] Each microlens in the microlens array may be associated with a respective emitter of the plurality of emitters 320. For example, each microlens may have an emitter where the Tx signal is received and where the Rx signal is focused. As an example, microlens 311 is associated with emitter 321. Similarly, each microlens 312-315 also has a respective emitter 322-325. In this regard, for a given pitch (i.e., microlens edge length), the microlens focal length may be optimized for best transmit and receive coupling to the underlying emitter. This arrangement may therefore increase the effective fill factor of the Rx light at each emitter while also expanding the Tx light received at the microlens from each emitter before the Tx light leaves the OPA architecture 300.

[0054]

[0059] The multiple emitters 320 can be configured to convert emitted light from a waveguide to free space and vice versa. The emitters can also generate specific phase and intensity profiles to further increase the effective fill factor of the Rx light and improve the wavefront of the Tx light. The phase and intensity profiles can be determined using inverse design or other techniques to account for how the transmitted light changes as it propagates to and through the microlens array. The phase profile can differ from the flat profile of a conventional grating emitter, and the intensity profile can differ from the Gaussian intensity profile of a conventional grating emitter. However, in some implementations, the emitter can be a Gaussian field profile grating emitter.

[0055]

[0060] The phase and amplitude modulator 330 may enable sensing and measuring the Rx light, modifying the Tx light, and combining the input light into a single waveguide or fiber. Each emitter may be associated with a phase and amplitude modulator. As shown in FIG. 3, each emitter may be connected to a respective phase and amplitude modulator. As an example, the emitter 320 may be associated with a phase and amplitude modulator 330. The Rx light received by the phase and amplitude modulators 331-335 may be provided to a receiver component including a sensor, and the Tx light from the phase and amplitude modulators 331-335 may be provided to each emitter of the multiple emitters 320. The architecture of the multiple phase and amplitude modulators 330 may include at least one layer of phase and amplitude modulators with at least one phase and amplitude modulator connected to an emitter of the multiple emitters 320. In some examples, the phase and amplitude modulator architecture may include multiple layers of phase and amplitude modulators, where a phase and amplitude modulator in a first layer may be connected in series with one or more phase and amplitude modulators in a second layer.

[0056]

[0061] 4 illustrates an exemplary optical control architecture (OCA) 400 that may be included within the OPA receiver 103, 203, the FPA OPA 204, or both. The OCA 400 includes a laser source 406, multiple phase and amplitude modulator layers 410, multiple PDs 412, one or more waveguide tap couplers 414, one or more multiplexers or demultiplexers 416, multiple reflector, partial reflector, or retroreflector photonic structures 418, multiple emitters 420, a photonics circulator 422, a PD 424, and one or more processors 430. In some cases, one or more components of the OCA 400 may be formed on a PIC.

[0057]

[0062] Laser source 406 may be used to generate one or more control wavelengths based on instructions from one or more processors 430, as discussed above. The one or more control wavelengths may be propagated along a waveguide of OCA 400, as indicated by arrow 408a.

[0058]

[0063] The one or more phase and amplitude modulator layers 410 include an optically controlled (OC) phase and amplitude modulator layer 410b. The OC phase and amplitude modulator layer 410b includes multiple PDs 412. As discussed above, multiple PDs 412 can be coupled to each phase and amplitude modulator in a phase and amplitude modulator layer. FIG. 4 shows that individual PDs of the multiple PDs 412 can be coupled to two or more phase and amplitude modulators in a layer via waveguide tap couplers, such as waveguide tap coupler 414. The multiple PDs 412 can be configured to measure one or more values, such as intensity, power, and / or relative phase, of one or more control wavelengths. The measurements can be used to control and / or analyze the Rx light (e.g., as feedback). The measurements can be used to control and / or analyze components of the OCA 400.

[0059]

[0064] In some cases, various PDs may be configured to measure values ​​in various waveguides of the phase and amplitude modulator layer to determine relative phase. For example, one arrangement may include one PD measuring the intensity / power of waveguide A, one PD measuring the intensity / power of adjacent waveguide B, and one PD measuring the relative phase between waveguides A and B by mixing the two signals on one PD. Such an arrangement may provide the basic building block of a multi-PD PD measurement system. Additionally, the same or similar arrangement may be repeated for adjacent waveguides (e.g., B and C, C and D, etc.). The measurements taken by the three exemplary PDs on waveguides A-B may be expressed as A, B, C, D, and D, respectively. 2 , B 2 and A 2 +B 2 +2ABcosΦ AB The three measurements can be expressed as Φ AB, i.e., the determination of the relative phase of light or signals propagating through waveguides A and B. Absolute phase measurements are also possible by using a portion of the control wavelength as a local oscillator to coherently interfere with the retroreflector control wavelength propagating back and forth through the PIC with multiple PDs. This may require additional multiple phase modulators in the local oscillator path with associated feedback electronics to phase-lock the two together.

[0060]

[0065] 4 additionally illustrates one or more phase and amplitude modulator layers 410, such as phase and amplitude modulator layer 410b. Phase and amplitude modulator layer 410b includes one or more multiplexers or demultiplexers 416 operatively connected thereto. In some cases, one or more multiplexers or demultiplexers 416 may route received light or signals to one or more of multiple PDs 412 for detection, electronic amplification and buffering, and / or analog-to-digital conversion. Received light or signals (e.g., from the imaging device's environment, from another OPA of the imaging device) are represented by arrow 432. In this regard, received light 432b may be received by multiple emitters 420 and propagated to one or more multiplexers or demultiplexers 416 via the OCA architecture, as indicated by arrow 432b.

[0061]

[0066] The plurality of reflectors 418 may be configured to reflect light or signals transmitted from the plurality of emitters 420, such as one or more control wavelengths represented by arrow 408b, back to the plurality of emitters 420. The one or more reflected back control wavelengths may propagate through the OCA 400 and be directed to the PD 424 via the circulator 422. Measurements of the signals measured by the PD 424 may be processed by one or more processors 430. The measurements may be used to control and / or analyze components of the OCA 400.

[0062]

[0067] The illustrated one or more processors 430 are operatively connected to the multiple phase and amplitude modulator layers 410, the multiple PDs 412, and the PD 424. In this regard, the one or more processors may be configured to drive (e.g., modify the phase and / or amplitude of) at least one layer of phase and amplitude modulators. Additionally, the one or more processors may be configured to utilize values ​​measured by the PDs to drive at least one layer of modulators. In some cases, one or more multiplexers or demultiplexers may route signals to the PDs for detection, electronic amplification and buffering, and analog-to-digital conversion.

[0063]

[0068] For example, the above measurements from the multiple PDs 412 and 424 may be used in controlling (e.g., to drive) the multiple phase and amplitude modulator layers 410. In some cases, the one or more processors 430 may use these measurements to compensate for determined phase errors (e.g., in the OC phase and amplitude modulator layer 410a) and drive the multiple phase and amplitude modulator layers 410 (e.g., layer 410b) to provide specific phases to the multiple emitters 420.

[0064]

[0069] The exemplary imaging devices and exemplary architectures discussed above may be utilized as part of one or more user devices. In this regard, the exemplary imaging devices and exemplary architectures may be operatively connected to and / or incorporated within one or more user devices, such as dedicated camera devices, mobile phones, tablets, laptops, desktop computers, or other computing devices capable of communicating information with the imaging devices. Figure 5 illustrates an exemplary user device 500 that includes an imaging device, such as imaging devices 101 and 201 discussed above. Imaging devices 101 and 201 may include architectures such as those described with reference to Figures 3 and 4.

[0065]

[0070] The user device 500 may include one or more processors, memory, data, and instructions. Memory stores information accessible by one or more processors, including instructions and data that can be executed or otherwise used by the processor. Memory may be any type of memory capable of storing information accessible by a processor, including a computing device-readable medium. Memory may be a non-transitory medium, such as a hard drive, memory card, optical disk, solid state, etc. Some systems may include various combinations of the foregoing, whereby various portions of instructions and data are stored on various types of media. Instructions may be any set of instructions that are executed directly by a processor (e.g., machine code) or indirectly (e.g., script). For example, instructions may be stored as computing device code on a computing device-readable medium. In this regard, the terms “instructions,” “module,” and “program” may be used interchangeably herein. Instructions may be stored in object code format for direct processing by a processor or in any other computing device language, including a script or collection of independent source code modules that are interpreted or pre-compiled as needed.

[0066]

[0071] The processor may be any conventional processor, such as a commercially available CPU. Alternatively, each processor may be a dedicated device, such as an ASIC, a graphics processing unit (GPU), a tensor processing unit (TPU), or other hardware-based processor. While FIG. 5 functionally depicts the processor, memory, and other elements of a given computing device as being within the same block, such a device may actually include multiple processors, computing devices, or memories, which may or may not be housed within the same physical enclosure. Similarly, the memory may be a hard drive or other storage medium located in a different enclosure than that of the processor, e.g., in a cloud computing system of server 802. Thus, reference to a processor or memory should be understood to include reference to a collection of processors or memories, which may or may not operate in parallel.

[0067]

[0072] The user device 500 may include all of the components typically used in connection with a computing device, such as the processor and memory described above, as well as a user interface subsystem for receiving input from a user and presenting information (e.g., text, images, and / or other graphical elements via audio and / or haptic feedback) to the user. The user interface subsystem may include one or more display devices operable to display one or more user inputs (e.g., at least one front-facing (user) camera, mouse, keyboard, touchscreen, and / or microphone) and information (e.g., text, images, and / or other graphical elements). Other output devices, such as speakers, may also provide information to the user.

[0068]

[0073] Alternatively, imaging devices such as imaging devices 101 and 201 may be standalone devices capable of communicating with a processor of a user device via a wireless or wired connection. Such user devices may communicate with the imaging device via one or more networks. The one or more networks may include various configurations and protocols, including short-range communication protocols, such as Bluetooth™, Bluetooth LE™, the Internet, the World Wide Web, an intranet, a virtual private network, a wide area network, a local network, a private network using one or more company proprietary communication protocols, Ethernet, WiFi, and HTTP, as well as various combinations of the foregoing. Such communication may be facilitated by any device capable of transmitting data to and from other computing devices, such as a modem and a wireless interface.

[0069] Exemplary Methods

[0074] The imaging devices 101 and 201 discussed above may be used in a method for capturing one or more images using the imaging device. FIG. 6 shows an exemplary method 600 for capturing one or more images using the imaging device. At block 610, the method includes generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs. For example, one or more processors 218 and 430 (e.g., CMOS processors) of the imaging device 201 may be configured to generate one or more control wavelengths 408 via one or more light sources 219 and 406 (e.g., light emitting diodes (LEDs), PIC integrated lasers, externally coupled lasers, etc.). As discussed above, the one or more control wavelengths may be within a particular wavelength band (e.g., a color band), may be based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscopes, accelerometers, IMUs, etc.), may correspond to a portion of the FOV on the imaging devices 101 and 201, may be within a particular focal plane relative to the imaging devices 101 and 201, and / or may correspond to a particular intensity.

[0070]

[0075] In some cases, the one or more control wavelengths may be determined based on one or more inputs from a user. The one or more inputs may be received at a user interface of a user computing device (e.g., user device 500), such as a dedicated camera device, a mobile phone, a tablet, a laptop, a desktop computer, or other computing device capable of communicating information with an imaging device. In this regard, imaging devices 101 and 201 may be incorporated into any of the aforementioned devices. Alternatively, imaging devices 101 and 201 may be standalone devices capable of communicating with a processor of the aforementioned device via a wireless or wired connection. For example, a user may select a mode or feature that determines which control wavelengths and corresponding processing are performed on the incident light. For example, a user may select infrared image capture. In such an example, the one or more control wavelengths may be within the infrared band.

[0071]

[0076] At block 620, the method further includes receiving light from the environment of the imaging device at an optical phased array (OPA). For example, the OPA receivers 103 and 203 of the imaging devices 101 and 201 may be configured to receive light from the environment of the imaging devices 101 and 201. Specifically, multiple emitters or optical antennas 215, 320, 420 of the OPA may be configured to receive light from the environment of the imaging devices 101 and 201.

[0072]

[0077] At block 630, the method further includes driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of an optical phased array (OPA) to apply the one or more control wavelengths to process the received light. For example, the one or more processors 218 and 430 of the imaging device may be configured to apply the one or more control wavelengths to the Rx light by driving the plurality of phase and amplitude modulators 212 and 330.

[0073]

[0078] As discussed above, the control wavelength can be within a particular color band (e.g., blue band, green band, red band, infrared band, etc.) to closely match the optical propagation characteristics of the Rx spectral band. In such an example, if the control wavelength is within the red band, the Rx light can be processed using a control wavelength within the red band so that the Rx light red band exhibits the same functional performance as the associated red control wavelength. These bands can be decomposed into more subbands to improve this correlation. The control wavelength can be spectrally multiplexed / demultiplexed around the Rx light band to provide higher signal contrast for the Rx light, temporally multiplexed with higher peak power, and / or coherently mixed with a local oscillator in the PD.

[0074]

[0079] Additionally or alternatively, feedback on the Rx light using one or more control wavelengths may be based on one or more sensor measurements from one or more sensors 209 (e.g., gyroscopes, accelerometers, inertial measurement units (IMUs), etc.). In some cases, feedback based on the one or more sensor measurements may be updated during each time step (e.g., a time corresponding to receipt of one or more measurements). In this regard, the one or more control wavelengths may process the Rx light to cancel or compensate for noise or movement detected by the one or more sensors 209. In some cases, feedback on the Rx light using one or more control wavelengths may be updated based on one or more sensor measurements during each time step (e.g., a time corresponding to receipt of one or more measurements).

[0075]

[0080] Additionally or alternatively, feedback on the Rx light using one or more control wavelengths may be generated based on one or more sensor (e.g., gyroscope, accelerometer, inertial measurement unit (IMU), etc.) measurements from one or more sensors 209. In some cases, feedback based on the one or more sensor measurements may be updated during each time step (e.g., a time corresponding to receipt of one or more measurements). In this regard, the one or more control wavelengths may process the Rx light to cancel or compensate for noise or movement detected by the one or more sensors 209. In some cases, feedback on the Rx light using one or more control wavelengths may be updated based on the one or more sensor measurements during each time step (e.g., a time corresponding to receipt of one or more measurements).

[0076]

[0081] Additionally or alternatively, one or more control wavelengths may be used to view the Rx light within a portion of the FOV on the imaging device. In this regard, the one or more control wavelengths may process the Rx light such that only the Rx light within a particular portion of the FOV is routed through the imaging device and captured as an image. Thus, the entire FOV or a portion of the FOV may be captured without having to physically move the imaging device 201.

[0077]

[0082] Additionally or alternatively, one or more control wavelengths can be used to set an image plane. The image plane can be at a specific distance from the imaging device. In this regard, feedback on the Rx light using one or more control wavelengths can manipulate the Rx light so that light at a specific image plane is focused on the imaging device. A series of these images at various planes can be stored in rapid temporal succession to form an image hypercube from which an image plane of interest can be selected or a three-dimensional image can be rendered.

[0078]

[0083] Additionally or alternatively, the one or more control wavelengths can be used as feedback for a particular intensity or can be generated based on a target pattern. The target pattern can include an example image with particular intensity measurements that are compared or correlated with the Rx light. In this regard, the one or more control wavelengths can process the Rx light such that glint or high intensity values ​​cannot persist during the process.

[0079]

[0084] As discussed above, the control wavelength(s) and functionality(ies) enabled thereby may be determined based on one or more inputs from a user. The input(s) may be received at a user interface. In this regard, the user may select a mode or feature that determines which control wavelengths are generated and, by extension, how the processing of received light is performed.

[0080]

[0085] In some cases, driving the multiple phase and amplitude modulators may be based on one or more measurements from one or more PDs. In this regard, the multiple phase and amplitude modulators 212 and 330 of the OPA receivers 103 and 203 may be driven based on feedback related to measurements associated with the propagating one or more control wavelengths 408. In some cases, the feedback may additionally be related to measurements associated with the received light 432. The measurements from the multiple PDs 213, 412 and 424 may be used in controlling (e.g., driving) the multiple phase and amplitude modulators 212, 330 and their layer 410. In some cases, the one or more processors 218 and 430 may use the measurements to drive the multiple phase and amplitude modulator layer 410 as well as compensate for determined phase errors.

[0081]

[0086] At block 640, the method further includes capturing, by one or more processors of the imaging devices, an image based on the processed received light. For example, the one or more processors 218 and 430 may capture or record an image of the processed light using the FPAs 105 and 205 of the imaging devices 101 and 201. In some cases, the captured image may be stored in the memory 220 of the imaging devices 101 and 201.

[0082]

[0087] In some cases, the method may further include transmitting one or more control wavelengths by the OPA or its emitters, reflecting the one or more control wavelengths back by the OPA or its one or more reflectors, and receiving the one or more control wavelengths by the OPA or its emitters. In this regard, the multiple emitters or antennas 211, 320, 420 of the OPA receivers 103 and 203 may be configured to transmit one or more control wavelengths 408. The one or more control wavelengths 408b may be reflected back to the multiple emitters 211, 320, 420 via the one or more reflectors 210 and 418. In this regard, the reflected back control wavelengths may be used for phase and wavefront control of the Rx light, as discussed above.

[0083]

[0088] In some cases, the method may further include transmitting the processed received light by the OPA to a second OPA and transmitting the processed received light by the second OPA to an FPA of the imaging device. In this regard, the OPA receivers 103 and 203 may transmit the processed received light to the FPAs OPA104 and 204. The processed received light may be received by emitters 215, 320, and 420 of the FPAs OPA104 and 204. In addition, the processed received light may be further transmitted by the FPAs OPA104 and 204 to the FPA105 and 205 by their emitters 215, 320, and 420. In this regard, the FPAs 105 and 205 may receive light from the FPAs OPA104 and 204 and record an image based on the received light. The recorded image may be stored in a memory (e.g., memory 220) of the imaging device 201.

[0084]

[0089] In some cases, the amplitude modulator arrays of FPA 105 and 205 may be used to adjust, on an element-by-element basis, the amount of light directed to the elements of FPA 205. This adjustment may also be used for, for example, brightness adjustment, glint removal, aperture adjustment, and other amplitude-dependent operations.

[0085]

[0090] In some cases, the method may further include transmitting one or more control wavelengths by the second OPA or its emitters, reflecting the one or more control wavelengths back by the second OPA or its one or more reflectors, and receiving the one or more control wavelengths by the second OPA or its emitters. In this regard, the multiple emitters or antennas 215, 320, 420 of the FPA OPAs 104 and 204 may be configured to transmit one or more control wavelengths 408. The one or more control wavelengths 408b may be reflected back to the multiple emitters 215, 320, 420 via the one or more reflectors 214 and 418. In this regard, the reflected back control wavelengths may be used for phase and wavefront control of the processed Rx light, as discussed above.

[0086]

[0091] In some cases, the method may further include driving, by one or more processors of the imaging device, a plurality of phase and amplitude modulators of the second OPA to apply one or more control wavelengths to further process the processed received light. In this regard, the one or more processors 218 and 430 of the imaging device may be configured to apply one or more control wavelengths to the Rx light at the FPA OPA 104, 204 by driving the plurality of phase and amplitude modulators 216 and 330.

[0087]

[0092] As discussed above, the one or more control wavelengths may be determined based on one or more inputs from a user. The one or more inputs may be received at a user interface. In this regard, the user may select a mode or feature that determines which control wavelengths are generated, and by extension, the processing of the received light.

[0088]

[0093] In some cases, driving the multiple phase and amplitude modulators may be based on one or more measurements from one or more PDs. In this regard, the multiple phase and amplitude modulators 216 and 330 of the FPA OPA 104, 204 may be driven based on feedback related to measurements associated with the propagating one or more control wavelengths 408. In some cases, the feedback may additionally be related to measurements associated with the received light 432. The measurements from the multiple PDs 217, 412 and 424 may be used in controlling (e.g., driving) the multiple phase and amplitude modulators 216, 330 and their layer 410. In some cases, the one or more processors 218 and 430 may use the measurements to compensate for determined phase errors and drive the multiple phase and amplitude modulator layer 410.

[0089]

[0094] In some cases, the OPA receivers 103, 203 and FPA OPAs 104, 204 may be configured to perform various types of Rx light processing, in conjunction with one or more processors 218 and 430 of the imaging devices 101 and 201, as discussed above. In this regard, the one or more processors 218 and 430 of the imaging devices 101 and 201 may be configured to generate one or more control wavelengths via one or more light sources 219 and 406. In one example, the one or more control wavelengths, which may be based on one or more sensor measurements of the sensor 209, may correspond to a portion of the FOV of the imaging device, and / or may be within a particular focal plane, may be transmitted, reflected back, and used for phase and wavefront control of the Rx light at the OPA receivers 103 and 203. In such an example, one or more control wavelengths, which may be within a particular color band and / or may correspond to a particular intensity or may be generated based on a target pattern, may be transmitted, reflected back, and used to process the Rx light in FPA OPA104 and 204.

[0090]

[0095] In some cases, the one or more control wavelengths may be multiple control wavelengths, such as a first control wavelength and a second control wavelength. In such cases, the method steps discussed above may be repeated based on the number of control wavelengths. For example, if a user selects a full-color image or an image including all colors in the visible light spectrum, a control wavelength corresponding to each band of visible light may be used to repeatedly process the Rx light. In this regard, one or more processors of the imaging device may record and collect 1 to N images in successive time steps, each of the 1 to N images being in a different visible light band. The recorded images may be stored in a memory of the imaging device. In such cases, the method may further include generating a composite image by one or more processors of the imaging device. In this regard, the one or more processors of the imaging device may be further configured to generate a full-color image using the 1 to N stored images.

[0091]

[0096] The features and methodologies described herein may provide a scalable imaging device capable of complex image processing using one or more injected or applied control wavelengths and wavefronts to maintain the proper relationship between received light components and the image, resulting in improved image capture capabilities. In this regard, the imaging device may be capable of performing real-time color processing and intensity manipulation, image stabilization to compensate for external vibrations and disturbances, the ability to capture panoramic images without moving the imaging device, and the ability to correct the focal plane of the image in real time (focal plane adjustment).

[0092]

[0097] Unless otherwise stated, the foregoing alternatives are not mutually exclusive and may be implemented in various combinations to realize unique advantages. Because these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the foregoing description of embodiments should be construed as illustrative, rather than limiting, of the subject matter defined by the claims. Additionally, the provision of several examples described herein and clauses expressed as "such as," "including," etc., should not be construed as limiting the subject matter of the claims to any particular examples; rather, the examples are intended to illustrate only some of many possible embodiments. Furthermore, the same reference numbers in different drawings may identify the same or similar elements. [Explanation of symbols]

[0093] 101 Imaging Device 102 Photonics Integrated Circuits 103 OPA receiver 104 Focal Plane Array Optical Phased Array 105 Focal Plane Array 106 Microlens Array 107 Microlens Array 108 Control Unit 201 Imaging Device 202 Photonics Integrated Circuits 203 Optical Phased Array Receiver 204 Focal Plane Array Optical Phased Array 205 Focal Plane Array 206 Microlens Array 207 Microlens Array 208 Control Unit 209 Sensor 210 Reflector 211 Emitter 212 Phase and Amplitude Modulator 213 Photodiode 214 Reflector 215 Emitter 216 Phase and Amplitude Modulator 217 Photodiode 218 processors 219 Light source 220 memory 300 Optical Phased Array Architecture 310~315 Microlens Array 320~325 emitter 330~335 Phase and Amplitude Modulators 340 Arrow 342 Arrow 350 dashed line 400 Optical Control Architecture 406 Laser Source 408 Control Wavelength 408a Control Wavelength 408b control wavelength 410 Phase and Amplitude Modulator Layer 410a Phase and Amplitude Modulator Layer 410b Phase and Amplitude Modulator Layer 412 Photodiode 414 Waveguide Tap Coupler 416 Demultiplexer 418 Reflector 420 Emitter 422 Photonics Cardiovascular 424 Photodiode 430 processor 432 Received Light 432a Received light 432b Received light 500 user devices 610 Block 620 Block 630 Block 640 blocks 802 Server

Claims

1. 1. A method of capturing one or more images using an imaging device, comprising: generating, by one or more processors of the imaging device, one or more control wavelengths based on one or more user inputs; receiving light from an environment of the imaging device in an optical phased array (OPA); measuring, with one or more photodiodes, one or more measurements associated with the one or more control wavelengths; driving, by the one or more processors of the imaging device, a plurality of phase and amplitude modulators of the OPA based on one or more measurements associated with the one or more control wavelengths to apply the one or more control wavelengths to process the received light; capturing, by the one or more processors of the imaging device, an image based on the processed received light; A method comprising:

2. generating, by the one or more processors of the imaging device, the one or more control wavelengths based on the one or more user inputs includes generating a first control wavelength and a second control wavelength; In the OPA, receiving light from the environment of the imaging device includes receiving light at a first time step and a second time step; Driving, by the one or more processors of the imaging device, the plurality of phase and amplitude modulators of the OPA to apply the one or more control wavelengths to process the received light includes driving the plurality of phase and amplitude modulators at a first time step to apply the first control wavelength to process the light received at the first time step, and driving the plurality of phase and amplitude modulators at a second time step to apply the second control wavelength to process the light received at the second time step; and 2. The method of claim 1, wherein capturing an image based on the processed received light by the one or more processors of the imaging device comprises capturing a first image at the first time step based on the processed light received at the first time step, and capturing a second image at the second time step based on the processed light received at the second time step.

3. The method of claim 2 , further comprising generating, by the one or more processors of the imaging device, a composite image of the first image and the second image.

4. transmitting the one or more control wavelengths by the OPA of the imaging device; reflecting the one or more control wavelengths back through the OPA of the imaging device; and receiving, by an OPA of the imaging device, the one or more control wavelengths; The method of claim 1 further comprising:

5. The method of claim 1 , wherein the one or more user inputs are received from a user interface.

6. further comprising measuring, with the one or more photodiodes, one or more second measurements associated with the received light; The method of claim 1 , wherein driving the plurality of phase and amplitude modulators of the OPA is further based on the one or more second measurements associated with the received light.

7. The method of claim 6 , wherein the one or more measurements and the one or more second measurements are at least one of: i) intensity, ii) power, and iii) relative phase.

8. transmitting, by the OPA, the processed received light to a second OPA; transmitting, by the second OPA, the processed received light to a focal plane array (FPA) of the imaging device; The method of claim 1 further comprising:

9. The method of claim 8 , further comprising adjusting the amount of light received at one or more elements of the FPA with an amplitude modulator array of the FPA.

10. 9. The method of claim 8, further comprising applying, by the one or more processors of the imaging device, the one or more control wavelengths to drive a plurality of phase and amplitude modulators of the second OPA to further process the processed received light.

11. The method of claim 8 , wherein the one or more control wavelengths is a plurality of control wavelengths.

12. applying, by the one or more processors of the imaging device, the plurality of control wavelengths to drive the plurality of phase and amplitude modulators of the OPA to process the received light includes applying a first control wavelength of the plurality of control wavelengths; 12. The method of claim 11 , wherein applying, by the one or more processors of the imaging device, the one or more control wavelengths to drive a plurality of phase and amplitude modulators of the second OPA to further process the processed received light comprises applying a second control wavelength of the plurality of control wavelengths, the second control wavelength being different from the first control wavelength.

13. 1. An imaging device configured to capture one or more images, comprising:

1. An optical phased array (OPA) receiver, comprising: a plurality of emitters configured to transmit and receive light; a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device; and one or more photodiodes configured to measure one or more values ​​associated with light propagating through the imaging device; an optical phased array (OPA) receiver including: A focal plane array (FPA) OPA, comprising: a plurality of emitters configured to transmit and receive light; a plurality of phase and amplitude modulators configured to modulate light propagating through the imaging device; and one or more photodiodes configured to measure one or more values ​​associated with light propagating through the imaging device; a focal plane array (FPA) OPA including: A control unit, one or more processors; and one or more light sources configured to generate light; a control unit including: an FPA configured to receive light from the FPA OPA and record an image based on the received light; wherein the OPA receiver, the FPA OPA, and the control unit are disposed on a photonics integrated circuit (PIC).

14. 14. The imaging device of claim 13, wherein the OPA receiver further comprises one or more reflectors associated with an emitter of the plurality of emitters, the one or more reflectors configured to reflect one or more control wavelengths back to the emitter of the plurality of emitters.

15. The imaging device of claim 13 , further comprising one or more microlens arrays associated with at least one of: i) the OPA receiver; and ii) the FPA OPA.

16. 14. The imaging device of claim 13, further comprising one or more sensors configured to collect one or more sensor measurements, the one or more sensors comprising: i) at least one of a gyroscope, an accelerometer, and an inertial measurement unit (IMU).

17. The imaging device of claim 13 arranged in a stacked configuration.

18. the one or more processors generating one or more control wavelengths based on one or more user inputs; i) driving the plurality of phase and amplitude modulators of the OPA receiver to apply the one or more control wavelengths to process received light, and ii) driving at least one of the plurality of phase and amplitude modulators of the FPA OPA to apply the one or more control wavelengths to process received light; capturing an image based on the processed received light; and The imaging device of claim 13 , configured to:

Images